Interesting Project Metrics

I’ve been wonder for some about about metrics to evaluate the relative architectural cleanliness of various database implementations. To that end, I wrote a simple program that eat Visual Studio 7 projects files and analyzes the source files. Here are the results:

	Nfs Engine	Vulcan	Firebird 2	MySQL Server
Total Modules	429	633	232	123
Total Lines	63432	227814	126274	214356
Code Modules	206	218	70	99
Header Modules	221	394	162	15
Preprocessed Modules	0	16	0	0
Other Modules	2	5	0	9
Number Functions	2839	4706	1633	4960
Average Arguments	5.00	8.65	13.08	7.58
Average FunctionLines	14.86	32.46	55.95	31.70
Average Code Lines	11.80	21.20	37.12	26.90
Average Internal Comments	0.94	6.10	11.92	2.59
Average Internal WhiteSpace	2.12	5.16	6.92	2.21

 

The analysis program doesn’t try to follow conditional compilation, so everything is included whether active or not.

The Netfrastructure engine is roughly equivalent in functionality to Firebird. The Netfrastructure numbers, however, are for the database engine only, excluding the Java Virtual Machine and template engine. Since the trigger and procedure language in Netfrastructure are Java, this isn’t a strict apples to apples comparison. On the other hand,the Netfrastructure engine includes the remote server, which Vulcan does not.

The Vulcan numbers are taken from the engine provider current code base. A small number of modules that, due to conditional compilation, couldn’t make it through the analysis program were omitted. Post-processed modules were also omitted. Since Vulcan contains quite of bit of archival, disabled Firebird code, its numbers are slightly bloated.

The Firebird number are taken from “engine” msvc7 project. Since Firebird 2 doesn’t use custom development steps, the preprocessed modules aren’t included (the project hasn’t been built, so the corresponding post-processed modules are not included either. I don’t actually know what is the Firebird 2 engine build, but I assume it doesn’t include DSQL and possible other common stuff.

The MySQL numbers are from their Windows source kit. I believe that they also use static libraries for cross component modules, so I suspect this is less than the full server. But it does give a feeling.

I think the two most interesting sets of number are the average number of arguments per function and the average number of code lines per function (code lines exclude comments and white space). It is most interesting that in each case, Vulcan falls halfway between Netfrastructure and Firebird 2. The average number of arguments is a good metric of the quality of a design. Bad (or in this case eroded) designs have to pass everything but the kitchen sink, and sometimes that. The Firebird 2 numbers are particularly scary because many additional parameters are passed covertly through thread data. The average code lines per function is a good metric of modularity — the degree to which common code is cleanly factored out.

The comment related metrics are substantially misleading since they are computed relative to number of functions rather than code lines — fewer code lines will always mean fewer comments. Even so, it is clear that Firebird has something to teach me and MySQL about internal commenting.

Both the ProjectAnalyst and ProjectsSummary projects are checked into the Vulcan tree under src. If you want to play or analyze Firebird 1.0 or 1.5, I’d like to see the results. You may also want to add more metrics. ProjectAnalyst generates xml (sans header) summary files, ProjectsSummary turns a set of xml summary into an HTML table.

--
Jim Starkey
Netfrastructure, Inc.
978 526-1376