Eiffel Home Page (Web) -- Getting started with Eiffel (local) Eiffel Home PageEiffel Home Page

Previous, Up, NextPrevious sectionUpNext section


           Help reading this manual

In earlier sections we saw how EiffelStudio provides extensive documentation on your systems. That information was qualitative. Project managers and developers will also be interested in quantitative information about their projects. You can get such information through the Metrics Tab of the Context Tool, which enables you to perform a number of operations, detailed over the next few pages:

Methodological observations

Although the field of software metrics is a rich one with an abundant literature, its methodological basis is sometimes subject to question. One should resist the tendency to believe numbers just because they are numbers ("lies, damn lies, and metrics").

Software engineers and their managers expect, however, to reap at least some of the benefits that precise quantification has brought to other engineering fields. Such is the purpose of software metrics, defined as quantitative estimates of product and project properties . Object-oriented development, with the rich software structures that it induces, is a particularly amenable to metric analysis. Even when some of the measures do not seem to bring much by themselves, comparing them to those of other projects may reveal significant peculiarities of a system or of some of its parts.

The metrics capabilities of EiffelStudio were designed with these observations in mind. They result from a conservative approach, where no metric is provided without a credible assumption that it reflects some meaningful project or product attribute. For example, you will find a way to define a new metric as a linear combination of existing ones, but not a way to compute arbitrary arithmetic operations, since it isn't clear that -- say -- multiplying two metrics ever makes sense.

Metric terminology

The following terms are used in the presentation of EiffelStudio metric mechanisms.

A metric -- not to be confused with a measure -- is a quantitative property of software products or processes whose possible values are numbers. A measure is the value of a metric for a certain product or process.

For example, we can evaluate the metric "number of classes in the system", called just Classes , by counting the classes in our system. This yields a measure.

We may distinguish between product metrics , which measure properties of the elements being turned out (code, designs, documentation, bug reports...) and process metrics , which measure properties of the process used to turn them out (salaries, expenses, time spent, delays...). The current metric facilities of EiffelStudio are primarily product-oriented but include a process metric: "number of compilations".

Any metric should be relevant: related to some interesting property of the processes or products being measured, such as cost, estimated number of bugs, ease of maintenance... A metric theory is a set of metric definitions accompanied with a set of convincing arguments to show that the metrics are relevant. Neither EiffelStudio nor this manual provide a metric theory.

The numbers yielded by measures are meaningless unless we describe what they refer to. Every metric is relative to a certain unit , specified as part of its definition. For example the unit for a metric that counts classes, such as Classes , is called CLASS .

EiffelStudio provides a set of predefined units. Some simply serve to count occurrences of certain construct specimens in the software; examples include CLASS , CLUSTER , FEATURE , LINE , The metric RATIO describes metrics whose values are divisions, for example "average number of classes per cluster", obtained by dividing the number of classes by the number of classes.

Any metric applies over one or more scope types . A scope type is a type of product or process over which the metric is measured; for product metric, examples include feature (meaning that we will compute a metric over a single feature), class , cluster , system . They obey an order relation corresponding to containment order: a feature belongs to a class, a class to a cluster and so on.

A scope is an instance of a scope type. For example a given cluster is an instance of the scope type cluster .

To compute a measure is to apply a certain metric over a certain scope of an applicable scope type. For example we may compute the value of the metric Classes over a certain system.

Kinds of metrics

The EiffelStudio metric framework provides a number of predefined metrics but also enables users to define their own metrics in terms of the predefined ones according to a taxonomy illustrated as follows:

Metrics are divided into elementary and composite:

Among elementary metrics, we make a further distinction between raw and derived:

The precise definition of selection criterion for a raw metric is: a property with a fixed set of possible values (two or more) characterizing the patterns being counted by the metric. Without the notions of selection criteria and derived metrics, EiffelStudio would need to have predefined (raw) metrics including all possible combinations, such as "deferred and no invariant". This would quickly grow out of hand.

Defining a derived metric

Let us define a new derived metric. In the Context Tool, choose the Metrics Tab. Click the Define new metric button:

In the dialog that comes up, the first tab, selected by default, is the one we want, Derived :

In the Raw Metric entry, currently set to Classes , bring up the menu and select Features . Note the large number of selection criteria applicable to the raw metric Features (number of features), reflecting the many angles under which it is possible to classify features:

Assume you want a metric called Public_functions_full_contracts that counts the number of features that are functions, are exported, and have both a precondition and a postcondition. Enter its name and check the selected criteria:

Click OK to add this new metric to the set of available metrics. It's actually been entered as the default metric to be computed, although we'll compute it only in a short while:

Note that the Unit field indicates the unit automatically assigned to this derived metric, Feature , the same as the unit for the raw metric Features from which you have derived it.

Composite metrics

A composite metrics applies one or more mathematical operators to a set of metrics, themselves either elementary (raw or derived) or already composite. In line with the conservative approach mentioned earlier, EiffelStudio permits only three kinds of mathematical operators:

Let's define a linear metric Direct_dependents that counts heirs (direct descendants) and direct clients of a class. Click Define new metric again and choose the tab Linear . Bring up the Metric menu:

To define the first term of the sum, select Clients . Do not click the button marked OK (we are not done yet as we need a second term) but instead click the + button next to Metric :

The field at the bottom reflects the current state of the linear combination, which only includes the term Clients , with the default coefficient 1. Now bring up the Metric menu again, choose Heirs this time, and click + again. The expression for the sum appears in the bottom field:

The name of our new linear metric will be Direct_dependents . Enter it into the top field, New_metric_name (if you forget, you will be reminded), and click OK . The name of the new metric and its unit become the defaults in the corresponding fields of the Context Tool.

Let's define one more metric, this time a ratio. We want to know the "branching factor": average number of heirs per class. It will be the ratio of metrics Heirs and Classes . Click Define new metric then Metric ratio :

Select Heirs in the Numerator menu on the left, Classes in the Denominator menu on the right; do not check the box "Display as percentage" at the bottom (we prefer to see the branching factor as a number); enter the metric name Average_heirs , more clear than "branching factor", in the field at the top, and click OK . You now have three user-defined metrics available in addition to predefined metrics, as you can check by bringing up the Metric menu on the Context Tool:

Applicable scope types

The reason Average_heirs is grayed out on the last figure is that it's not applicable to the current scope. Each metric indeed has a minimum "applicable scope type"; for example it doesn't make sense to compute the number of attributes in a feature, since the smallest construct in which attributes appear is a class, bigger than a feature. Similarly, the minimum scope type for Classes (the denominator of Average_heirs ) is Cluster , since we can only start counting classes at that level.

Computing measures

Let's now compute a few measures. These will all be relative to the entire system, so the first thing to do is to change the Scope field of the Context Tool (the second field from the left at the top on the preceding figure) to System , the next-to-last entry on the corresponding menu.

The measures that you get should be identical or very close to those shown next. Differences, if any, may result from changes in the EiffelStudio delivery, especially EiffelBase, or operations that you have performed on the Guided Tour system.

First let's use a predefined metric to compute the number of classes in our system. In the Metrics field bring up the menu and choose Classes --> All . The Unit field is updated to the unit of that metric:

For more clarity, change the name of the next measure in the leftmost top field, set by default to Result1 , to Class_count . Now compute that measure by clicking the Calculate metric button, at the place shown on the above figure. This displays the resulting measure in the field just below:

As you see, our system has 183 classes. Refining the measure -- for example by restricting the scope to certain clusters -- would show that all but a handful belong to the EiffelBase library. But you already know this, so don't perform these measures yet (you can do them later).

Saving measures

When you compute a measure, it's not kept anywhere, and the next measure will displace it. To keep the one you just made, click the button labeled Add measure to the list and save it on the last figure:

The operation has had two effects:

Metric information will be kept in the subdirectory Metrics of the project directory. The generated metric_file.xml file contains both the metrics you have defined, and the measures you have performed. This makes it possible to use it both for:

To obtain more measures, apply the three metrics that you defined earlier -- with names Public_functions_full_contracts , Direct_dependents and Average_heirs -- to the entire system and save the corresponding measures. You can use appropriate names for the results or use the EiffelStudio defaults. The result will be as follows:

Operations on metrics

Click the button labeled Manage metrics on the last figure. This brings up the metric management dialog:

You can use it to rearrange the order of the metrics you have defined (by selecting one of them and clicking Up or Down ) and delete any of them. The Formula field shows the definition -- arithmetic, or boolean -- of the selected metric. The Import button will let you, if you have defined metrics in another project, select the corresponding XML file to add them to those of your current project.

Here (possibly after having tried rearranging the metrics) just click Exit .

Metric archives

What does a measure mean? You don't necessarily know in the absolute, but you might want to compare your results to those of other projects. For example, if you have measured the average number of invariants in classes of your system, you might be curious to know how this compares to ISE's EiffelBase library.

The notion of metric archive addresses this need. You may:

At any time in a project, select any measurement archive, local or at a URL, as the reference archive; in that case all the measures you perform will be compared to those of the reference archive. You may select various comparison formats: percentage (the default), difference percentage, plain value.

ISE has established a Web site, http://metrics.eiffel.com , as a publicly available reference for metric collections on ISE's own libraries (EiffelBase, EiffelVision, ...). This provides an invaluable source of comparisons for other projects within and without ISE.

Let's see how this works. We'll compare the number of classes in our system to the number of classes in EiffelVision. Click the last button, Compare to archive , in the Context Tool:

This brings up the archive comparison dialog:

You can Browse to select a local archive (a .xml file) for comparison, but if you have access to the Internet enter the following URL in the File or URL file:

If you do not have access to the Web, Browse instead to the file vision.xml of the documentation directory, which has the same information, although possibly less up to date.

Click the Set button next to Ratio of computed measure to archive to select this format for displaying the comparisons.

Click Close . This updates the display of computed measures as shown by the following figure. All future measures that you perform will be immediately compared to the values in the selected archive if the metric is available on both sides.

The option that we saw in the Compare to archive dialog enables you to define an existing archive as reference for your current and future measurements.

The other two self-explanatory options, shown in the picture of the dialog on the previous page, enable you to create your own archive, and to update an existing one. You can then make this archive available, in a local file or on the Web, for other projects to perform comparisons.

An archive, stored as an XML file, includes both the project metrics and its current measures. The metrics will be made available for import to projects selecting the archive for comparison.

Previous, Up, NextPrevious sectionUpNext section

Eiffel Home Page (Web) -- Getting started with Eiffel (local)