The following criteria may be used when evaluating a UML modeling tool. There are a number of products out there that meet these criteria. Since you want the best, take the time to evaluate the products you test against this list. If you have any criteria that you particularly value that are not in this list, please let us know.

Repository Support

For a large project, a repository is necessary for the sharing of component designs between developers. Two or more developers can share components of a model or even collaborate on the development of a single component by defining ownership and sharing rights at the appropriate level.

A repository is generally built on top of a database, which provides data sharing and concurrency control features. By providing locking and read-only access, the repository permits one developer to own the model while allowing others to read the model and its components, as well as to incorporate these components into their own designs. Important: The tool should allow you to pull in only the components that you want from another model, without having to import the entire model.

Another interesting way to build the repository is on top of the source code for a project, using a source-code control system to provide concurrency control. The benefit of this approach is a higher degree of synchronization between the code and the model. Another benefit is the elimination of yet another data source. Don't forget that if you use a database for a repository you must back-up this data store separately and perform a three-way synchronization between the model, the repository and the source-code instead of just a two-way synchronization between the code and the model.

With modeling tools that support a repository, changes to any component should be automatically propagated to any design which imports the component.

Round-Trip Engineering

The ability to both forward and reverse engineer source code (Java, C++, CORBA IDL) is a complex requirement that vendors support with varying degrees of success. The successful combination of these two features, forward and reverse engineering, is defined as round-trip engineering.

Forward engineering is very useful the first time that code is generated from a model. This will save you much of the mundane work of keying in classes, attributes and methods.

Reverse engineering is very useful both to transform code into a model when no model previously existed, as well as to resynchronize a model with the code at the end of an iteration.

During an iterative development cycle, once a model has been updated as part of the iteration, another round of forward engineering should allow code to be refreshed with any new classes, methods or attributes that have been added to the model. This step is less commonly adopted by developers because many tools can hopelessly mangle source code in the process. The problem is that the source code contains much more than the model; tools must be very adept at reconstructing the source code that existed prior to the new round of forward engineering.

At minimum, the modeling tool should successfully support forward engineering the first time and reverse engineering throughout the process. Also, the tool should have no trouble reverse engineering the full Java language. Make sure to verify this feature for your own source code since we have seen well-established tools fail on reverse engineering Java features such as inner classes. You will be stuck commenting out the offending code each time you reverse engineer - very painful indeed.

HTML Documentation

The object modeling tool should provide seamless generation of HTML documentation for an object model and its components. HTML documentation provides a static view of the object model that any developer using the model can refer to quickly in a browser, without having to launch the modeling tool itself. Also, by producing HTML as documentation, the number of required licenses for the modeling tool can be reduced by the number of people that need read-only access to the model information.

The HTML documentation should include a bitmap picture of each of the diagrams in the model and should provide navigation throughout the model through the use of hyperlinks. The amount of time required to generate the HTML should be reasonable. A number of products available today support these features with varying amounts of success. Again, make sure to test this feature yourself. A check mark on a feature chart is no guarantee of success.

Full UML 1.1 Support

While many tools claim full support for UML 1.1, in reality this is a complex requirement and some tools may not live up to advertized claims for full support. At minimum, the diagrams which should be supported are the Use Case, Class, Collaboration, Sequence, Package, and State diagrams.

Pick Lists for Classes and Methods

The modeling tool should provide pick lists in several key interfaces:

Collaboration and Sequence Diagrams - The tool should allow an object to be assigned to a class from a list of the classes in the model. It should allow the messages sent between objects to be chosen from a valid list of methods for the object(class) which is receiving the message.

Class Diagram - The tool should allow the importing of classes from other packages or models based on selection of a class from a list of classes in the package.

The pick list feature contributes significantly to the intuitiveness of the modeling tool and may be considered a must-have feature. The development of sequence and collaboration diagrams is greatly facilitated by being able to quickly select the message you want to send from one object to another.

Data Modeling Integration

The object modeling tool should allow integration with data modeling facilities. There are many ways to provide this functionality. One way is for the UML tool to provide a feature allowing an object model to be transformed into DDL, which is the SQL needed to create tables for classes. Another way is for the UML tool to export metadata to a data modeling tool which can import the metadata and use it as the basis for a data model. An advanced, integrated set of tools should allow the data models and object models to be synchronized after each iteration of the design.

Versioning

The modeling tool should allow versions to be saved so that when subsequent iterations are begun, the previous version is available for restoral or to preserve existing code which relies on that version.

Model Navigation

The modeling tool should provide strong navigational support to allow a developer to navigate through all the diagrams and classes in the model. A directory or pick list of classes sorted by name is one way to allow a designer to jump to the desired class on a diagram.

For large diagrams, the tool should provide ease of navigation when zooming and panning.

The tool should also allow ease of navigating to the source code for a class when round-trip engineering is being used.

Printing Support

The modeling tool should provide accurate renditions of large diagrams to be produced through multi-page printing. Print preview and scaling functionality should be supported to allow ease of fitting the diagram to the desired number of pages. The ability to fit a diagram to a single page is high on this list. Unfortunately, we have found that many tools have difficulty performing this important task in a seamless manner.

Diagram Views

The modeling tool should facilitate customization of the view of a class and its details. For instance, it should be possible to exclude all get/set methods from the diagram since they tend to clutter, rather than clarify a diagram. The full signature of methods should be allowed to be shown or hidden easily, depending on the level of detail desired. The visibility of attributes and methods (private, protected, public) should be another dimension used to select what to show or hide on the diagram.

Exporting Diagrams

One key feature that is often overlooked is the ability to export diagrams into a format that may be imported into either a word processing document or a web page. The most popular graphics formats used for export are GIF, PNG and JPEG. When exporting, the tool should allow you to define the preferred resolution and size of the graphic that is produced. The requirement for this functionality originates from authors who are ambitious and want to write a UML book which includes diagrams, or who wish to display their work on a web site.<wink>

Scripting

Scripting is another powerful feature that should be supported by a modeling tool. With scripting, the power user may create scripts that can directly access the object model within the modeling tool for the purpose of creating other artifacts. Examples of these artifacts are project management spreadsheets for the project under development, customized documentation, customized code, reports, and metrics. An example of customizable code is the signatures for collection classes and the get/set methods used to access these collection classes.

To facilitate scripting, the modeling tool should expose interfaces to the object model internal to itself that provides access to the components of the object model under development. (If this sounds circular, read it again.) For instance, the script writer should be able to access the collection of classes in a class diagram through an iteration, and then should be able to access properties of the classes through accessor methods on the class object. Of course, the scripting language itself should be object-oriented; one obvious choice is the Java language itself, another is the Python scripting language.

Robustness

You want a UML tool with rock-solid reliability, to prevent users from losing potentially hours of productivity when the tool crashes in the middle of a design session. Or corrupts a model which hasn't been backed up. We have personally seen a number of leading tools cause hours of work to be lost due to a crash or file corruption. If you are a developer, you know the feeling of disdain for 'productivity applications' that are less productive than raw coding tools. If you are a manager, you've seen the resentment developers will show when being required to use an unreliable tool.

Today robustness may often be found in applications implemented in Java (JVM run-time protection) or as open-source projects (web-wide, parallel debugging). The quickest way to find out if a particular UML tool is robust is to ask around in a newsgroup such as comp.object. You are sure to get an earful!

Another strategy to apply here, which we recommend that tool vendors adopt, is borrowed from office productivity applications. The strategy is to have the UML tool automatically save a model in the background at periodic intervals.

In the Future...

And now for our wish list for the future. The current maturity of modeling tools indicates that tool vendors are ready to ramp it up to the next level of sophistication. We hope to see some of the following features appear in next-generation products.

Integrated Editor

During iterative development of a model, it is very productive to look at the UML diagram and matching source code in adjacent windows. Products which support this coordination of views could add an extra dimension of power to the model designer's toolbox of options by adding powerful source code editing features directly to the modeling tool. While the modeling tool needn't be the designer's primary editor, it is quite useful to be able to change the name or signature of a method directly in the code and have the change immediately reflected in the model.

Tops on the list of desirable features is keyboard emulation of popular editors such as emacs. Another hot feature is color coding of the code to highlight language keywords, comments, etc. This feature facilitates readability of the code. An essential feature is the ability to jump to the matching line of code when selecting a class, attribute or method on a class diagram. Most of all, the editor should be fast and easy to use.

As a variation on this theme, another solution is to allow the modeling tool to communicate with the developer's favorite editor. As an example, a hot key could allow the modeling tool to switch the active window to the companion editor, with the buffer positioned to the matching line of code.

Auto-generation

One feature that we would really like to see in the near future is the ability for modeling tools to help in the generation of interaction and state diagrams.

This is how it should work: the modeling tool should facilitate the creation of a trace file during the execution of an existing program. The purpose of the trace file would be to capture the interaction between objects as they are passing messages between each other. After the trace file is created, the modeling tool would be used to analyze the trace in order to find the patterns of object interactions. The modeling tool should allow the user to select from a group of classes to analyze. The tool could then present each unique set of interactions that have been recorded by the trace file for these classes, and allow the user to select which interactions to model. Finally, the tool should be able to generate either a sequence or collaboration diagram based on actual, recorded object interactions.

Pretty cool? If this sounds too futuristic it shouldn't, because the tracing technique has already been implemented quite successfully by tools which help developers track down performance bottlenecks in their programs. One nice example of products in this category is JProbe from the KL Group, which is used to analyze the performance of Java programs.

It should also be possible to auto-generate state diagrams using the same technique; one modification to the previously described sequence would be to allow the user to specify the name of a base class for the states in the state machine. The modeling tool would trace interactions between derived classes of the base class. From this trace, the modeling tool could create a state diagram by diagramming each of the recorded state transitions.

Management Tools

If you are the project manager type, then most likely you want to be able to measure how well your O-O project is progressing. A nice feature that should be integrated with modeling tools is the ability to export modeling information into a tool that will allow you to track the progress of both the design and implementation of your project. The spreadsheet is an ideal tool to apply to this solution because of its familiarity and flexibility. Project management tools are also ideal candidates.

How would this feature work? At the high-level, typically what you would like to track are the classes in the model and the people who are assigned to work on them. You want to know when someone started working on the class and what level of completion has been attained. At the next level of detail, you want to know about the methods for each class. At this level you may want to know which methods have been included in interaction diagrams or, during the implementation phase, how much of the code has been completed for each method.

For this feature to be effective, you want 'smart' update of your project management information. Unlike reporting tools, which always generate a new report from scratch, you only want to export everything the first time. After the initial export, your modeling tool should only be required to update your management tool with new information. Depending on the level of control desired by the user, the modeling tool could present the user with a list of updates before exporting.

One of the nice dividends that having a project management link should provide is the ability to target completion dates for the analysis and design phases of a project. The way this would work is by calculating the rate of progress and using this rate to calculate the anticipated completion dates based on the remaining tasks to be done to complete the model.

Metrics

When your project starts to mature, you may want to have access to the metrics for your model. Metrics can give the O-O architect some immediate feedback on the viability of a particular model. Some of the metrics of interest include: the number of super-classes in a class hierarchy, the number of methods per class, the number of attributes per class, the number of get/set methods, the number of methods overridden, the lines of code for each method, the percentage of public, private and protected methods, the degree of coupling per class (the number of different classes that this class knows about), and the percentage of methods commented.

Metrics could be provided through a reporting interface, or, better yet, through a link to a spreadsheet similar to the project management link described before.

XMI: Tying it all together

The XMI standard from the Object Management Group (OMG) is one of the most exciting recent developments in the UML developer community. XMI is an interchange format which has the potential to finally allow seamless sharing of models between best-of-breed development tools. For example, rather than writing scripts within a UML modeling tool to create reports, instead a user could simply export the model under development using XMI and import the model into a specialized report writing tool. In fact, this paradigm would apply equally well to the features discussed earlier: O-O metrics tracking and project management. Moreover, since XMI uses XML to represent model information, a bevy of XML solutions will immediately be available, such as XSL stylesheets for browser-based presentation and XQL query tools for search capability.

The XMI standard is complex and it will take time to shake down the many compatibility issues that will inevitably arise (who said standards are not open to interpretation?) before deployment can become widespread. However, since XMI was developed by industry heavyweights IBM and Unisys, amongst a number of other leading companies, it is anticipated that products will be forthcoming soon. It is up to the user community to drive the requirement for XMI support in UML tools by including it toward the top of feature checklists. For further information on XMI, see the excellent IBM web page.

As an example of how XMI may be put to use, take a look at our project, Transforming XMI to HTML. The project demonstrates how XSL stylesheets can be used to produce an HTML rendition of a UML model.

Reader Criteria

If you have any ideas for new features in UML modelling tools that are not in this list, we'd love to hear from you.

Thank-you to the readers who have email'ed us criteria which they value in UML tools. Take a look at their original suggestions.