1.10. Auto generation of Class and Sequence diagrams¶
Just want to know how to use this feature? See the How to use diagram auto-creation section below.
As described in the previous section Visual Paradigm (VP) is used to create class and sequence diagrams from the GDA Java source code. To assist with keeping such diagrams used within the documentation up to date with the source code the creation process has been automated.
An example of an auto-generated class diagram is shown below.
This is the auto-generated version of the gda.scan.ScanBase
class diagram, see
section Class Diagrams in
Using Visual Paradigm,
for a non-auto-generated version of the same class diagram.
Note, in this document auto-geneated class and sequence diagrams are drawn with a bold orange border, whereas non-auto-generated diagrams use a blue border.
An example of an auto-generated sequence diagram is shown below.
This is the auto-generated version of the
gda.scan.MultiScan.MultiScan(java.util.List)
sequence diagram, see
section Sequence Diagrams in
Using Visual Paradigm,
for a non-auto-generated version of the same sequence diagram.
To achieve diagram auto creation various tools and capabilities included
in VP have been combined with specifically developed tools. The process
consists of a number of steps that have been combined into the single
GDA Developer Guide make
target create_dia
(see
this section of
Documentation Makefile and Scripts
for details).
The steps that the above target combines are the following:
For class diagrams:
Create UML models from Java source code using VP Instant Reverse and save them in VP
.vpp
project files (equivalent to process described in Importing Java source code into Visual Paradigm). This is done by calling theInstantReverse.sh
script included with VP (see Using Instant Reverse with command).Using the model created in the previous step generate required class diagrams and save them in the
.vpp
project file. Class diagrams are created using the specifically developed VP plug-in GenerateCD (see this section for details) that is executed by calling the VP scriptPlugin.sh
(see Running Visual Paradigm plug-in silently).Diagrams created in previous step are exported from
.vpp
as.png
files, for inclusion in the documentation, by calling VP scriptExportDiagramImage.sh
(see Exporting image with command).
For sequence diagrams, unlike for class diagrams, there is a Visual Paradigm supplied script
InstantReverseSD.sh
that combines the Instant Reverse of step 1. above, with creation of diagrams done in step 2. There is no documentation available in the VP User’s Guide for theInstantReverseSD.sh
script but its arguments and operation are similar toInstantReverse.sh
, described in Using Instant Reverse with command, with the exception that the-path
argument can contain a:
(colon) separated list of paths to src folders containing the Java code to be included in the created VP model.vpp
project file. I.e. theInstantReverseSD.sh
script can create a model using multiple src code directories in a single call to the script; to create the same model usingInstantReverse.sh
would take multiple calls each passing a single src directory to be used in the-path
argument.
All of the above steps for both class and sequence diagram creation are
combined in the Python script vp_create_models_and_diagrams.py
that is called
by the Makefile target create_dia
. For details of how to call this script
directly from the command-line see
this
section of
Documentation Makefile and Scripts
in this document. During processing this script outputs to stdout details of
the actions it is performing including the complete calls to all the VP scripts
used. From this output the user can therefore ascertain how to call these VP
scripts directly from the command-line, e.g. for testing individual steps using
the source code they are developing.
The main aim of providing and using such a process is to remove the necessity to manage and maintain a centralized VP project containing the UML model, generated from the GDA Java source, that includes all class and sequence diagrams used by the documentation.
1.10.1. Auto generated Class Diagrams vs Visual Paradigm application created Class Diagrams¶
The class diagrams created by the diagram auto-generation process are not the same as those created when using Visual Paradigm through its GUI application. The reasons for this are two-fold:
VP is a proprietary closed-source, licensed application, therefore it is not possible to examine the algorithm it uses to determine the structure of the class diagram it produces for a given Java class, i.e. as regards how relationships between classes are represented in UML, neither does the UML specification include such an algorithm. Therefore it is seemingly up to CASE (Computer Aided Software Engineering) tool manufacturers to determine how they will represent a class relationship re-engineered from Java source. Presumably all tools produce the same class diagrams when only rather simple class inheritance is used, however, how each manufacturer deals with nuances of various interface relationships when implemented in Java, e.g. may lead to different UML arrow-head being used by different manufacturers, etc. When developing a tool that automatically generates a class diagram by plugging-in to a UML model generated by an unknown algorithm it is not clear how all relationships attached to a class should be represented, without spending many hours investigating the many perturbations that are possible.
The class diagrams created using the Visual Paradigm GUI application, based on UML models generated by re-engineering (i.e. using VP Instant Reverse) of GDA Java source code (using the process described in section Class Diagrams of this document), often produces diagrams too complex and too large for inclusion in this document (though useful for understanding the source code when able to be accessed inside the VP application).
For the above reasons the class diagrams created during the auto generation process described here are termed simple class diagrams. These often do not contain all the classes that a VP application generated diagram for the same class does. To limit the complexity of the simple diagram the GenerateCD VP plug-in used to generate the diagram (see below) only includes classes that are not in the same package as the diagram’s subject class when that class has a direct relationship with the class, i.e. all classes in the same package as the subject class are included, irrespective of the level of indirection, but no classes are included from outside the package if their only relationship is indirect.
If in the future it is determined the class diagrams produced by this automated process are not sufficiently detailed the GenerateCD VP plug-in can be modified. However, due to its reliance on the VP Open API (see section VP plug-ins below) this is not straight forward due to the current rudimentary nature of its documentation.
1.10.2. Configuring Class and Sequence Diagram auto-creation¶
The diagrams created when make create_dia
calls vp_create_models_and_diagrams.py
are determined from the contents of
defn_models.yaml
(stored in documentation’s GDA_Developer_Guide/scripts
directory). This
YMAL formatted file (accessed using
PyYAML) contains the information required to:
locate the Java source code used to create UML model(s) using VP Instant Reverse (i.e. used to create the
.vpp
file)list the classes (contained in the UML model) used to create the class and/or sequence diagrams
specify directory where generated
.png
files are to be saved
For detailed instructions about the structure and information required please see comments at start of the defn_models.yaml file.
IMPORTANTThe location of the Java source directories, under which the classes to be
included in the model, are defined in each model_<unique_id>
block of the
YMAL file using the model_src
element, e.g.:
model_src:
- gda-core.git/uk.ac.gda.api/src
- gda-core.git/uk.ac.gda.core/src
The order the directories are listed directly effects the model produced, .e.g. the above will not produce the same model as the following:
model_src:
- gda-core.git/uk.ac.gda.core/src
- gda-core.git/uk.ac.gda.api/src
This is due to internal operation of Visual Paradigm and cannot be altered. In
some cases, e.g. creation of sequence diagram of
gda.scan.MultiScan.MultiScan(List)
a change in order will cause the
InstantReverseSD.sh
to return an error.
defn_models.yaml
file.
1.10.3. GenerateCD Visual Paradigm plug-in¶
VP is delivered with a number of scripts that enable the user to execute
some VP operations from the command line, e.g. InstantReverse.sh
,
InstantReverseSD.sh
and ExportDiagramImage.sh
, these are used in the
diagram auto generation process (see above).
For a complete list of included scripts see VP User’s Guide section
Command-line Interface.
In order to automate the generation of all UML diagrams used in this
documentation (using Makefile rules and support scripts) a way of generating
class diagrams from the command-line is required, however, VP does not
include a script to create class diagrams. On closer inspection of a
VP supplied script it can be seen it is a wrapper to a Java
application packaged with VP, e.g. script ExportDiagramImage.sh
executes com.vp.cmd.ExportDiagramImage
. The VP script Plugin.sh
(see
Running Visual Paradigm plug-in silently) provides the ability to execute
arbitrary user supplied functionality in VP provided in the form of a
VP plug-in. Therefore the GenerateCD
plug-in was developed to provide
a way of generating class diagrams from the command line.
1.10.3.1. VP plug-ins¶
VP is a Java application therefore a VP plug-in is implemented as a Java class(es) that interacts with VP through the VP Open API:
“Visual Paradigm Open API is a publicly available application programming interface that provides developers with programmatic access to a Visual Paradigm software platform for extending its functionalities for specific purpose… write your own plug-ins for changing model elements, generating new diagrams or printing out organized model data, etc.”
In addition to the Open API section of VP User’s Guide the following online resources are available to assist with VP plug-in development:
Sadly, during development of the GenerateCD VP plug-in, the contents of the Javadoc were limited, hopefully this will improve.
1.10.3.2. GenerateCD VP plug-in implementation and installation¶
Just want to know how to build and install plug-in? See the How to build and install plug-in section below.
The generate class diagram plug-in is implemented in
gda-documentation/GDA_Developer_Guide/vp/plugins/GenerateCD/src/com/osl/vp/plugin/GenerateCD.java
,
which creates the com.osl.vp.plugin.GenerateCD
class (see
GenerateCD
Javadoc). To enable use of this plug-in by VP its class file and plugin.xml
file (describing the plug-in) must be installed into the user’s VP
configuration directory, default Linux location for this is
~/.config/VisualParadigm
. For VP install instructions see
Installing VP section of this
document, for installation of the plug-in see below.
VP version 16.2 (July 2020) runs using Java version 11, the exact JVM
it uses can be discovered from the vp.log
(default location
~/.config/VisualParadigm
) from message produced when the Visual Paradigm (GUI)
application is started. (Version 16.1 used Java version 9.) A VP plug-in
must be built with a version of Java compatible with that used by the VP
application in order for it to successfully run. With VP version 16.2 Java
versions 8, 9 and 11 have been used to build a GenerateCD plug-in
that successfully runs in VP.
1.10.3.2.1. How to build and install plug-in¶
The plug-in is built and installed using the GDA Developer’s Guide Makefile
target vp_plugins
, e.g. in gda-documentation/GDA_Developer_Guide
directory
use:
make vp_plugins
Access to a Java version compatible to VP version in use (see above), is required for the make to complete successfully.
The vp_plugins
target uses 2 variables:
VP_INSTALL_DIR
- containing the VP install location, this is required to access VP .jar containing the VP Open API implementation. The variable’s default value is:/dls_sw/apps/VisualParadigm/recent
See Installing VP for information about how VP will have been installed.
VP_USER_DIR
- containing the user’s VP configuration location (on Linux this is usually~/.config/VisualParadigm
), this is required as user plug-ins are installed in this location. The variable’s default value is:~/.config/VisualParadigm
If required one or both of the values of these variables can be changed from their default by supplying a new value using:
make VP_INSTALL_DIR=<location> VP_USER_DIR=<location> vp_plugins
E.g. if using a version of VP installed in user’s home directory
~/local/Visual_Paradigm
the following will build and install the
GenerateCD VP plug-in, when executed in
gda-documentation/GDA_Developer_Guide
:
make VP_INSTALL_DIR=~/local/Visual_Paradigm vp_plugins
As the VP_USER_DIR
variable has not been changed from its default value the
plugin will be installed into ~/.config/VisualParadigm
.
1.10.3.2.1.1. Javadoc¶
To build the plug-in’s Javadoc
GenerateCD,
use the following in the gda-documentation/GDA_Developer_Guide
directory:
make vp_plugins_docs
This uses the single variable VP_INSTALL_DIR
to obtain the VP install location
to be used. The default value is the same as used by make target vp_plugins
(see above), it can also be changed from the command-line, e.g.:
make VP_INSTALL_DIR=~/local/Visual_Paradigm vp_plugins_docs
1.10.3.2.1.2. Clean¶
To clean the products created by vp_plugins
use:
make vp_plugins_clean
This uses the single variable VP_USER_DIR
to obtain the user’s VP
configuration directory location, i.e. where VP plug-ins are installed. The
default value is the same as used by make target vp_plugins
(see above), it can also be changed from the command-line, e.g.:
make VP_USER_DIR=~/.non_default/VisualParadigm vp_plugins_clean
1.10.4. How to use diagram auto-creation¶
1.10.4.1. Quick start guide¶
All make
commands listed in table below must be executed in the
gda-documentation/GDA_Developer_Guide
directory inside an activated
documentation Python virtual environment as detailed
Building the GDA Developer and User Guides at Diamond.
Step
Instruction
More information
1
make vp_plugins
If not previously done build and install GenerateCD VP plug-in, requires access to Java compilier compatible with Java version in use by Visual Paradigm, see GenerateCD VP plug-in implementation and installation above.
2
Check contents of
defn_meta.yaml
See additional information below about how to use non-default location of YAML file providing GDA Java source and Visual Paradigm install information. Default version of file uses environment variable
$WORKSPACE
to provide path to top-level directories; build will fail if this is not defined.3
Detail diagrams to be created in
defn_model.yaml
If not previously done update the information regarding the diagrams to be created, see additional information below and Configuring Class and Sequence Diagram auto-creation above.
4
make create_dia
Must be executed in active Python environment (configured for building of GDA documentation), see additional information below about how to use non-default location of YAML files providing GDA Java source and Visual Paradigm install information, and descriptions of diagrams to be created.
5
Diagrams created
The generated diagrams will be found in directory specified in `diagrams_output_dir` entries of defn_models.yaml.
1.10.4.2. Slower start guide¶
Build and install GenerateCD VP plug-in (see GenerateCD VP plug-in implementation and installation above). This only has to be done on user’s first use of the plug-in and only needs repeating on change to GenerateCD or location of user’s Visual Paradigm configuration directory (default location
~/.config/VisualParadigm
).The file defn_meta.yaml (located in same directory as
defn_models.yaml
) stores various information needed by the auto-creation process:documentation root directory location
Java source code root directory location
location where created VP models are to be stored
VP installation location
This can be updated as required following description in file. Alternatively, to use a different
defn_meta.yaml
file specify this using the variableDEFN_META
in call tomake create_dia
(see step 4. below). The default version of file uses the environment variable$WORKSPACE
to provide path to top-level directories; build will fail if this is not defined. The$WORKSPACE
environment variable is used by the Jenkins nightly build of this documentation and therefore it is important that changes to this file pushed to Gerrit maintain use of this variable.As noted above in Configuring Class and Sequence Diagram auto-creation, the information regarding the diagrams to be created is contained in defn_models.yaml, stored in documentation’s
GDA_Developer_Guide/scripts
directory. If adding new diagram(s) update as necessary following description in file’s header. To use a differentdefn_models.yaml
specify this using the variableDEFN_MODELS
in call tomake create_dia
(see step 4. below).In active Python environment (configured for building of GDA documentation as described in section Building the GDA Developer and User Guides at Diamond of this document):
make create_dia
This make target calls the Python script
vp_create_models_and_diagrams.py
which will determine what to do based upon contents of defn_models.yaml and defn_meta.yaml, stored in documentation’sGDA_Developer_Guide/scripts
directory (see steps 2. and 3. immediately above).To use different YAML definition files one or both of the following variables can be passed in the call to
create_dia
:DEFN_MODELS
- path todefn_models.yaml
file to be usedDEFN_META
- path todefn_meta.yaml
file to be used
E.g. if wishing to use different locations than specified in documentation’s defn_meta.yaml but create the same diagrams then only variable
DEFN_META
needs to be supplied:make DEFN_META=<path_to>/defn_meta.yaml create_dia
For details of how to call
vp_create_models_and_diagrams.py
directly from the command-line see this section of Documentation Makefile and Scripts in this document. The time taken for the above to complete will depend mostly upon the amount of Java source files that are included under directories specified inmodel_src
entries ofdefn_models.yaml
, as this directly relates to how long the Instant Reverse step of the process takes, which is the most time consuming part of generating the diagrams.The generated diagrams will be found in directory specified in
diagrams_output_dir
entries ofdefn_models.yaml
.