10. GDA Configuration

The GDA configuration is complex. There are multiple aspects of configuration which are described in more detail below.

Configuration is defined for various scopes, a specific installation may require multiple scopes, for example:

• Beamline config

• Group config

• Facility config

All instances of GDA also include some core configuration that is always included.

10.1. Beamline Configuration Service

Configuration in GDA is managed by an instance of BeamlineConfigurationService. This provides a single source of truth for all configuration used in GDA. It is instantiated by the OSGi framework and gathers its configuration from multiple sources.

10.1.1. Configuration Sources

10.1.1.1. Command Line Arguments

Any arguments passed to the GDA application (either server or client) are parsed and values extracted for use in the rest of the configuration. The recognised arguments are

Flag/option	Purpose
--gda-config/-c	Location of the config directory
--profile/-p	Add a spring profile to be enabled when loading spring context
--properties/-k	Add a properties file to be included in the configuration
--spring-xml/-x	Add a spring xml file to be included in the configuration
--logging/-l	Add a logging file to be included in the configuration

Duplicate options of the above will be combined (using either form) so that all values will be available in the configuration, eg to set mulitple profiles you could use

$ ./gda-server -p profile_one --profile profile_two

These arguments add to the configuration of later sources. To instead replace any other sources and use these exclusively, there is a --no-default-<property> option that takes no arguments for each of the above, eg, --no-default-profile replaces the default profiles instead of appending to them.

Any options not listed here will be made available as properties with - replaced with .. Eg --gda-custom-property custom_value will set gda.custom.property to custom_value.

Duplicate options not listed in the table above will be taken with a ‘last value wins’ approach.

This is implemented in uk.ac.diamond.daq.configuration.source.CliOptions

10.1.1.2. Environment Variables

Any environment variable with a GDA_ prefix will be made available as a property in GDA with the name lower-cased and the _ replaced with ., eg having GDA_CUSTOM_PROPERTY=custom_value in the environment when starting GDA will result in gda.custom.property being set to custom_value.

Certain variables are used to provide specific configuration options. The values are assumed to be comma separated lists.

Environment Variable	Purpose
GDA_CONFIG	Location of the config directory
GDA_PROFILES	Add a spring profile to be enabled when loading spring context
GDA_PROPERTIES	Add a properties file to be included in the configuration
GDA_SPRING_XML	Add a spring xml file to be included in the configuration
GDA_LOGGING	Add a logging file to be included in the configuration

These variables add to the configuration of later sources. To instead replace any other sources and use these exclusively, there is an equivalent GDA_NO_DEFAULT_<name> option that takes no arguments for each of the above, eg, GDA_NO_DEFAULT_SPRING_XML replaces the default spring configuration instead of appending to it.

This is implemented in uk.ac.diamond.daq.configuration.source.GdaEnvironment

10.1.1.3. Configuration Directory

The config directory is the main source of configuration. The directory used will be taken from either the command line, environment variables or system properties if defined, and the working directory if not. The files and properties to use are defined in a file named config.toml at the root of the directory.

This file is in the TOML format, and has fields corresponding to each of the core configuration elements (spring xml, properties, logging, profiles etc). For full details of this file format, see the [Config Directory] section.

This file also includes the ability to bring in additional config directories so that common configuration can be shared between multiple instances.

This is implemented by the classes in uk.ac.diamond.daq.configuration.source.directory.

10.1.1.4. Core plugins

There are some configuration elements that are required for all beamlines. To avoid these having to be added as boilerplate everywhere, they can be provided by plugins and included in the build product. These sources can only provide, spring xml files, properties files and logging configuration.

Beans within these core sources are still loaded in the same application context as user provided configuration so references can be used from one to the other.

These sources are provided by implementing uk.ac.diamond.daq.configuration.CoreConfigurationSource, and registering an instance of it as an OSGi service. These services are then gathered by the BeamlineConfigurationService and queried without needing a direct dependency.

The main implementation used by the core plugins is uk.ac.diamond.daq.configuration.PluginConfigurationSource.

10.1.2. Config Directory

The config directory is the main source of configuration for an instance of GDA. The layout is fairly flexible and the locations of files within it are all listed by the config.toml file at the root of the directory. That said, by convention the layout is kept fairly consistent and will generally look something like that listed below.

10.1.2.1. Fields available in config.toml files

10.1.2.1.1. Base fields

• spring-xml - the files to be included in the spring context

• properties - files containing key=value pairs to be loaded as properties

• logging - logging configuration files

• profiles - default spring profiles to be enabled

• system - map of key/value pairs to be set as system properties at startup

• defaults - map of key/value pairs to be set as properties. Once the process has started, these are treated the same as the properties loaded from the properties files. They are included separately so that the values can be used in interpolation for other values in the file without having to have already loaded the properties files. Many of these values are expected to be fallbacks for keys usually passed via the command line or environment.

10.1.2.1.2. Process specific fields

• server - map of base fields for values that should only be applied for server processes.

• client - client equivalent of server field. Map of base fields that should only be applied for client processes.

10.1.2.1.3. Included extra configuration directories

• extras - additional config directories to be included in this configuration. Each directory can be provided as just the path to the directory to be included (in which case the config.toml file from that directory will be used to determine layout), or as a map of all the base fields that will be appended to the default values from the included directory’s config.toml.

Any of these fields can be omitted if they are not required. There are no defaults and if nothing is contributed for a field, an empty list will be used. An empty file is also valid, as is a directory that has no config.toml file (in which case no directory sources are used).

10.1.2.2. Full Layout

The full layout specified by the configuration is therefore something like:

spring-xml = <values_by_mode>
properties = <values_by_mode>
logging = <values_by_mode>
profiles = <values_by_mode>
defaults = {key: value, ...}
system = {key: value, ...}

[server]
spring-xml = <values_by_mode>
properties = <values_by_mode>
logging = <values_by_mode>
profiles = <values_by_mode>
defaults = {key: value, ...}
system = {key: value, ...}

[client]
spring-xml = <values_by_mode>
properties = <values_by_mode>
logging = <values_by_mode>
profiles = <values_by_mode>
defaults = {key: value, ...}
system = {key: value, ...}

[extras.name]
spring-xml = <values_by_mode>
properties = <values_by_mode>
logging = <values_by_mode>
profiles = <values_by_mode>
defaults = {key: value, ...}
system = {key: value, ...}

[extras.name.server]
spring-xml = <values_by_mode>
properties = <values_by_mode>
logging = <values_by_mode>
profiles = <values_by_mode>
defaults = {key: value, ...}
system = {key: value, ...}

[extras.name.client]
spring-xml = <values_by_mode>
properties = <values_by_mode>
logging = <values_by_mode>
profiles = <values_by_mode>
defaults = {key: value, ...}
system = {key: value, ...}

Due to the way TOML is designed, these can sometimes take alternative forms if it makes it clearer, for instance the system section can use the table form instead.

[system]
# While keys do not need to be quoted if they are purely alphanumeric, it is
# recommended that they are for consistency as values with `.`s in are
# meaningful to toml and will need to be quoted.
"key" = "value"
"key2" = "value2"

10.1.2.3. Alternate value forms

Most fields in the layout (anything listed as <values_by_mode> above) are flexible and can either be a single value, a list of values, or specialised by mode as required.

10.1.2.3.1. Single value

This value will be used for all modes

field_name = "single_value"

10.1.2.3.2. Multiple values

These values will all be used for all modes

field_name = ["value_one", "value_two"]

10.1.2.3.3. Mode specific values

Common values will all be used for all modes, whereas the mode map will offer those values only when the relevant mode is enabled. The values for each sub field (common or mode) can also be a single value or a list of values.

field_name.common = "value"
field_name.mode.mode_name = "value_for_mode_name" # single value
field_name.mode.other_mode = ["values", "for", "other_mode"] # multiple values

or alternatively (see toml spec)

[field_name]
common = "value"
[field_name.mode]
mode_name = "value_for_mode_name"
other_mode = ["values", "for", "other_mode"]

10.1.3. Debugging the loaded configuration

The configuration sources being used after all the sources are combined can be examined by setting the gda.config.debug system property to true (via -D argument to command line or system of the config.toml). This will cause the fully resolved configuration to be printed to stdout when it is loaded. It is not enabled by default as for the majority of uses will create identical output and as it is written to stdout instead of logs, it is not easily accessible.

10.2. Typical configuration directory

The config directory for a beamline is the reference point for all of its configuration. The layout of the directory is fairly flexible but by convention looks similar to the following:

config
├── clients
│  ├── _common
│  │  └── client.xml
│  ├── dummy
│  │  └── client.xml
│  └── live
│     └── client.xml
├── component.def
├── config.toml
├── etc
│  └── live
│     ├── p45-gda.json
│     └── p45_profile.sh
├── properties
│  ├── _common
│  │  ├── common_instance_java.properties
│  │  └── logging_customisation.xml
│  ├── dummy
│  │  ├── dummy_instance_java.properties
│  │  └── JCALibrary.properties
│  └── live
│     ├── JCALibrary.properties
│     └── live_instance_java.properties
├── lookupTables/
├── scripts
│  └── localStation.py
├── servers
│  ├── _common
│  │  └── server.xml
│  ├── dummy
│  │  └── server.xml
│  └── live
│     └── server.xml
└── xml
   ├── beamlinestaff.xml
   └── user_permissions.xml

10.2.1. Sub-directories

10.2.1.1. clients

Location for Spring configurations for beans required by the client. It is usually subdivided by mode with a _common directory used by all modes.

10.2.1.2. etc

Location for general beamline configuration not directly used by GDA. This can include items such as DLS launcher configurations which should include different entries for different versions of GDA. This is no longer recommended as there is no longer a fixed path to the currently used deployment of GDA.

10.2.1.3. lookupTables

Default location for lookup tables for the Scannable objects which use them.

10.2.1.4. properties

Property files containing key/value pairs used to configure aspects of the GDA applications. Usually subdivided by mode. This directory usually contains logging customisation files as well as JCA property files. GDA properties files can include additional files via include properties but this is not often required when additional files can be specified in config.toml.

10.2.1.5. scripts

Location of beamline-specific Jython scripts. It should usually contain the GDA startup script file localStation.py (although this is configurable). This directory is not included in the config.toml but should be included in the spring configuration via a ScriptProject instance.

10.2.1.6. servers

Location for Spring configurations for beans required for the server. It is usually subdivided by mode with a _common directory used by all other modes.

10.2.1.7. xml

Location for default user authorisation settings for GDA object access control if enabled.

10.2.2. Example config.toml

These values are not required to match the file below but this is a common example for beamlines that have been converted from the previous standard.

server.spring-xml = [
    "servers/_common/server.xml",
    "servers/${gda.mode}/server.xml"
]
client.spring-xml = [
    "clients/_common/client.xml",
    "clients/${gda.mode}/client.xml"
]
properties = [
    "properties/_common/common_java.properties",
    "properties/${gda.mode}/${gda.mode}_instance_java.properties",
]
[defaults]
"gda.mode" = "dummy"

[system]
# Note this value needs to include `${gda.config}` as values are not treated as
# paths and therefore aren't made relative to the directory
"gov.aps.jca.JCALibrary.properties" = "${gda.config}/properties/${gda.mode}/JCALibrary.properties"

[server.system]
"java.awt.headless" = "true"

# If a beamline has a group config shared between multiple beamlines they
# can be included as additional 'extras'.
[extras]
shared = "relative/path/to/shared-config"

# If additional customisation is required, the expanded table version can be
# used.
[extras.dls]
root = "relative/path/to/dls-config"
server.spring-xml = [
    # Any non-default dls-config spring-xml to include
]
# Other non-default values can be included from here

10.3. Spring configuration

10.3.1. The gda namespace

10.3.1.1. Using the namespace in your XML files

To use elements from the gda namespace, you will need to declare the namespace by adding the following attribute to the <beans> element in your Spring configuration:

xmlns:gda="http://www.diamond.ac.uk/schema/gda/gda"

You will also need to add these entries to the xsi:schemaLocation attribute:

http://www.diamond.ac.uk/schema/gda/gda http://www.diamond.ac.uk/schema/gda/gda/gda-gda-1.2.xsd

Alternatively, if you do not need a specific version you can use

http://www.diamond.ac.uk/schema/gda/gda http://www.diamond.ac.uk/schema/gda/gda/gda.xsd

to use the latest schema. If this is updated in future, the config will not need to be updated.

10.3.1.2. Elements available in the gda namespace

• motor: This creates a ScannableMotor wrapping a mode specific Motor implementation meaning the same bean definition can be used in both live and dummy modes. The classes used in each mode can be configured using gda.spring.device.default.motor.<mode> (where <mode> is replaced by the relevant mode).

• imotor: This is similar to motor but returns the defined bean to the parent instead of injecting it into the context. This means that it can be used within ScannableGroups or similar. In most cases, motor is the better choice.

• monitor: Similar to motor but for monitors. It defaults to the most common live and dummy monitors but the classes used can be configured using gda.spring.device.default.monitor.<mode>.

• proxy: This is intended for use on the client and allows client side beans to reference beans that will be exported from the server rather than be defined locally. The id attribute should match the ID of the bean being exported. It is possible to also add interface elements to register the imported bean as an OSGi service implementing the given interfaces.

• allOfType: An element that will be replaced by a collection of all instances of the specified class defined in the application context. This can be useful when contributions are defined behind spring profiles and may not always be available. This is used in the core config to gather ScriptProjects and MetadataEntries amongst other things.

• relative-path: An element to resolve a path against the location of the file containing it. This is useful when including an external configuration where the included files can’t know their locations ahead of time.

10.3.1.3. Adding the namespace to your XML Catalog in Eclipse

If you use elements from the gda namespace in your XML files, you should add the schema for the gda namespace to the Eclipse XML Catalog, so that Eclipse can validate XML files containing these custom elements. To do this:

• Open the Eclipse preferences (Window → Preferences)

• Go to XML → XML Catalog

• Click “Add…”

• Enter the following details:

  • Location: click “Workspace…” and select uk.ac.gda.core/src/gda/spring/namespaces/gda/gda-gda-1.2.xsd

  • Key Type: choose “Namespace Name”

  • Key: enter http://www.diamond.ac.uk/schema/gda/gda/gda-gda-1.0.xsd

10.3.2. Setting names for Findables

By default all beans that implement Findable will have their names set to match the id of the bean. If this is not required, a name property can be set as normal and will override the id.

NB. Previous versions of GDA required an instance of FindableNameSetterPostProcessor to be defined somewhere in the spring configuration. This is no longer required.

10.3.3. Using GDA properties in Spring

Properties available via LocalProperties or System Properties can be used in Spring bean definitions using the ${gda.property} syntax. This can also include defaults for when the property is not defined, ${gda.property:fallback_value}.

<property name="hostname" value="${gda.images.camerahost}" />

10.3.4. Instantiating EPICS devices

For example:

<bean id="S1_top_motor" class="gda.device.motor.EpicsMotor">
    <property name="pvName" value="BL04J-AL-SLITS-01:Y:PLUS" />
</bean>

10.3.5. Using mode specific devices

To keep the experience when using dummy mode as close to using the real beamline as possible, it is useful to make dummy versions of the beamline’s devices available when running in dummy mode.

There are two common ways of doing this. One is to have different devices for each mode that present similar (if not identical) APIs and have them created in the relevant mode’s config. eg

<!-- in dummy mode config -->
<bean name="device" class="com.example.dummy.DummyDevice">
    <...>
</bean>

<!-- in live mode config -->
<bean name="device" class="com.example.live.LiveDevice">
    <...>
</bean>

only one of these will ever be created so the names will not clash. A common use for this is with Monitors (EpicsMonitor and DummyMonitor).

Another common pattern is create a single device for the users to interact with, but inject a different controller depending on the mode.

<!-- in common config -->
<bean name="device" class="com.example.device.Device">
    <property name="controller" ref="device_controller" />
    <...>
</bean>

<!-- in dummy mode config -->
<bean name="device_controller" class="com.example.dummy.DummyDeviceController">
    <...>
</bean>

<!-- in live mode config -->
<bean name="device_controller" class="com.example.live.LiveDeviceController">
    <...>
</bean>

In this case the interface presented to the user is the same in both cases (even the same class) but the underlying control is delegated to the appropriate mode’s controller. A common use of this style is with motors (ScannableMotor as the common device backed by either EpicsMotor or DummyMotor).

To simplify the creation of motors there is a motor factory, made available via the gda namespace (see above). This creates the common scannable motor and a mode specific controller without needing all three to be specified explicitly.

<gda:motor id="motor" pv="BLXXI:MO:EXAMPLE:X" />

This will create an EpicsMotor in live mode using the specified pv, and a DummyMotor in dummy mode. There is finer control available via further attributes to control which classes are used and to set limits etc. See gda.spring.device.SpringMotorDefinitionParser.

10.3.6. Importing one file into another

<import resource="S1.xml" />

Effectively, the <import> is replaced with the contents of the imported file. All the beans are in the same Spring context and can reference beans from other files.

10.3.7. Please use the ref attribute!!!

Instead of this:

<bean id="s1_bottom" class="gda.device.scannable.ScannableMotor">
    <property name="motorName" value="S1_bottom_motor" />
</bean>

you can do this:

<bean id="s1_bottom" class="gda.device.scannable.ScannableMotor">
    <property name="motor" ref="S1_bottom_motor" />
</bean>

Note the property is motor, not motorName, and this uses the ref attribute

• which plugs the S1_bottom_motor motor into the s1_bottom object (so the ScannableMotor doesn’t need to use the Finder to get the underlying motor - it’s already wired up using Spring).

Since Spring has this dependency injection capability, there’s no need to use the Finder in new classes - Spring can be used to do the wiring.

10.3.8. Property editors

PropertyEditor Javadoc is a standard Java interface concerned with converting text representations of property values into their ‘real’ types (among other things).

In Spring they are used to convert the text values used in Spring configuration files into the type required by the bean being instantiated. Spring has built-in support for many types already, but by putting this in your Spring configuration:

<import resource="classpath:gda/spring/propertyeditors/registration.xml" />

you will also be able to set properties of these types:

• double[][] - 2D double array

• org.apache.commons.math.linear.RealMatrix Commons Math matrix and any other types supported by the PropertyEditors listed in the GdaPropertyEditorRegistrar class.

10.4. Logging

Logging messages can be generated not only by GDA classes, but also by third-party libraries such as Apache Commons Configuration. GDA classes typically use the SLF4J API for logging. Log entries from code that uses Commons Logging or Log4j are redirected into SLF4J using two SLF4J bindings: Commons Logging over SLF4J and Log4j over SLF4J.

GDA uses Logback as the SLF4J implementation, so logging entries are passed from SLF4J to Logback.

There is core logging configuration included by default to set up the logging from the client to be forwarded to the server to be handled consistently and to provide the interface used by the log panel but additional configuration can be provided for each instance. This often includes settings such as limiting the level of specific loggers.

10.4.1. Log Server configuration

The GDA server sets up logging appenders and receivers to accept messages from the client and to write to a central log file. These can be configured using the core properties below (among others).

Property	Description
gda.logs.dir	the directory in which GDA log file, for example gda-server.log, is to be stored
gda.server.logging.port	the port that is opened to allow clients to send logging events
gda.logserver.out.port	the port that is opened to allow log panels to listen to log events

The core server logging configuration can be found in uk.ac.diamond.daq.server/resource/core_config/logging/server_logging.xml.

10.4.2. Log Panel

GDA also provides a simple log viewer utility (available as gda logpanel when using gda command) for viewing the logging messages received from the central Log server.

10.4.3. Default logging configuration

Both GDA servers and clients use the same common configuration to set the level of third party loggers and to set context variables. This configuration can be found in the uk.ac.diamond.daq.configuration plugin in uk.ac.diamond.daq.configuration/resource/core_config/logging/client_and_server_logging.xml.

10.4.4. Adding to logging configuration

Additional instance specific logging configuration can be provided via any of the configuration sources described above. These files should use the included logback element and can include additional appenders and further restrict logger levels.

Including a root element will add the configuration to the root and will not replace it. An example config file could be something like

<?xml version="1.0" encoding="UTF-8"?>
<included>
	<logger name="gda.device.detector.DummyDAServer" level="INFO" />
	<logger name="gda.device.detector.DAServer" level="INFO" />
	<logger name="uk.ac.gda.server.ncd.plotting.ListenerDispatcher" level="INFO" />
	<logger name="org.apache.http" level="INFO" />
	<logger name="gda.scan.MultithreadedScanDataPointPipeline" level="INFO" />
	<logger name="gda.scan.ScanDataPointPopulatorAndPublisher" level="INFO" />
	<root>
		<level value="DEBUG"/>
	</root>
</included>

10.4.5. Using property placeholders in Logback configuration files

You can use properties available to GDA in a Logback configuration file using the ${gda.property} syntax.

10.5. Recording Jython terminal output

The server can be configured to record all text sent to clients’ Jython terminals to a file in the current data/visit directory. To do this a RedirectableFileLogger should be created in Spring and provided an ObservablePathProvider which tracks the data/visit directory. When the file location changes, a note will be left in the last file directing a reader to the new file, and a note will be made in the new file indicating where the log was transferred from. Adapters can then be used on the same observables that a JythonTerminal observes.

For example, using the JythonServerFacade defined in the core configuration,

<bean id="terminal_logger" class="gda.jython.logger.RedirectableFileLogger">
    <constructor-arg ref="terminallog_path_provider" />
</bean>
<bean class="gda.jython.logger.OutputTerminalAdapter">
    <constructor-arg ref="jython_server_facade" />
    <constructor-arg ref="terminal_logger"/>
</bean>
<bean class="gda.jython.logger.InputTerminalAdapter">
    <constructor-arg ref="command_server" />
    <constructor-arg ref="terminal_logger"/>
</bean>
<bean class="gda.jython.logger.ScanDataPointAdapter">
    <constructor-arg ref="jython_server_facade" />
    <constructor-arg ref="terminal_logger"/>
</bean>
<bean class="gda.jython.logger.BatonChangedAdapter">
    <constructor-arg ref="jython_server_facade" />
    <constructor-arg ref="terminal_logger"/>
</bean>

where the terminallog_path_provider bean might be a dummy:

<bean id="terminallog_path_provider" class="gda.data.SimpleObservablePathProvider">
   <property name="path" value="/tmp/gdaterminal.log" />
</bean>

or one that tracks the server’s visit metadata:

<bean id="terminallog_path_provider" class="gda.data.ObservablePathConstructor">
   <property name="template" value="${gda.data.scan.datawriter.datadir}/gdaterminal.log" />
   <property name="gdaMetadata" ref="GDAMetadata" />
</bean>

Note: the InputTerminalAdapter receives the commands typed into all clients’ terminals (via the command server).

10.6. Java Properties and Customising the GDA

Properties to be shared throughout the code base are stored as name-value pairs in the properties files provided by the combined configuration. This is a common way to customise GDA behaviour.

Variable interpolation may be used when defining the java properties in the same manner as Spring or Logback xml files.

GDA property files are read using Commons Configuration (v2). More information about the property file format can be found in the Properties files section of the Commons Configuration User’s Guide.

A property file can include another property file by using the include directive. For more information, see the [Includes][_CommonsConfigurationProperties#Includes] section in the User’s Guide.

10.7. Metadata

10.7.1. ICAT

The ICAT subsystem talks to a database (an RDBMS or an xml file) which maps user id’s to beamlines and experiments. It is used to find which experiments the user who started each client belongs to and so where any data should be written to by scans collected under the command of that client.

If no ICAT is used then data is always written to the same folder as defined by the gda.data.scan.datawriter.datadir java property.

Some examples of different ICAT systems:

To always write to the same data dir:

gda.data                         = /scratch/data
gda.data.scan.datawriter.datadir = ${gda.data}

For no ICAT, but to change the data directory in every different experiment:

gda.data                         = /scratch/data
gda.data.metadata.icat.type      = gda.data.metadata.icat.NullIcat
gda.data.scan.datawriter.datadir = ${gda.data}/$visit$
# change this for every experiment:
gda.defVisit                     = myVisit

To use an ICAT to dynamically change the output directory depending on who is running the GDA client which has the baton (control of the beamline):

gda.data                               = /scratch/data
gda.data.scan.datawriter.datadir       = ${gda.data}/$year$/$visit$
gda.data.metadata.icat.type            = gda.data.metadata.icat.DLSIcat
gda.data.metadata.icat.url             = jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(HOST=${oracle.host})(PROTOCOL=tcp)(PORT=${oracle.port}))(CONNECT_DATA=(SID=xe)))
gda.data.metadata.icat.shift_tolerance = 1440
gda.data.metadata.dlsicat.user         = icatusername
gda.data.metadata.dlsicat.password     = icatpassword

The DLSIcat class provides connectivity to the ICAT database. There is an alternate class in the uk.ac.gda.core plugin called XMLIcat which uses an XML file as a database. This is primarily for use in unit testing or offsite demonstrations, but could also be used by other facilities if they do a database dump into that format.

At Diamond, the scripts in /dls_sw/dasc/bin/iKittenScripts can be used to diagnose problems with ICAT. For example:

• getCurrentVisit will show the ID of the current visit, if there is one.

• getFedIdsForCurrentVisit will show details of users on the current visit.

• getDetailsOfVisit will show information about a particular visit.

10.8. Role-Based Access Control

10.8.1. Concepts

The idea behind the RBAC system is not to provide cast-iron security to the system, but to prevent users from accidentally operating hardware at the wrong time which could damage their experiment. E.g. preventing users from moving optical equipment which have been tuned or hardware shared between different branches.

10.8.1.1. Device protection

Every piece of equipment (class extending Device) can be assigned a protection level (default is 1), and every user assigned an authorisation level (default is 1, but this can be configured to a different value). When certain methods of an object representing a device are called then the user’s authorisation is compared to the protection level; if the authorisation is equal or greater than the protection level then the method is called, otherwise a gda.jython.accesscontrol.AccessDeniedException is thrown.

GDA developers do not have to write any special code in their device classes to implement this access logic. However if there is a method in a class or interface that needs protection (e.g. Motor.moveTo()) then in the Java source code you should annotate that method using the tag:

@!MethodAccessProtected(isProtected=true)

The RBAC system will search through the source code using reflection to find such annotations.

The idea is that certain methods will always check for protection whereas some will always be called no matter what the authorisation of the user (e.g. all users can ask the position of any motor, but may only be able to control certain motors).

10.8.1.2. Client authorisation

When a GDA client is started the user id is either taken from the OS or a different user id and password may be entered. If different details are entered they are authenticated against whatever Authoriser has been configured (the single-sign-on LDAP system at Diamond).

The login information is passed to the Command Server (Jython Server) from the client via the InterfaceProvider class. The client makes a unique registration with the Command Server so the source of each command may be identified. The Command Server then determines the authorisation level of that client by comparing the FedID that the Client uses against an xml database on that beamline. If the FedID is listed in the xml then the stored authorisation level is used, otherwise the client is given the default.

10.8.1.3. The Baton

On beamlines implementing the RBAC system there is the option to use a baton to determine which client has control of the beamline if the beamline may have multiple clients running. Clients holding the baton have control of the beamline subject to their authorisation level, whereas other clients cannot operate the protected methods of any hardware no matter what their authorisation level.

A baton holder may release the baton, so any client can pick it up or it can give the baton to a specific client.

A client may also take the baton from the baton holder if the holder has a lower authorisation level.

There is a GUI panel which lists all the clients on the beamline and has controls for releasing/taking/passing the baton.

10.8.2. Configuration options

10.8.2.1. java properties

To switch on rbac, set the following java property (true by default):

gda.accesscontrol.useAccessControl = true

To switch on baton control (true by default):

gda.accesscontrol.useBatonControl = true

To set the default authorisation level (default is 1):

gda.accesscontrol.defaultAuthorisationLevel = 1

To set the default authorisation level for staff (default is 2):

gda.accesscontrol.defaultStaffAuthorisationLevel = 2

For the first client after the server starts to automatically pick up the baton:

gda.accesscontrol.firstClientTakesBaton = true

To tell the server which class to use for authorisation (This class must implement gda.jython.authoriser.Authoriser and is mainly to distinguish between users and staff):

gda.gui.AcquisitionGUI.authorisationMethod = gda.jython.authoriser.LdapAuthoriser

To tell the server which class to use for authentication (implements gda.jython.authenticator.Authenticator interface. If this is not defined then no login screen will appear and OS authentication will be used - this is recommended):

gda.gui.AcquisitionGUI.authenticationMethod = gda.jython.authenticator.LdapAuthenticator

(for Diamond we now use ldap as the cclrc ldap server can also be used to distinguish staff and user accounts)

10.8.2.2. XML files

A common implementation of Authoriser is the FileAuthoriser - in fact, the provided LDAPAuthoriser implementation uses a FileAuthoriser to provide a local override. FileAuthoriser uses an xml file, which it looks for in ${gda.user.permissions.dir} (or ${gda.config}/xml/ if the first property isn’t defined). Both have the same format, which is:

<user_permissions>
	<user_id>N</user_id>
</user_permissions>

Where user_id identifies a particular user, and N is the numeric permission level. Entries in the file user_permissions.xml indicate what users exist and their device permissions.

To identify specific accounts as staff there should be an xml file placed in the configuration which lists all staff (non-user) accounts which are outside of the ldap server.

This is xml/beamlinestaff.xml and a copy of this is in the example configuration.

The difference between staff and users in the RBAC system is that staff get a different default permission level and if no visit can be found in the ICAT system for that user ID then the default visit is used (defined by a java.property). If a non-staff account has no associated visit in the ICAT system then the UI will not start.

10.8.2.3. Alternative SingleFileAuthoriser

A drawback of the above FileAuthoriser is that it can require the same information to be duplicated in two files if the beamline doesn’t want to use the LdapAuthoriser. To avoid this, a SingleFileAuthoriser can be used instead which uses a single permissions.xml file to determine both the authorisation level of users and whether they are staff or not. The format for this file is:

<?xml version="1.0" encoding="utf-8"?>
<permissions>
  <user fedid="staff_1" level="2" staff="true"/>
  <user fedid="staff_2" level="3" staff="true"/>

  <user fedid="user_1" level="2" />
</permissions>

In this format, only fedid is required and the level/staff attributes are only needed if they differ from the defaults (usually) 1 and false.

10.8.2.4. Setting Device Protection Levels

All devices (that extend Device) support setting a protection level. This can be set via spring using a property

<bean id="device_name" class="gda.device.class">
    <property name="protectionLevel" value="3" />
</bean>

10.8.3. How to setup RBAC on a beamline on Diamond

1. Set the java properties as listed above.

2. Add the beamlinestaff.xml (or permissions.xml) file to your configuration to list which accounts should be treated as staff

3. Restart the GDA !

10.9. Memory configuration

With recent versions of GDA, the default java memory options are becoming insufficient for reliable operation of the RCP GDA client. While the uk.ac.gda.example.product script in the uk.ac.gda.example.feature (part of the gda-core repo) sets the starting and maximum size of the general heap:

-Xms256m
-Xmx1024m

10.9.1. JVM Options

There are two main options which control the behaviour of the JVM garbage collector:

Parameter & Default setting	Purpose
-Xms64m	Default starting size of the jvm heap.
-Xmx1024m	Default maximum size of the jvm heap.

In addition, there are a few options which help to monitor the jvm garbage collector and change it’s behaviour.

Parameter	Purpose
-verbose:gc	Request that the jvm print details of each garbage collection.
-XX:+PrintGCDetails	Causes additional information to be printed.
-XX:+PrintGCTimeStamps	Will add a time stamp at the start of each collection
-XX:+PrintHeapAtGC	Prints even more information about each garbage collection.
-XX:+DisableExplicitGC	Request that the jvm ignore System.gc() calls.
-XX:+ExplicitGCInvokesConcurrent	Use a different GC mechanism for System.gc() calls.

10.9.2. JVM defaults

The current system defaults can be seen using the following command:

$ java -server -XX:+PrintFlagsFinal -version 2>&1 | grep -i -E 'heapsize|permsize'
uintx AdaptivePermSizeWeight               = 20               {product}
uintx ErgoHeapSizeLimit                    = 0                {product}
uintx InitialHeapSize                     := 66328512         {product}
uintx LargePageHeapSizeThreshold           = 134217728        {product}
uintx MaxHeapSize                         := 1063256064       {product}
uintx MaxPermSize                          = 67108864         {pd product}
uintx PermSize                             = 16777216         {pd product}

If we add in the suggested memory configuration options, we can see the effect that these options have:

$ java -server  -Xms256m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=256m \
>      -XX:+PrintFlagsFinal -version 2>&1 | grep -i -E 'heapsize|permsize'
uintx AdaptivePermSizeWeight               = 20               {product}
uintx ErgoHeapSizeLimit                    = 0                {product}
uintx InitialHeapSize                     := 268435456        {product}
uintx LargePageHeapSizeThreshold           = 134217728        {product}
uintx MaxHeapSize                         := 1073741824       {product}
uintx MaxPermSize                         := 268435456        {pd product}
uintx PermSize                            := 134217728        {pd product}

Note: If the above commands show no output, or reduced output, try adding the parameter:

-XX:+UnlockDiagnosticVMOptions

10.9.3. Optimising starting and maximum values

The JVM allows you to specify different starting and maximum values so that you can optimise memory use in your application.

The options -Xms256m -Xmx1024m say:

• Start by allocating 256MB of memory to the heap for this application, but allow the heap to grow up to 1GB if the application requires it.

If the memory required by the application increases above 256MB, then the JVM will request more memory from the operating system and resize the application heap. If the memory required by the application increases above 1GB, then eventually the application will crash with a java.lang.OutOfMemoryError exception.

Unfortunately every time the JVM increases the size of the heap, time is required to manage the expansion. If a GDA client allocates 400MB of memory during the process of starting up, it will start with 256MB, increase it by a small block when it goes above that, increase it a little more when it goes over that new limit and keep increasing the size until it no longer needs to increase the heap any more. Each of these heap expansions will add time to the start up of the GDA client.

For optimal results, it is a good idea to use jvisualvm to profile how much memory the GDA client typically needs, then add a little more for future expansion.

For instance, if a client allocates 400MB just for startup, rises to 480MB after a few basic scans, but rises to 1.6GB during intense use, then -Xms512m -Xmx2g might be a good starting point.

The same applies to Permanent Generation space.

For instance, if jvisualvm shows you that a client uses 60MB when starting up with the scan perspective, rises to 90MB after opening the PyDev scripts perspective and peaks at 150MB during a long session, then the --XX:PermSize=96m -XX:MaxPermSize=192m might be sufficient.

10.9.4. Out of memory exceptions

Most out of memory exceptions are a simple matter of the jvm running out of general heap space, which can often be corrected by increasing the -Xmx value.

When you run out of heap space, the error will be of the form:

java.lang.OutOfMemoryError: Java heap space

Other out of memory exceptions are more obscure though.

In java 6, the garbage collector will generate an out of memory exception if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered. This may indicate that you are creating an excessive number of temporary objects in an already memory constrained jvm. It might be fixed by just giving the jvm more heap space, but it may also be indicative of other problems.

When garbage collection was taking over your application, the error will be of the form:

java.lang.OutOfMemoryError: GC overhead limit exceeded

When you run out of PermGen space, the error may occur in a random thread, so any error of the form:

java.lang.OutOfMemoryError: PermGen space

is an indication that -XX:MaxPermSize= may need to be increased.

An example of an error due to MaxPermSize being too low, logged in a gda_output log file was:

Logging exception:
java.lang.OutOfMemoryError: PermGen space
Exception in thread "jReality ToolSystem EventQueue" java.lang.OutOfMemoryError: PermGen space
Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: PermGen space

Finally, on 32-bit RHEL machines, there also appears to be an issue when you increase the total of permgen space and heap to around 2.6GB.

In one test, using -Xmx2048m -XX:MaxPermSize=768m, the jvm refuses to load the client. This can be shown with:

$ java -Xmx2048m -XX:MaxPermSize=768m -version
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

With -Xmx2048m -XX:MaxPermSize=640m, the jvm allows the RCP GDA client to load, but then within 300 to 1000 seconds, the client would crash with a fatal error in the java runtime:

# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 32756 bytes for ChunkPool::allocate. Out of swap space?
#
#  Internal Error (allocation.cpp:166), pid=6265, tid=69073808
#  Error: ChunkPool::allocate
#
# JRE version: 6.0_24-b07
# Java VM: Java HotSpot(TM) Client VM (19.1-b02 mixed mode linux-x86 )
# An error report file with more information is saved as:
# /home/gfn74536/hs_err_pid6265.log

This is an internal error, not an exception, so it cannot be caught or handled, and appears to be the jit compiler itself running out of memory, even though there is plenty of both heap and PermGen space available.

The limits of the Windows 32bit jvm and the Windows or Linux 64bit jvm’s have not been explored.

10.9.5. Pauses due to garbage collection

If your GDA client suffers from frequent noticable pauses, you may want to try adding the option:

-verbose:gc

This will log details of the heap before and after each garbage collection, along with the time that the collection took. Since the JVM performs both minor and full collections, some will take very little time to complete while others may take seconds to complete.

Further options can be used to provide more information at each garbage collection.

-XX:+PrintGCDetails
-XX:+PrintGCTimeStamps
-XX:+PrintHeapAtGC

Although use of the System.gc() call is discouraged, some parts of the GDA software still use it. As System.gc() forces an immediate explicit full garbage collection, each call could cause the RCP GDA client to pause for more than a second.

If explicit garbage collection is causing problems, you can instruct the jvm to ignore calls to System.gc() by the use of the option:

-XX:DisableExplicitGC

Another option [1], which was introduced in Java 6, appears to offer halfway house between full explicit garbage collection and none:

-XX:+ExplicitGCInvokesConcurrent

[1] http://java.sun.com/performance/reference/whitepapers/6_performance.html#2.2.2

This option uses the Concurrent Mark Sweep Collector which aims to keep garbage collection-induced pauses short. This has not been tested, but might be worth investigating if both disabling explicit garbage collections and leaving it enabled cause problems.

10.9.6. Further information

For Java 6 the official tuning guide can be found in the document Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning [2] and there is additional information in the Java SE 6 Performance White Paper. [3]

[2] http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html

[3] http://java.sun.com/performance/reference/whitepapers/6_performance.html