22. GDA on MX Beamlines¶
22.1. Data Collection¶
Data collection is the MX method of taking data at a certain position on a crystal. The user can define all the parameters for the data collection in the GDA client GUI. Multiple Data Collections can be set-up and executed from the client.
The Data Collection process takes place in two stages: set-up on the client side and actual data collection on the server side.
22.1.1. Client side¶
The client side Java code is responsible for displaying the correct parameters and view on the GUI and then set up all the parameters needed to define the Data Collection and pass them to the server to carry out the scan.
The client side code is written in java.
22.1.1.1. Process¶
This represents the top level of the GUI (i.e. what the user sees when starting the client). This same class is used for all mode options with the MX operations (e.g. Grid Scan, Line Scan, Fluorescence). It creates an instance of a ScanBlock.
This object is what changes the view shown on the client depending on what mode is selected. It registers the appropriate mode page (DataCollectionDetailsPage) and gets the DataCollectionScanManager containing the DataCollectionModel to pass in as input to the page. See the ‘Mode’ table.
Creates the DataCollectionModel and ultimately begin and manages the running of the DC.
A higher level description of what parameters should actually appear in the GUI screen. This object creates an instance of ExptTableModel and sets the values to those in-putted by the user from the client.
Contains details of the parameters that are included in the GUI view (e.g. Exposure time, number of images etc). These are saved as objects within
ExptTableModel(e.g. NumImages).ExptTableModelis a 2D array where each row contains the GUI parameters for a single defined Data Collection run. The client allows multiple DC scans to be submitted, therefore there may be more than one row in theExptTableModel.
E.g.
Sample
NumImgs
ExpTime
Oscillation
…
Sample 1
1
0.050
0.1
…
Sample 2
1
0.080
0.4
…
The ‘DetailsPage’ connects the GUI with the model and perform validation on the input parameters. It creates a
DataCollectionCommonobject which sets the client view to the current DC (row) and fills the parameter fields with their associated values.
The ScanFormView creates a ScanBlock instance with the input mode as SampleMode (*). The ScanBlock creates and registers the DataCollectionDetailsPage as the view to show, with the DataCollectionScanManager as an input. This manager contains the DataCollectionModel with the ExptTableModel parameters and set values. The DataCollectionDetailsPage uses DataCollectionCommon to set the view in the client to the current selected sample row and displays the correct parameter values for that sample. On submission from the client, the DataCollectionDetailsPage calls a runScan on the DataCollectionCommon object which calls on the manager to run the scan with the given parameters.
22.1.1.2. Running the Data Collection¶
This is carried out by the DataCollectionScanManager. It carries out the following tasks:
Does some additional validation of input parameters
Converts the ExptTableModel (which is client side only) to and ExtendedCollectRequests object which is used by the server to access the parameters. The actual conversion is done on the client side from the DataCollectionModel within the manager and saved to a file. Each row of the ExptTableModel (i.e. each sample) is converted into 1 ExtendedCollectRequest (ECR) held within the
ExtendedCollectRequestsobject.Constructs a string with Python commands to call and start the ‘
data_collection.py’ script on the server with the location of the ECR file as input. e.g.
import data_collection; reload(data_collection); data_collection.Start('" + ecrFileWithTime.getAbsolutePath() + "'" + String.format(", openClosedExptShutter=%s)", getShutterController().isShutterOpen() ? "True" : False), ...
22.1.1.3. ExtendedCollectRequests¶
The ExptTableModel containing the parameters on the client is converted into ExtendedCollectRequests for use by the server. The ExtendedCollectRequests contains ExtendedCollectRequest representing a single DC scan. Each ExtendedCollectRequest contains information required for the scan as well as the collect_request object containing an oscillation_sequence.
Both collect_request and oscillation_sequence come from the dna code base (external to GDA but still available as part of the MX deployment). They are still in use for legacy reasons. An oscillation_sequence is just a subset of parameters representing a sequence of images collected in one go. While in theory there can be more than one oscillation_sequence and collect_request within an ExtendedCollectRequest, in reality there is only ever one of each.
The client performs the conversion and saves the ECR to an xml file. See an example of the output here.
22.1.2. Server side¶
The server runs Python scripts contained in the ‘server’ directory.
It will run the Python script, data_collection, which was specified in the string command generated by the DataCollectionScanManager using the ECRs read from the path provided by the manager (i.e. data_collection.Start(...) ).
N.B. The server side Python scripts contain many beamline specific implementations within them (in the form of if statements). Be careful not to disturb these if changing the code and use your own if statements for aspects only relevant to your beamline.
22.1.2.1. data_collection.py¶
This is the script that is an interface to the client. The script carries out the following steps:
It gets the ECRs from the xml and validates the parameters.
calls on
datacollection.factory.createto create a ‘CollectRequestHandler’ (crh) from theHandleCollectRequestclass.factory.createsets the parameters in the crh which contains the detector, camera, attenuator, etc.calls execute on the crh object. The
HandleCollectRequestsclass runs the next stage of executing and running the scan.
22.1.2.2. HandleCollectRequest¶
The HandleCollectRequest object sets up and manages the scan. Fore more details on the exact process it carries out, see the MX Data Collection use case.
There are 2 variants of Data Collection within the HandleCollectRequest where the code splits: zebra and non-zebra (standard). These denote the 2 types of detector triggering systems used on the MX beamlines.
zebra: position compare triggering from the zebra.
non-zebra: position compare triggering from the motor controller.
22.2. Grid Scans¶
A Grid Scan is used to locate the crystal within the sample. The beam scans across the sample and analysis of the data returns the diffraction intensity to indicate the location to run a Data Collection on. This information is return to the client and displayed on the client GUI.
22.2.1. Client side¶
A similar process is carried out on the client as in the Data Collection case. Again the client side uses Java classes (some of which were introduced previously).
Creates the GridScanParametersModel using the GridScanParameters (gathered from an xml files or set to default values if there is no xml file) and ultimately begin and manages the running of the Grid Scan.
An object containing all the relevant parameters and values. These may be saved to an xml files if user has already entered values or can be set to default values if not.
A higher level description of what parameters and associated values should be displayed in the GUI screen. Uses the GridScanParameters.
The ‘DetailsPage’ connects the GUI with the model and perform validation on the input parameters. It gets an instance of the GridScanManager.
The ScanFormView creates a ScanBlock instance with the input mode as GridMode (*). The ScanBlock creates and registers the GridScanDetailsPage as the view to show, which subsequently gets an instance of the GridScanManager. This manager contains the GridScanParametersModel (with the GridScanParameters). On submission from the client, the GridScanDetailsPage calls a runScan which calls on the manager to run the scan with the given parameters. This calls the runGridScanXML() method in the GridScanManager which carries out the following steps:
Writes the
GridScanParametersto an xml file.Constructs a string with Python commands to call and start the ‘
diffractionGridScanStart_xml.py’ script on the server with the location of the xml parameters file as input.
22.2.2. Server side¶
The server runs Python scripts contained in the ‘server’ directory.
It will run the Python script, diffractionGridScanStart_xml, which was specified in the string command generated by the GridScanManager with the path to the parameter xml file as an input (i.e. diffractionGridScanStart_xml.Start(pathToXml, ...) ).
22.2.2.1. diffractionGridScanStart_xml.py¶
This script is the interface to the client and runs the Grid Scan on the server side. The Python relationship diagram below shows the key Python files involved in the scan:
It gets the parameters from the xml file passed to it.
calls on
gridscan.factory.createto create a handler. There are 3 variants to the handler that is created (these can be identified in the diagram above):pilatus (standard) - position compare triggering from the motor controller. Scans row by row or column by column (depending on whether there are less rows or columns). MX beamline examples: I04-1.
zebra - position compare triggering from the zebra. Currently only allows for column by column scanning. MX beamline examples: I03, I04, I24.
pmac - zebra triggering with ‘snake like’ scanning.
calls run on the handler to run the Grid Scan.
22.2.3. Client side feedback¶
Grid Scan is the only mode that feeds data back into the client GUI. The analysed results are bought back in and the diffraction intensity in each scan position is plotted on top of the scan area. An overview of the process is given below.
The data analysis software (scisoft) saves the analysis results in ISpyB and sends out a UDP packet with an ID to identify the run and frame. GridScanService listens out for a UDP packet and if received it queries the ISPyB database for the Image Quality Indicator (IQI) with that ID. Meanwhile the GUI image viewer (AbstractGridMode.getSidePlot()) gets the ImageQualityIndicatorModel (IQIM) which is an observer of both the GridScanService and the gridScanController. The gridScanController is listening to events from the actual grid scan code (on the server side) which is publishing information relating to the scan progress and set-up. The GUI image viewer also creates an instance of MxGridProfile. After setting the ‘scores’ held with the viewer to the values extracted from the IQIM, it calls on the MxGridProfile to update the plot in the GUI accordingly.
The GUI will update as it receives each UDP packet from each scan position as the data is analysed.
Note: This cannot be tested in ‘dummy’ mode.
22.3. Line Scan¶
As the name would suggest, this mode carries out a scan across a line defined by the user. The client offers the option to define the ‘Number of wedges’ (i.e. the number of different scan positions) and the number of ‘Image per wedge’ (i.e. the number of images taken as each position.)
There are 2 types of line scan options:
Wedged: move to 1 xyx position, capture images and then move to the next. There is 1 trigger for each position. E.g. Number of wedges = 36, Image per wedge = 10 => this translates into 36 positions, with 10 images being taken as each position results in 360 images. In this case the detector would be triggered 36 times.
Helical: constantly taking images while the sample is rotated (so 4 simultaneous moves occur in the x,y,z and omega directions). In this case there is only ever 1 wedge and so only 1 trigger sent to the detector. E.g Number of wedges = 1 (fixed), Image per wedge = 360 => this results in 360 images from a single trigger sent to the detector.
22.3.1. Client side¶
The client side code to run a Line Scan is essentially the same a Grid Scan.
Creates the LineScanParametersModel using the LineScanParameters (gathered from an xml files or set to default values if there is no xml file) and ultimately begin and manages the running of the Line Scan.
An object containing all the relevant parameters and values. These may be saved to an xml files if user has already entered values or can be set to default values if not.
A higher level description of what parameters and associated values should be displayed in the GUI screen. Uses the LineScanParameters.
The ‘DetailsPage’ connects the GUI with the model and perform validation on the input parameters. It gets an instance of the LineScanManager.
The ScanFormView creates a ScanBlock instance with the input mode as LineMode (*). The ScanBlock creates and registers the LineScanDetailsPage as the view to show, which subsequently gets an instance of the LineScanManager. This manager contains the LineScanParametersModel (with the LineScanParameters). On submission from the client, the LineScanDetailsPage calls a runScan() which calls on the manager to run the scan with the given parameters. This calls the runLineScanXML() method in the LineScanManager which carries out the following steps:
Writes the
LineScanParametersto an xml file.Constructs a string with Python commands to call and start the ‘
lineScanStart_xml.py’ script on the server with the location of the xml parameters file as input.
22.3.2. Server side¶
The server runs Python scripts contained in the ‘server’ directory.
It will run the Python script, lineScanStart_xml, which was specified in the string command generated by the LineScanManager with the path to the parameter xml file as an input (i.e. lineScanStart_xml.Start(pathToXml, ...) ).
22.3.2.1. lineScanStart_xml.py¶
This script is the interface to the client and starts the Line Scan. The Python relationship diagram below shows the key Python files involved in the scan:
gets the parameters from the xml file passed to it.
calls on
linescan.factory.createto create a handler. There are 4 variants to the handler that is created (as shown in the relation diagram above):wedged - runs the wedged scanning methods for a detector with non-zebra triggering (i.e. motion controller).
wedged_zebra - runs the wedged scanning methods for a detector with zebra triggering.
helical - runs the helical scanning methods for a detector with non-zebra triggering (i.e. motion controller).
helical_zebra - runs the helical scanning methods for a detector with zebra triggering.
calls run on the handler to run the Line Scan.
22.4. MX Databases¶
A schematic of the databases used by MX beamlines (including both client and server side) is shown below:
