CODE - Mobile and Flexible Instrumentation for full-field X-ray Diffraction Imaging
According to the requirements derived during the methodological development and inspired by the experience obtained during test experiments, the STROBOS-CODE partners have specified, designed, constructed, commissioned, and tested the dedicated CODE instrumentation for full-field X-ray diffraction imaging. The station is suited for operation at the IMAGE beamline of the synchrotron radiation source at KIT. Furthermore, its mobile and flexible layout allows its application for user experiments at the experimental end-stations of different state-of-the-art synchrotrons like PETRA lll and ESRF.
Sample Manipulation Module
The core module of the CODE-station is a sample-manipulator system for general full-field X-ray diffraction imaging, with particular focus on suitability for quasi in situ X-ray diffraction laminography (XDL).
In order to provide all degrees of freedom required by XDL, a solution consisting of three components was designed and realized (from bottom to top): 1) a tilt-stage varying the incidence angle θ to provide Bragg condition for the selected reciprocal lattice vector hhkl, 2) a highly precise and stable rotary table for the rotation about hhkl, and 3) a light-weight but high-precision miniatur parallel kinematics (3 translations and 3 rotations) for fine-alignment of the sample with respect to the axes defined by the components below as well as for selecting the region of interest within the sample.
Two complementary realizations were worked out (enabling adjustment to the demands of the specific measurement, i.e. the required precision, stability, angular range, available space, etc.). Their mechanical and electrical components were specified, purchased, and commissioned throughout 2017 and 2018.
Realization 1: Light-weight and compact solution. It employs a cradle-like so-called 1-circle-element for the laminographic tilt with an angular travel range of θ = ±15°.
Realization 2: Unlimited angular range in θ. This, in combination with sophisticated sample holders, allows a most flexible selecting of reciprocal lattice vectors for diffraction imaging measurements.
Both configurations shown in Fig 2. employ a so-called SpaceFab mounted on top of the system. It provides six degrees of freedom which not only enable a fast semi-automated angular fine-alignment, i.e. ensuring that the axis of the rotary stage ρ is alighned parallel to hhkl (ρ║hhkl), but also selecting the region of interest within the sample.
In 2019 the characterization of the sample manipulator was completed and demonstrated that the instrumentation achieves angular and spatial errors below 10-4° and 2 µm, respectively. By implementing suited software interfaces (via TANGO) all components of the instrumentation can now be controlled via a dedicated server. This server can either be integrated into a beamline control system or work completely autonomously as a plug-and-play-solution, in both cases providing a variety of semi-automated alignment scripts and measurement routines for a convinient user experience, see Fig. 2. The CODE instrumentation has already served during successful beamtimes in a development state at e.g. ID15A and ID19 at the ESRF, Grenoble, France, and in its final state at P07 and P23 at PETRAIII, Hamburg, Germany, see Fig. 3.
Detector Manipulation Module
Since in the case of X-ray diffraction imaging the propagation direction of the diffracted beam profile is rotated by twice the Bragg angle from the original incident beam direction, the project partners considered a suitable detector manipulation module, enabling a more flexible positioning of 2D detectors at positions in the upper hemisphere. Within the recent measurement campaigns, promissing devices for detector positioning were tested and a suitable 2-circle-element was purchased in 2019 and is now available for operation.
STROBOS - Camera Development
High-speed image-based process control requires continuous recording and analysis of image streams.
For this purpose we have developed the modular FPGA-based scientific streaming camera platform intended for rapid development of scientific data acquisition systems. The camera platform provides fast interconnects, customizable image-based triggers, and embedded process control logic. All these features are not available in commercial cameras. It is based on various CMOS active pixel sensors, mounted on mezzanine daughter cards. The daughter card is connected to the main readout board. For further data analysis the camera platform is seamlessly integrated in the UFO parallel processing framework.
The main benefits of the high-throughput camera platform are:
- Continuous data acquisition at full speed
- On-line image-based self-event trigger architecture (e.g. for fast rejection of use-less image)
- Automatic region-of-interest readout strategies
- Easily extendable to any available CMOS image sensor
- Fully programmable and controllable
For the UFO camera platform several image sensor boards are existing with a spatial resolution of 1 to 20 Megapixel and frame rates of up to 5000 frames per second in full sensor resolution. Even higher rates are possible at reduced or interpolated resolution by using the fast reject logic. Frame rate and resolution are only limited by the current sensor data throughput of about 6.5 GByte/s. The readout architecture itself is open for other sensors in order to match the specific requirements of scientific experiments. Due to its modular structure the camera electronics is easily extendable, e.g. for optical data transmission links.
Within STROBOS-CODE, the UFO high-throughput camera platform for continuous data streaming has been adopted to the requirements of diffraction imaging. The fast data link enabled tight coupling of programmable camera electronics and the UFO computing infrastructure. This heterogeneous DAQ setup represents concept for future data acquisition systems in general. First examples of embedded logic have been realized like real-time data reconstruction and fast feedback loops. During the course of the project, three sensor modules have been developed. The list of modules include two high-resolution cameras respectively of 2 and 4 Mpixels with a pixel pitch of 5.5 µm and frame rate of up to 330 fps, and one wide-field-of-view camera with 20 Mpixels pixel pitch of 6.4µm and frame rate of up to 30 fps.
To optimize continuous data streaming from the cameras to GPU-based computing infrastructure for fast 3D reconstruction, a flexible high-performance readout card has been developed. The readout card is integrated into the UFO camera platform and provides FMC connectors for images sensor boards like the 20 Mpixels sensor board shown in Fig. 4. The readout card is equipped with a Xilinx Virtex-7 FPGA and is connected to the computing infrastructure by PCIe Gen 3. The readout card has been designed as multi-purpose module and is foreseen for a wide field of DAQ applications. Two high pin count connectors are FMC VITA-57 compliant and provide a large number of high-speed in-terconnects to the application specific FMC card. DDR3 memories with 4 GB have been integrated and the firmware was optimized to operate with up to 15 GB/s.
In a traditional DMA architecture, data is at first written to the PC main system memory and in a second step distributed to the included GPUs for processing. Typically additional copy operations in the main memory are performed. Throughput and latency of the system are therefore limited by performance and load of the involved components and operating system. For the 5000 fps camera system the required data throughput exceeds 6 GB/s. Therefore new concepts have to be evaluated to ensure that high throughput with minimal load on the system architecture. We have developed a custom readout architecture that realizes fast communication between FPGA, CPUs and GPUs via Remote Direct Memory Access (RDMA). Our solution includes a Direct Memory Access (DMA) engine compatible with the Xilinx PCI-Express core, and appropriate Linux drivers. To enable direct low-latency communication between FPGA and GPU, a module for the UFO framework has been developed to manage memory transfers using AMD’s “Direct GMA” technology. The UFO camera is fully integrated in the UFO computing infrastructure. The camera abstraction library LibUCA allows to easily adopt for different sensor modules or pre-processing algorithms.
Cooling of the images sensor reduces noise and so improves the SNR. Together with the partners from the Shubnikov Crystallography Institute (SCI) a cooling system for the UFO cameras has been developed. The camera is housing designed, realized, and accomplished in SCI.
