Report of the
High Data-Rate Macromolecular Crystallography Meeting
ECM-30 2 September 2016
Report Date: 19 September 2016
This is the third of a series of three meetings in spring and summer 2016 on changes needed to existing major software packages for support of very high data rate macromolecular crystallography. This meeting was a satellite meeting of the European Crystallographic meeting in Basel Switzerland, 28 August – 1 September 2016. It was held at the Biozentrum in Basel on 2 September 2016. This meeting was chaired by Herbert J. Bernstein of Rochester Institute of Technology. This meeting was a follow-on meeting to the first meeting held at Brookhaven National Laboratory, 26 – 28 May 2016 and the second meeting held at an informal dinner gathering at the meeting of the American Crystallographic Association in Denver Colorado, 22 – 26 July 2016. The BNL meeting was organized by Herbert J. Bernstein of Rochester Institute of Technology, Nicholas K. Sauter of Lawrence Berkeley National Laboratory and Robert M. Sweet of Brookhaven National Laboratory.
This is a draft by HJB (yayahjb at gmail dot com) after review and correction by the participants
The meeting web site with slides of the talks is available at
The attendees at the meeting were:
Ivo Tews CCP4 WG2 Chair, University of Southampton
Naohiro Matsugaki KEK
Nicholas Sauter Lawrence Berkeley National Laboratory
Ezequiel (Zac) Panepucci SLS Paul Scherrer Institut
Markus Mathes Dectris, Ltd.
Daniel Eriksson Australian Synchrotron
Mark Könnecke SLS Paul Scherrer Institut
James Parkhurst DLS
Martin Savko SOLEIL Synchrotron
Frances C. Bernstein Brookhaven National Laboratory (ret.)
Jacinto López Sagaseta GlaxoSmithKline Vaccines S.r.l.
Robert Cheng leadXpro
David Hall DLS
Karine Röwer (Sparta) HZB
Andreas Förster Dectris, Ltd.
Borislav Angelov ELI
Artur Barczyk MAX IV
Jie Nan MAX IV
Herbert J. Bernstein Rochester Inst. Technology
Jun Aishima Australian Synchrotron
19 participants attended on-site, 1 participant attended remotely.
Issues Discussed at the Meeting
Inasmuch as some of the participants were new to these discussions, the meeting began with a brief review of the fact that the new generation of detectors and increased brightness at new MX beamlines require a careful reconsideration and re-engineering of networks, computers, software and process flows for macromolecular crystallography in order to deal efficiently with the new data flows without significant delays. The current status was reviewed (see the talks on http://www.medsbio.org/meetings/ECM_Sep2016_HDRMX_Meeting.html) with discussion between talks. The bottom line is that excellent progress has been made, and for some applications very high data rates are being achieved with excellent utilization of Eiger detectors with modest resource consumption, e.g. as reported by Andreas Förster and Zac Panepucci. In some cases capacity limits are being seen, for example in DIALS spot-finding benchmarks that see a 12 frame per second capacity in the nodes James Parkhurst reported on. In the end, the topics for which there was a consensus that further discussion and effort were required were (in the order discussed):
Compression Issues, both faithful and lossy
Data Volume Reduction in the Detector Control Unit (DCU)
Direct reading of HDF5
Touch once approaches
40 Gigabit per second network support
There was a long discussion but no agreement on precisely what is or is not needed in terms of further compressions and whether one needs to resort to lossy compressions, such as lossy wavelet compression, reduction to structure factors, binning and summing of images or whether all pixels of all images should be saved to ensure maximal future reprocessing opportunities, but it was clear that there will be some use cases in which such approaches will be appropriate. It was also clear that, at least in the short term, the lead in exploring the need for and support of such things as binning and higher compression-ratio compressions will have to lie with the beamlines rather than with Dectris,
After a lengthy collegial discussion in which Dectris clarified their plans and beamline scientists clarified their needs, it was unanimously agreed that the most workable approach to resolve both items 1 and 2 (compression issues and data volume reduction) was as follows:
Dectris is only responsible for providing the currently specified HDF5 FileWriter output and the streaming interface; and
The beamlines are responsible for processing the streaming interface to other formats;
But what should be archived or distributed should be NXmx compliant
On item 3, there was considerable interest in seeing the outcome in terms of timing of the conversion of XDS to direct reading of HDF5 files later this year, along with the implications of the improvements in parallelization, and perhaps use of Intel Knights Landing processors. There is hope of seeing a significant improvement in processing frame rates, but one cannot draw conclusions until this is all reduced to practice.
Item 4 relates strongly to how well the beamlines will implement their streaming interface solutions. If the new streaming interface boxes plant images where they can be directly accessed by applications without the need for excessive data motion, significant timing improvements are possible. This needs to be investigated further.
On 40 Gigabit network support, many of the beamlines that need it have or will be moving in that direction, except for the Eiger detector DCU links which consist of two 10 Gigabit links. Dectris indicated that they are seriously investigating the possibility of having an option of 40 Gigabit DCU support, but are not ready to commit yet on a specific schedule. On a related issue they made it clear that their DCU will not be a GPFS client.
All in attendance were reminded of the desirability of depositing test data with Wladek Minor and of providing code links on the hdrmx.meddsbio.org web site.
There was discussion of when and where to hold a next meeting. No resolution was reached. The moderator agreed to have at least one major HDRMX e-meeting between now and next summer and to at least have an informal HDRMX meeting at the IUCr meeting in India in summer 2017. Further meeting planning will be done via email.