Description of the Raw Data Format at BL4/ID2

This is a very fast written description, of what is stored in the header together with the image data. For a summary of all keywords that are currently in use with SAXS data look into the file ~boesecke/programs/work/include/SaxsDefinition.h. This C-header file contains also all transformations that are in use. The actual raw data file of the gasfilled detector, that is produced by spec is defined at the end (see also the C-program: ~boesecke/programs/work/include/GetDldData.C

 

Center_1 = 23 ;

Center_2 = 24 ;

DDummy = 0.1 ;

Dim_1 = 1024 ;

Dim_2 = 1024 ;

Dummy = -1 ;

Image = 1 ;

Offset_1 = 0 ;

Offset_2 = 0 ;

Psize_1 = 0.025 ;

Psize_2 = 0.026 ;

SampleDistance = 0.995386 ;

SaxsDataVersion = 1.0 ;

Title = 0 ;

WaveLength = 7.69043e-11 ;

 

The actual structure was implemented on the base of the ESRF data format, which has been constructed Peter Daly. The ESRF data format does not have a very consistent definition and has been criticised for it. The programs used at BL4/SAXS are using the ESRF data format only as a way to store the data. The definition given below are independent of the format and can be implemented in other data bases later. Even the programs are not based on the ESRF data format and can be updated to an other format, if necessary. The data is stored in an ASCII header followed by a binary section. A basic header contains the following keywords (apart from the ESRF data format keywords which define the backbone) are:

 

/n{/n

...

HeaderID = EH:000001:000000:000000 ;

ByteOrder = HighByteFirst ;

DataType = FloatValue ;

Size = <length of binary data in bytes>;

...

Center_1 = 23 ;/n

Center_2 = 24 ;/n

DDummy = 0.1 ;/n

Dim_1 = 1024 ;/n

Dim_2 = 1024 ;/n

Dummy = -1 ;/n

Image = 1 ;/n

Offset_1 = 0 ;/n

Offset_2 = 0 ;/n

Psize_1 = 0.025 ;/n

Psize_2 = 0.026 ;/n

SampleDistance = 0.995386 ;/n

SaxsDataVersion = 1.0 ;/n

Title = 0 ;/n

WaveLength = 7.69043e-11 ;/n

}/n

Exactly <Size> bytes of <binary data>

 

Coordinate System

 

The origin (0,0) is at the lower left corner. The first pixel touches with its lower left corner the origin. The uper edges of the first pixel are 1.0 in both directions. All following pixels are defined accordingly. This definition makes the handling of binning procedures easier.

Remark: This is like it should be, but for the moment there is still a shift by 0.5, 0.5, i.e. the center of the first pixel is (0.0,0.0)).

Therefore: SaxsDataVersion = 1.0 ;

 

Description of keywords

 

ESRF Data Format Keywords

HeaderID, ByteOrder, DataType, Size

These are internal keywords of the ESRF data format and will not be discussed.

 

The header must be padded with blanks that its size is a multiple of 512 bytes.

 

An ESRF data format file can contain an unlimited number of header/binary data

blocks.

 

SAXS Format Keywords

The following keywords are followed by integer values:

- Image:

sequence number of the data in a single data file. The data is processed according to the number following after Image (default 1, but can be any integer number). The image number must be unique in the data file.

- SaxsDataVersion = 1.0:

Version of the SAXS data format (must be 1.0)

- Dim_1, Dim_2:

Dimension of the actual data array (column, row)

 

The following keywords are followed by float values:

- Dummy:

Dummy value

- DDummy:

All value between Dummy-DDummy and Dummy+DDummy are not valid.

- Offset_1, Offset_2:

Actual start position (in pixel) the data (in pixels)

- Center_1, Center_2:

Position of the primary beam on the detector (in pixels)

- Psize_1, Psize_2:

Pixel size in meter

- SampleDistance:

Distance between sample and detector center

(Center_1, Center_2).

 

- WaveLength:

wavelength in meter

- Title:

A single line string the describes the data.

 

Remark: The definition of the center as the point where the primary beam (virtually) hits the detector is only valid if the detector is perpendicular to the primary beam.

 

Raw Data Format Keywords

 

The previously defined keywords are used through all the treatment of the data file, raw data keywords are only available in the raw data file.

 

The raw data files contain a lot of other keywords which are defined in the following way. HM stands for Historgramming Memory, HS is an abbreviation for Histogramming Scaler.

 

DetectorName = X-ray Image Intensifier with FRELON CCD ;

-should be a unique name of the used detector

 

DetectorPosition = 0.995139 ;

+ Position of the detector on the bench in meter (inside the detector tube)

The sample distance can be calculated from the DetectorPosition with an

offset. E.g. if the detector is at home (closest distance):

DetectorPosition = 0. The SampleDistance of the X-ray Image Intensifier

from the Sample is DetectorPosition + 1.xxx meter.

 

HMFile = /data/opid02/inhouse/ccd220897/flatfield2208970002 ;

- full path of the raw data file (info only)

 

HMFrame = 1 ;

+histogramming memory frame one (number is converted to image number)

 

HMStartTime = Fri Aug 22 19:18:51 1997 ;

-starting time of the first frame (info only)

 

HS32C01 = 1.09726e+07 ;

HS32C02 = 1.17135e+07 ;

HS32C03 = 0 ;

HS32C04 = 369815 ;

HS32C05 = 1.35419e+07 ;

...

HS32C15 = 7.67908e+06 ;

...

HS32C32 = 0 ;

+content of 32 channel histogramming scaler 1 to 32 (HS32C01 to HS32C32)

 

HS32Depth = 24 ;

+depth of the value (usally 24 bit)

 

HS32F01 = 7.56e+07 ;

HS32F02 = 7.27e+07 ;

HS32F03 = 3.73e+07 ;

HS32F04 = 1.1913e+09 ;

HS32F05 = 2.11339e+07 ;

HS32F06 = 6.9124e+08 ;

...

HS32F15 = 1e-06 ;

...

HS32F31 = 1 ;

HS32F32 = 1 ;

+multiplication factor for channel 1 to 32

 

HS32Len = 32 ;

+number of the used channels

 

HS32N01 = PIN1 ;

HS32N02 = PIN2 ;

HS32N03 = PIN3 ;

HS32N04 = PIN41 ;

HS32N05 = PIN42 ;

HS32N06 = PIN5 ;

HS32N07 = PIN6 ;

HS32N08 = PIN7 ;

...

HS32N15 = time ;

...

HS32N31 = 0 ;

HS32N32 = 0 ;

+names of channel 1 to 32

 

HS32Z01 = 44.76 ;

HS32Z02 = 26.92 ;

...

HS32Z15 = 0 ;

...

HS32Z31 = 0 ;

HS32Z32 = 0 ;

+ zero values of the channels

 

HSAnode = 6 ;

+channel number in which contains the anode counts (channel 1 to 32)

 

HSAnodeS = 231 ;

+alternative channel that contains divided the anode counts (0 if not used)

 

HSI0 = 4 ;

+channel number of the I0 monitor

 

HSI0S = 3 ;

+alternative channel of the I0 monitor that contains divided counts (0 = unused)

 

HSI1 = 6 ;

+channel number of the counter for the transmitted intensity I1

 

HSI1S = 8 ;

+alternative counter for I1

 

HSTime = 15 ;

+channel which contains the counting time

 

HSTimeS = 23 ;

+alternative channel of the counting time

 

MachineInfo = Ie=141.81mA,gap46=21.80mm,taper46= 0.01mm,gap26=200.00mm,taper26

= 0.00mm ;

-single line string with machine information (info only)

 

OpticsInfo = optics ;

-single line string with optics information (info only)

 

StationInfo = id2 ;

-single line string with station information (info only)

 

SubTitle = 3to4 ;

-single line string with sample or experiment information (info only)

 

DetectorInfo =

- single line string with detector information (not used for the moment)

 

Example:

The time is measured in channel 15 (HSTime = 15). The value HS32Z15 is

not used for the time:

 

Channel 15 has the name: time

The calibrated value of channel 15 is:

(HS32C15)*HS32F15 = 7.67908e+06*1e-06 = 7.67908 seconds

 

Channel 1 hast the name: PIN1

The calibrated value of channel 1 is (the value HS32Z15 is not used for the

time):

(HS32C01-HS32Z01*time)*HS32F01 = (1.09726e+07 counts-44.76 counts/sec*7.67908 sec)*7.56e+07 photons/count = 8.29503e+14 photons

 

Remark: Normalization of the data to absolute scattering intensities Together with the absolute detector response, these values are sufficient to normalize the data automatically to absolute scattering intensities.

 

Spec raw data file of the gasfilled detector data

The first line of a block contains its number of lines, the last line contains -1 as a separator

 

The header filename is followed by hm (for histogramming memory file) The scaler data are defined in the Scalar header block. The binary data are defined in the Image header block. The current version number is 2.2. Older version are checked by a string comparison. The run number is just a number between 1 and 999, which is used to distinguish subsequently taken data. One line per item is used

General Header: 4, run number, 2.2 (version number)

and separator(-1)

 

Scalar header: Length of header (including separator),

scalar memory ID (2 for scalar, 3 for

histogramming memory),

first channel,

last channel,

first scalar,

last scalar,

two zero for padding,

length of scalar data in bytes,

separator (-1),

Scalar data: Length of scalar data block (including sep.),

scalar data ID (5 for scalar data)

(last scalar - first scalar) *

(last channel - first channel) lines of data

separator (-1),

 

Image header: length of header (including separator),

3 (presumably signifying hist. memory ???),

data length in bits (16 or 32),

x size,

y size,

number of images,

two zero for padding,

size of each image in bytes,

x offset

y offset

separator (-1)

 

Timing Header: length of header (including separator),

"TIMING"

DLD_HMSTARTTIME

for (i=first_frameno;i<=last_frameno;i++) {

DLD_HMDELTATIME

}

separator (-1)

 

Scaler Calibration Header:

length of header (including separator),

"SCALERCALIB"

DLD_SCALER_DEPTH

DLD_SCALER_I0

DLD_SCALER_I1

DLD_SCALER_ANODE

DLD_SCALER_TIME

first_scalerno

last_scalerno

for (i=first_scalerno-1;i<last_scalerno;i++) {

DLD_SCALER_NAME

DLD_SCALER_ZERO

DLD_SCALER_CALIB

}

DLD_SCALER_I0S

DLD_SCALER_I1S

DLD_SCALER_ANODES

DLD_SCALER_TIMES

separator (-1)

 

Experiment Header:

length of header (including separator),

"EXPERIMENT"

DLD_CENTER_1

DLD_CENTER_2

DLD_PIXSIZE_1

DLD_PIXSIZE_2

DLD_WAVELENGTH

DLD_SAMPLEDISTANCE

DLD_TITLE

DLD_SUBTITLE

DLD_DETECTORPOSITION

separator (-1)

Info Header:

length of header (including separator),

"INFO"

DLD_DETECTORTYPE

DLD_MACHINEINFO

DLD_OPTICSINFO

DLD_STATIONINFO

DLD_PROPOSALINFO

separator (-1)

 

EXAMPLE:

It is actually an CCD file but there is not a big difference.

(Only data size).

 

 

4

634

2.2

-1

10

2

1

32

1

1

0

0

128

-1

35

5

1219

865

0

2287

23015

19152

3182

3590

0

0

0

0

1021533

499

20000357

9766

0

0

0

0

0

3337

20479643

20479643

0

0

0

0

0

0

0

0

-1

12

3

16

1024

1024

1

0

0

2097152

0

0

-1

5

TIMING

Wed May 7 19:30:45 1997

0.006

-1

110

SCALERCALIB

25.585

5

7

13

15

1

32

PIN1

20.99

7.5617e+07

PIN2

1.49

72700000

PIN3

0

37300000

PIN41

106.15

4.29349e+08

PIN42

933.65

1.03513e+07

PIN5

1014.9

-65985562914.776428

PIN6

238.35

1.95772e+06

PIN7

150.2

126627372650.48988

I0

0

9670000

I02

0

1

I1

0

1

I2

0

1

anode

0

1

anode2

0

1

time

1e-06

time2

0

0.000205

thc1

0

0.002048

thc2

206.8

2009.7

vfc23

0

1

vfc24

0

1

thc5

0

1

vfc26

0

1

vfc27

0

1

thcsa

0

0.0005

0

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

10

7

14

16

-1

12

EXPERIMENT

256

256

0.000366

0.000361

9.88717e-11

0.999968

working proteins

olivier

0.160975

-1

8

INFO

X-ray Image Intensifier with FRELON CCD

Ie= 0.00mA,gap46=199.98mm,taper46= 0.02mm,gap26=20.40mm,taper26= 0.00mm

optics

id2

stopped flow

-1