Module edfio (Data Format Version 2.30) *

SYNOPSIS *

INCLUDE FILES *

TO LINK WITH *

RESTRICTIONS *

DESCRIPTION *

EXAMPLE READ DATA *

EXAMPLE WRITE DATA *

Setup Routines *

File Access Routines *

File History Routines *

Additional Routines *

HISTORY *

# include edfio.h

edfio.h

bslio.h

bslio.c

The file size is limited by the long integer format. On machines with 4 byte long integers additional blocks after 2^31 bytes = 2Gbytes cannot be accessed. Therefore, if a file contains several data blocks the absolute size of a single data file should not exceed 2GBytes.

EDF data format specific file access routines (read and write routines) The file format is described in ´SaxsKeywords.pdf´. The data files can contain several data blocks that can be accessed by data numbers and chain numbers. Chain numbers are positive integer numbers. A data block consists of a text block ("ASCII header") followed by a binary block ("binary data"). The block type is written at the top of the text block after the keyword "EDF_DataBlockID". The value "1.Image.Psd" marks data block number 1 of DataChain 1. Data chain 1 contains always primary (scientific) data, therefore the extension ".Psd". The value "1.Image.2" marks data block number 1 of DataChain 2 with additional image data, e.g. error data.

The following convention should be followed:

• primary image data : memory number 1;
• error image data : memory number 2;

Internally, a general chain is defined. It has chain number 0 and data number 0.

Data blocks with different chain numbers but the same data numbers belong together (e.g. data array (chain number 1) and associated error array (chain number 2)).

The input routines can also read bsl files. Bsl frame numbers (franum) and memory numbers (memnum) are in the following way internally converted to edf data numbers and chain numbers:

Remark: Only a single data block of a data chain can be present in memory at a given time. Subsequent reading deallocates the header data of the previous block and allocates memory for the header of the block from which is read. Binary data is not automatically deallocated and must explicitely be freed, before reading another binary data.

Read all images from data chain 1 of an edf file and convert to float.

Remark: Only a single data block of a data chain can be present in memory at a given time. Header values must be written before binary data is written. After writing, header and binary data are automatically deallocated and are not available any more.

Write 10 images with 512X512 float pixels into an edf file.

Write/don´t write file with general header

writetodisk : 1, write general header

writetodisk : 0, do not write general header (default)

0: success

Set the byteorder for all bsl input files

Changes the byte order of all bsl input files to byteorder

• byteorder : HighByteFirst, big endian byte order
• byteorder : LowByteFirst, little endian byte order

(default byte order: INTERNAL_BYTEORDER)

0: success

Reads the headers in 'DataChain' and searches for the minimum and maximum data number.

int DataChain (i) : data chain (0: general, 1: key_1, 2: key_2)

long int * pMinNumber (o) : minimum data number

long int * pMaxNumber (o) : maximum data number

int *pErrorValue (o) : error value

int *pstatus (o) : SAXS status

1 : successful

0 : failed

Searches for header 'DataNumber' in 'DataChain' and inquires for the dimension of the two dimensional data array.

If the header or the dimension of the data array does not exists, the return value is FALSE and a specific error value is returned. This error is not fatal and can be used as a test for the existence of a header.

long int DataNumber (i) : data number

int DataChain (i) : data chain (0: general, 1: key_1, 2: key_2)

long int Dim[3] (o) : Dim[0] Dimension, Dim[1], Dim[2]

long int * pDataArraySize (o) : total size of float data array in bytes

int *pErrorValue (o) : error value

int *pstatus (o) : SAXS status

1 : data header found, if *pstatus == Success

0 : data header not found,

*pstatus == status_error;

*pErrorValue == CouldNotFindHeader;

Open file 'fname' with "new", "old", "any" or "read"

Opens the file 'fname' with mode "new", "old", "any" or "read" and return a stream (success: 0 .. MaxFiles-1, error: -1).

• "new": open a new file for read/write, an existing file with the same name is overwritten
• "old": open an existing file for read/write and check file format
• "any": open either an existing file and check its file format or open a new file for read/write
• "read": open an existing file for read and check its file format

If an existing file is opened with "old", "any" or "read" its file start marker is checked and according to it is opened as an edf file or a bsl file.

-1: failed

0 or positive number: stream

22-Mar-1998 Peter Boesecke

Close an edf data file

Closes the edf data file stream

Writes a header symbol list into the header

return value FALSE if not found and no other error

return( int ) FALSE : data header not found,

*pstatus = status_error;

*pErrorValue=(CouldNotFindHeader, RoutineSucceeded);

TRUE : data header found or error,

*pstatus = Success or status_error;

*pErrorValue = <any>

currently not available

Searches for header 'DataNumber' in 'DataChain'. If it does not exists, it is created. It writes keyword and value into the header. The value is a character string.

In case of success the return value is 1, otherwise 0.

Searches for header 'DataNumber' in 'DataChain'. If it does not exists, it is created. It writes keyword and value as text into the header. The value is a float value.

In case of success the return value is 1, otherwise 0.

Searches for header 'DataNumber' in 'DataChain'. If it does not exists, it is created. It writes keyword and value as text into the header. The value is a long integer value.

In case of success the return value is 1, otherwise 0.

Write 2d data without conversion

Searches for header 'DataNumber' in 'DataChain'. If it does not exists, it is created. The data type, data size and array dimensions are written into the header. A 2d float data array (float Data[Dim[1]][Dim[2]) is written after the end of the header.

flat * pData (i) pointer to the start of the data array

long int Dim[0] (reserved)

Dim[1] (i) dimension 1

Dim[2] (i) dimension 2

23-Jul-1999 DataValueOffset added

Writes 2d float data to file

Searches for header 'DataNumber' in 'DataChain'. If it does not exists, it is created. The data type, data size and array dimensions are written into the header. A 2d float data array (float Data[Dim[1]][Dim[2]) is written after the end of the header.

float *pData (i) pointer to the start of the data array

long int Dim[0] (reserved)

Dim[1] (i) dimension 1

Dim[2] (i) dimension 2

Reads all user keys and values from a header

Reads all user keys and values from a header and returns them in symbollist.

return value FALSE if not found and no other error

return( int ) FALSE : data header not found,

*pstatus = status_error;

*pErrorValue=(CouldNotFindHeader, RoutineSucceeded);

TRUE : data header found or error,

*pstatus = Success or status_error;

*pErrorValue = <any>

currently not available

Reads a line

Searches for 'keyword' in the header 'DataNumber' in 'DataChain'. If the header or the keyword does not exists, the return value is 0 and a specific error value is returned. This error is not fatal and can be used as a test for the existence of the keyword or the header. The ´Value´ string specified by ´keyword´ is copied after the location pointed to by Value. The minimum allocated size for Value must be MaxValLen+1.

return value FALSE if not found and no other error

return( int ) FALSE : data header not found,

*pstatus = status_error;

*pErrorValue=(CouldNotFindHeader, RoutineSucceeded);

TRUE : data header found or error,

*pstatus = Success or status_error;

*pErrorValue = <any>

--------------------------------------------------------------------------+*/

int edf_read_header_line ( int stream, long int DataNumber, int DataChain,

const char * keyword, char * Value,

int * pErrorValue, int * pstatus ) /*---*/

Reads a float value

Searches for header 'DataNumber' in 'DataChain'. In this headers it searches for 'keyword'. If the header or the keyword do not exists, the return value is 0 and a specific error value is returned. This error is not fatal and can be used as a test for the existence of the keyword or the header. A pointer to the float value specified by 'keyword' is returned in 'Value'.

return value FALSE if not found and no other error

return( int ) FALSE : data header not found,

*pstatus = status_error;

*pErrorValue=(CouldNotFindHeader, RoutineSucceeded);

TRUE : data header found or error,

*pstatus = Success or status_error;

*pErrorValue = <any>

Reads a long integer value

Searches for header 'DataNumber' in 'DataChain'. In this headers it searches for 'keyword'. If the header or the keyword do not exists, the return value is 0 and a specific error value is returned. This error is not fatal and can be used as a test for the existence of the keyword or the header. A pointer to the long int value specified by 'keyword' is returned in 'Value'.

return value FALSE if not found and no other error

return( int ) FALSE : data header not found,

*pstatus = status_error;

*pErrorValue=(CouldNotFindHeader, RoutineSucceeded);

TRUE : data header found or error,

*pstatus = Success or status_error;

*pErrorValue = <any>

--------------------------------------------------------------------------+*/

Reads 2d data without type conversion

Searches for header 'DataNumber' in 'DataChain'. If it does not exist the routine stops with an error. A data array with the dimension Data[Dim[1],Dim[2]] is read from the file. The pointer &&Data[0,0] is returned in ppData. The data type, the data value offset, the byte order and the raster configuration of the array are returned.

If the stored array is only 1 dimensional, the data is read with Dim[2] set to 1.

The returned array has a length of *pDataArraySize bytes. The data buffer is allocated and must be released explicitly.

long int Dim[0] (o) 2

Dim[1] (o) dimension 1

Dim[2] (o) dimension 2

long int * pDataArraySize (o) size of the data array in bytes

void ** pData (o) pointer to the pointer of the data array

long * pDataValueOffset (o) data value offset of the array elements

int * pDataType (o) data type of the array elements (DType)

23-Jul-1999 PB pDataValueOffset, pByteOrder and pRasterConfiguration added

Read 2d data and convert to float

void edf_read_data_2d_float ( int stream, long int DataNumber,

int DataChain, long int Dim[],

long int * pDataArraySize, float **ppData,

int * pErrorValue, int * pstatus );

Searches for header 'DataNumber' in 'DataChain'. If it does not exist the routine stops with an error. A data array is read from the file and converted into a float array of the type float Data[Dim[1],Dim[2]]. The data array is allocated. The pointer &&Data[0,0] is returned in ppData. If the stored array is only 1 dimensional, the data is read with Dim[2] set to 1.

The data are read with the specification given in the header. All read data values are converted into float and the returned data array has a length of *pDataArraySize bytes. The data buffer is allocated and must be released explicitly.

The routines of this module are used to read and write history lines.

• 'int edf_history_new ( void )' must be called first. It clears and initializes the internal history list and argument list.
• 'int edf_history_skip ( void ) ' marks the next argument as not required
• 'int edf_history_take ( void ) ' marks the next argument as required (default)
• 'int edf_history_argv ( const char * substring )' is used to store the arguments of the call in an internal list. It can be called several times to pass all arguments.
• 'int edf_read_header_history ( int stream, long int DataNumber,
  int DataChain,
  int * pErrorValue, int * pstatus ) ' initializes the history list and reads the history strings from the file header.
• 'int edf_write_header_history ( int stream, long int DataNumber,
  int DataChain,
  int * pErrorValue, int * pstatus )' writes the history strings into the output file header.
• 'int edf_history_free ( void )' releases all internal lists.
• 'void hist_debug ( int debug )' sets the module into debug mode.

The length of a history line is limited to MaxHistoryLineSize-1.

1999-06-24 V1.0 Peter Boesecke

1999-11-08 V1.1 PB

This routines must be called first. If not already initialized, it initializes all history lists. Eventually existing history lines are removed and a new empty history line with size MaxHistoryLineSize is created. ´edf_history_argv´ adds parameters to this line. ´edf_write_header_history´ appends it with a new key to the history lines that were read with ´edf_read_header_history´.

In case of success the return value is 1, otherwise 0.

This routines marks the next parameter that is passed to edf_history_argv as not required.

In case of success the return value is 1, otherwise 0.

This routines marks the next parameter that is passed to edf_history_argv as required (opposite of edf_history_skip)

In case of success the return value is 1, otherwise 0.

Removes all history lines.

In case of success the return value is 1, otherwise 0.

Reads all history lines from the date file header and copies them to 'hline->key's. History lines have the keyword HISTORY_KEY_PREFIX'u', where 'u' is an unsigned positive integer. A new history line with the key HISTORY_KEY_PREFIX'last+1' and MaxHistoryLineSize bytes is created.

In case of success the return value is 1, otherwise 0.

Writes the history strings in 'history_root' into the date file header using the 'hline->key's as keywords. Writes the history string in 'history_argv_root' into the date file header using 'current_history_key' as keyword.

In case of success the return value is 1, otherwise 0.

Appends an argument to history line

Appends argument to history line

In case of success the return value is 1, otherwise 0.

Conversion to raster orientation 1

Conversion of the multi-dimensional array src with raster configuration number ´raster_configuration´ to the n-dimensional array dest with raster configuration number 1. The length n of data_dim is stored in data_dim[0]. The total length of the array is data_dim[n+1]. data_dim[i] is the length of coordinate i. Input and output array must be different. Sufficient memory must have been allocated for both arrays. The total number of elements in the array is specified in data_dim[n+1].

The raster configuration specifies only the way how data is stored. It does not influence the number of dimensions. Therefore, the dimension array is not changed even if, apparently, horizontal and vertical axes were changed.

An n-dimensional array has ´N = 2^n * (n!)´ different ways of storing its data in a regular raster. Each of the n axes can be stored in two different ways: up and down. This results in 2^n different possibilities of data storage. The n axes can be stored in any of the (n!) possible permutations. This results in N = 2^n * (n!) different ways of storing the data. A two dimensional array (n=2) can be stored in 2^2 * 2! = 8 different ways, a three dimensional array (n=3) can be stored in 2^3 * 3! = 8*6 = 48 different ways, and so on.

The data elements are stored in an array with dim_1 * dim_2 * ... * dim_n cells with identical size. The number of dimension is n. The first index (i_1) is the fastest. A specific raster configuration is given by the n-tupel (k_2, -k_3, -k_1), which means that the fastest index i_1 corresponds to the coordinate k_2, the medium fast index i_2 corresponds to the invers k_3 direction and the slowest coordinate i_3 corresponds to the invers k_1 direction.

Arrays can be stored in different ways depending on the relationship between offsets and fast and slow array indices. The configuration of the indices is specified by a raster configuration number D.

A unique raster configuration number for multi dimensional arrays is used that is defined on the basis of the following demands. The raster configuration number is called D in the text:

• Array indices are numbered from 1 to N.
• A one dimensional array can be stored from low array indices to high array indices (D=1) or from high array indices to low array indices (D=2).
• The array element Array(k_1, k_2, k_3, ... , k_N) is accessed with an offset I from the first element measured in element size

I = (i_1-1) + (i_2-1) * Dim[1] + (i_3-1) * Dim[2] * Dim[1] + ...
= Sum[ J=1,J<=N,J++ ]( i_J * Product[i=1,i<=J,i++](Dim[i]) )
(Dim[0]==1)

with i_1, i_2, i_3, ... replaced by k_1, k_2, k_3, ... in a special order that is specified by the raster configuration number D.

The raster configuration is described by grouping the array indices k_nn and the offset indices i_nn, e.g. for a 2-dimensional array where the fast and slow indices are interchanged.

Example for N=2 and D=5:

i_n 1, 2

k_nn 2, 1

• - The raster configuration number D is 1, when the array indices are ordered from "fast" (i_1) to "slow" (i_N), e.g. when i_1 = k_1, i_2 = k_2, etc.
• - The raster configuration D of the n-dimensional sub-array

Array[Dim_1, Dim_2, ... Dim_n] is the same as for the (n+1) dimensional array Array[Dim_1, Dim_2, ... Dim_n, Dim_(n+1)] if Dim_(n+1) == 1.

D ( Array[Dim_1, Dim_2, ... Dim_n, 1 ] ) == D ( Array[Dim_1, Dim_2, ... Dim_n] )

These demands give a unique description of multi-dimensional raster conformations with a configuration number D.

The raster configuration is defined as follows:

For the definition it is necessary to distinguish strictly between the array indices k_1, k_2, etc. and the offset indices i_1, i_2, etc.

A offset index with a small number, e.g. i_1 runs faster than an index with a higher number, e.g. i_3. The definition of the raster orientation is based on a standard orthogonal coordinate system with the first coordinate (x_1) horizontal and the second coordinate (x_2) vertical with respect to the observer. The position of the observer must be defined elsewhere. In standard scattering geometry the observer is located at the sample position and is looking against the detector. The origin of the coordinate system is at the lower left corner of the detector, independently of any detector pixel readout coordinate. The direction of the third coordinate x_3 is found with the vector product x_1 X x_2 = x_3. It is pointing against the observer. This might be usedful for representing three dimensional objects. It should be possible to find higher dimensional coordinates accordingly.

An n-dimensional array has ´N = 2^n * (n!)´ different ways of storing its data in a regular raster. Each of the n axes can be stored in two different ways: up and down. This results in 2^n different possibilities of data storage. The n axes can be stored in any of the (n!) possible permutations. This results in N = 2^n * (n!) different ways of storing the data. A two dimensional array (n=2) can be stored in 2^2 * 2! = 8 different ways, a three dimensional array (n=3) can be stored in 2^3 * 3! = 8*6 = 48 different ways, and so on.

The data elements are stored in an array with dim_1 * dim_2 * ... * dim_n cells with identical size. The number of data elements is n. The first index is the fastest. A specific raster configuration is given by the n-tupel (x_2, -x_3, -x_1), which means that the fastest index i_1 corresponds to the coordinate x_2, the medium fast index i_2 corresponds to the invers x_3 direction and the slowest coordinate i_3 corresponds to the invers x_1 direction. If the array has n_1, n_2 and n_3 elements in each direction the data origin X0 = (0,0,0) in the real world corresponds to the array element IX0 = (0,n_2,n_3).

In the standard configuration (D=1) all array indices k_nn are identical to the offset indices i_nn.

In the 1-dimensional case the standard configuration (numbered 1) is the configuration in which the array index increases with the coordinate. The second configuration is where the index is antiparallel to the coordinate (numbered 2).

n=1 has two configurations (A(2) = 2):

If the number of configurations for n coordinates is A(n) = 2^n * (n!), the number of configurations for n+1 coordinates is given by

1. 2 * (n+1) * A(n) = 2^(n+1) * (n+1)!

C(n) is the group of all possible configurations for n coordinates. If it is given, the configurations for n+1 coordinates can be built by inserting the configurations of the new coordinate (1 and -1) at each of the n+1 possible positions (n before each coordinate and 1 after the last coordinate). To have a well defined ordering the new coordinate is first added non-inverted after the end of all A(n) configurations and then inverted (A(n)*2). This is repeated subsequently from the end to the start before all n remaining positions.

(x1, x2, x3, ... , xn) -> (x1, x2, x3, ... , xn, xn+1)

(x1, x2, x3, ... , xn, -xn+1) +2

(x1, x2, x3, ... , xn+1, xn)

(x1, x2, x3, ... , -xn+1, xn) +2

...

( xn+1, x1, x2, x3, ... , xn)

(-xn+1, x1, x2, x3, ... ,-xn) +2

(n+1)*2

A(n) = 2^n * (n!)

The total number of configuration for n+1 coordinates is then

2. 2 * (n+1) * A(n) = 2^(n+1) * (n+1)! ,

which is equal to A(n+1).

n raster_configuration D Configuration

1 1 1

1 2 -1

2 1 1, 2

2 2 -1, 2

2 3 1,-2

2 4 -1,-2

2 5 2, 1

2 6 2,-1

2 7 -2, 1

2 8 -2,-1

3 1 1, 2, 3

3 2 -1, 2, 3

3 3 1,-2, 3

3 4 -1,-2, 3

3 5 2, 1, 3

3 6 2,-1, 3

3 7 -2, 1, 3

3 8 -2,-1, 3

3 9 1, 2,-3

3 10 -1, 2,-3

3 11 1,-2,-3

3 12 -1,-2,-3

3 13 2, 1,-3

3 14 2,-1,-3

3 15 -2, 1,-3

3 16 -2,-1,-3

3 17 1, 3, 2

...

3 32 -2,-3,-1

3 33 3, 1, 2

...

3 48 -3,-2,-1

The raster configuration 13 for n=3 (2, 1,-3) means that the first offset index of the array (which is the fastest) corresponds to the coordinate k_2, the second index corresponds to the coordinate k_1 and the third index to the inverted coordinate k_3.

The largest D (D = A(n)) is always the conformation where the direction and order of all array indices are inverted.

int status

0 : success

-1: failed

Returns the size of a data type element

DType data_type;

size_t edf_data_sizeof ( int data_type );

The size required for a data_type element is returned. Where data_type is the enum type DType.

NULL : error (e.g. for invalid data type)

>NULL : size of data_type

03-Mar-1998 PB Specification

Returns a pointer to the version string of the module edfio

Pointer to the version string

Return edf data format version string

Returns the edf data format version string.

Pointer to edf data format version string.

Prints the current filetable to the file ´out´

0: success

-1: failed

Returns the error message of ErrorValue

Allocates a buffer and copies the error message corresponding to ´ErrorValue´. It returns a pointer to the allocated buffer.

NoDataFormatError, UnknownErrorValue, RoutineFailed, RoutineSucceeded, CouldNotMallocMemory, CouldNotFreeHeaders, CouldNotGetBinaryArray, NoMoreStreamsAvailable, CouldNotOpenFile, EndOfFileDetected, CouldNotFindHeader, CouldNotFindSymbol, BadSizeDefinition, BadDataBlock, CouldNotFindKeyword, WriteDataError, ReadDataError, NoStreamOpen, NotESRFDataFile, NoDataBlocksFound, FileIsNotWritable, FileIsNotOpened, CouldNotCloseFile, CouldNotInsertChain, CouldNotInsertBlock, CouldNotInsertSymbol, MissingKeyDefinition, GeneralBlockNotFirst, ErrorCreatingGeneralBlock, ErrorReadingGeneralBlock, ErrorLocatingBlocks, CouldNotSetBuffer, NumberConversionFailed, DataConversionFailed, MissingArrayDimensions, Not2dData, CouldNotWriteDimension, CouldNotReadDimension, CouldNotWriteBinary, CannotReOpenGeneralBlock, CannotOpenAsBslFile

Set / reset module into debug mode

Sets/resets all sub-modules into debug mode