Standard data products
There are standard data products, which are the statistics files generated by the standard LOFAR system. These are typically used for programmes such as riometry, solar observing, scintillation experiments, station calibration, and so on. These standard data products are as follows:
SST -- Subband-Statistics
These record a 1-second average power, for each subband, for each of the 96 receiver units (RCU). This produces 96 files, with N records (where N is the number of seconds), where each record is 512 subbands in size.
BST -- Beamlet Statistics
These are 1-second average power, for each beamlet for each polarisation. There are either 244x16-bit, 488x8-bit or 976x4-bit beamlets. (In reality the 16-bit beamlets only have 12-bits of dynamic range, but are still packed into 16-bit words). This produces two files (X and Y polarisation), with N records (where N is the number of seconds), where each record is B beamlets in size.
ACC -- Array Covariance Cubes
ACC files (also sometimes referred to as Antenna Cross-Correlation) files contain visibility data for all the baseline pairs within the array, averaged to 1 second.
XST -- Cross-correlation Statistics
Similar to ACC files, but these record each second a single subband, rather than stepping through them. Additionally, rather than resetting and starting a new file after 512 seconds, it records continuously. Note that it is not possible to recrod both ACC and XST files at the same time.
It is important that observers appreciate how much data KAIRA can record, so that data storage and transport can be appropriately managed. The following table summarises the rates, assuming a standard configuration:
|Type||Files per day||Size per file||Daily total||Rate|
|SST||96||352149504||33.8 GB||391 kB/s|
|ACC||164||75497472||12.4 GB||143 kB/s|
|BST||4||167821248||0.7 GB||7.7 kB/s|
Non-standard data products
In addition to the standard data products, KAIRA can also record raw voltage level data. These are typically recorded with programmes such as KLP or KUG (but not both!). For these, the data rates can be extremely high and, as such, it does not make sense to indicate a daily total (as that would easily our existing on-site storage).
|Type||Files||Individual rate||Total rate|
|KUG||4||96 MB/s||383 MB/s|
|KLP||approx. 244/s||90 MB/s||380 MB/s|
Please note that any experiment requiring raw voltage-level data will require these non-standard products. This includes Incoherent Scatter Radar (ISR) experiments and Very-Long Baseline Interferometry (VLBI) observations. In all such cases, data storage and handling will need to be coordinated with the KAIRA team. In large data-volume cases, observers will be expected to provide their own storage.
These represent maximum values. Sometimes we decimate the data (pulsar observing) or average it (interplanetary scintillation observations). These data can be taken in addition to standard data products.
- "Computing for data acquisition", KAIRA document KAIRA-SGO-PRP-002.
- "LOFAR station data cookbook", ASTRON document LOFAR-ASTRON-MAN-064.