Marple loves CSV files. They are simple, can be read RAM efficient and everyone uses them!
The disadvantage is that CSV files have a lot of freedom, so you need to adhere to a minimum structure in order to get your data in Marple.
CSV files need to have:
- Signal names in the first row
- Delimited by comma (,) period(.) or semicolon (;)
- One column that defines the 'time' axis, this can be selected in Marple
- Rows of data
- Numeric data (Marple will ignore non-numeric data points)
- Time can be a number or datetime format. (YYYY/MM/DD hh:mm:ss preferably)
Sometimes there are multiple time axes, or the data is sampled at different sample rates. In those cases a CSV file is not a handy format... you might think!
Typically you can solve this by creating a set of CSV files, one for each time base (eg. sampling rate) and wrapping those together in a ZIP file. Marple will recognize this and use the same algorithms to decode the individual CSV files, but organize all data so you can easily visualise signals of different sample rate all at once.
HDF5 files are widely known for their excellent handling of large data sets.. and we agree! We love HDF5 data sets!
Unfortunately, HDF5 files come in different shapes and formats, so we need to agree to some structure.
Marple supports 2 types of HDF. We call it SIGNAL based and MATRIX based structures.
In a signal based structure the data consists of Groups (dictionary-like objects) where the signal names are used as keys. For every signal a Dataset (array) defines the data.
We also allow for signal grouping, as can be seen in the example below
A matrix structure is a bit more complex to construct, but allows for easier row-by-row processing. It also allows for extra information such as units.
MAT files are used a lot when using MATLAB or certain loggers such as dSpace, Speedgoat, .. MAT files are treated similarly to HDF5 files (as described above), so the same rules apply.
If you have a MAT file of version 7.3 or higher, this will be treated as a HDF5 file. MAT files of a lower version are significantly less efficient and will require more effort to be parsed by Marple.
The ULOG format is often used when dealing with systems that operate using the PX4 software or ecosystem. We think it's a great format because it is standardized!
As long as you adhere to the ULOG structure, .ulg formats are plug & play for Marple. No configuration needed!
Systems that operate on Ardupilot software or ecosystem often have a standard type of log file. These log files can be saved with either .log or .BIN extention.
Marple is able to understand these formats as long as they adhere to the standard MAVLink structure.