A stipulation of both Europlanet 2020 RI and the forthcoming 2024RI grants is that all data generated as part of the TA (and other) parts of the projects must be made open access. Most funding agencies are moving to the same system; all data must be open access and easily accessible, so by taking action as part of the 2024RI we are all falling into line with what funding agencies will demand.
An efficient and light data distribution system has been designed and installed during Europlanet 2020 RI for Planetary Science data: VESPA. The system is comprised of a number of "data services" installed in various institutes, which can be queried together from a single user interface (the VESPA portal). Selected results can then be sent to various tools which display the data and can perform some level of analysis (depending on data format). We here provide general guidelines to make your data available in VESPA.
Use cases for TAs (2020 & 2024)
• General organisation:
- TA data will be handled through a limited number of data servers installed in key TA institutes. You don't have to install any software on your side, but instead you have to provide a description of your data, and to make the data available on the network where they can be accessed on demand (with fixed IP, always on-line).
- Data servers will be connected to the VESPA portal by the key TA institutes. Each can accommodate several data services (or databases) from different TAs or experiments. They allow the databases to be queried on-line.
- Your data description will be converted into a database describing the data elements (usually files).
• Design:
Your expertise is required to decide the organisation of the dataset:
- What are the data elements? In general these consist in individual measurements files, e.g.: lab spectra, images, measurement files, etc — however, any grouping is allowed (e.g. tar/zipped archives). Each element will have a specific URL.
- Identify the relevant parameters describing these elements (e.g.: sample ID, sample composition and properties, experimental setup, coordinates of observed area, spectral range, illumination angles, acquisition date, provider, instrument, campaign/visit reference, URL of the data, etc). Such parameters as commmonly refered to as metadata - they are required to document the data (and therefore you already have them in some form) and in this case they will be used to find specific conditions when searching the databases.
• Initial setup:
- You have to provide a table describing all data elements with the selected parameters. The key institutes will ingest these in the data servers. The table is best provided as a simple csv file (US format please, i.e. with separator = coma - beware of localized versions of Excel if you start with this).
- A validation phase is of course required before publication. As science experts, you are requested to check if the resulting presentation in the VESPA portal makes sense for your users, and actually allows them to find what they're looking for in the resulting data services.
- Updates will require the ingestion of an updated csv file.
Examples of data services
You can browse the current data services in the VESPA portal to illustrate the final format.
- Click the "Show All" button to see all descriptive parameters
- You'll see that many parameters are empty - they have to be present for technical reasons, but are informed only when they make sense for a particular dataset.
- You can play with the interface. In particular the left panel can be used to select a subset of data
Relevant examples of data services include:
- A collection of amateur telescopic images: PVOL
- A spectral library of solid samples: pds_speclib
- When the data consists in a series of scalar parameters, it can be included in the table directly. E.g.: TNO properties retrieved from observations: TNOsarecool
Sample data
The attached file is the csv catalogue for the IKS data service (a spectrometer on-board the VEGA-1 spacecraft). It describes 104 files with relevant parameters:
Extra info
Tutorials for data providers are linked on this page
Use only if you feel comfortable.
TA-sample-data-for-VESPA-services