Building the resource descriptor for your EPN-TAP service in DaCHS

EPN2020-RI

EUROPLANET2020 Research Infrastructure 

H2020-INFRAIA-2014-2015 

Grant agreement no: 654208

Document: VESPA-WP6-3-020-TP-v0.1(62)

Building the resource descriptor for your EPN-TAP service in DaCHS

Date: $action.dateFormatter.formatGivenString("yyyy-MM-dd",$content.getLastModificationDate())

Responsible WP Leader: Stéphane Erard

Table of Contents

Introduction

This document describes how to build your Resource Descriptor (RD) for an EPN-TAP service using DaCHS. The full documentation of DaCHS is available at http://docs.g-vo.org/DaCHS/ref.html, including a section "Anatomy of an RD" that describes the RD structure and syntax in details.

Overall Structure

The RD file is an XML file. The properties of the RD file can be either set up as XML children elements or as attributes of their parent property. The two following examples are equivalent, the first show the attribute syntax, while the second illustrates the XML element child option.

• Attribute syntax:

<resource schema="my_service">
	[...]
</resource>

• Child syntax

<resource>
	<schema>my_service</schema>
	[...]
</resource>

The latter syntax is useful when the property has children properties.  

Service Metadata

The first property of an RD is the list of service metadata. They are specified in a series of <meta>[...]</meta> elements. The following <meta> elements should be present in your file:

<meta name="title">Nancay Decameter Array observation database</meta>

<meta name="description" format="plain">
	Decametric radio observation from Nancay decameter array.
	The Nancay Decameter Array (NDA) at the Station de
	Radioastronomie de Nancay (SRN) is a phased array of 144 
	"Teepee" helicoidal antenna, half of which being Right 
	Handed (RH) polarized and the other half being Left Handed
	(LH) polarized. Four receivers are currently connected to 
	the NDA, sampling data in spectral ranges within 5 to 80 MHz.
</meta>

<meta name="copyright">
	Rules of Use:<br/>
	SRN/NDA observations in open access can be freely used for 
	scientific purposes. Their acquisition, processing and 
	distribution is ensured by the SRN/NDA team, which can be 
	contacted for any questions and/or collaborative purposes. 
	Contact email: contact.nda@obs-nancay.fr
	<br/><br/>
	We kindly request the authors of any communications and 
	publications using these data to let us know about them, 
	include minimal citation to the reference and 
	acknowledgements as presented below.
	<br/><br/>
	Acknowledgement:<br/>
	The authors acknowledge the Station de Radioastronomie de 
	Nancay of the Observatoire de Paris (USR 704-CNRS, supported 
	by University d'Orleans, OSUC, and Region Centre in France) 
	for providing access to NDA observations accessible 
	online at http://www.obs-nancay.fr
	<br/><br/>
	Reference:<br/>
	A. Lecacheux, The Nancay Decameter Array: A Useful Step 
	Towards Giant, New Generation Radio Telescopes for Long 
	Wavelength Radio Astronomy, in Radio Astronomy at Long 
	Wavelengths, eds. R. G. Stone, K. W. Weiler, M. L. Goldstein, 
	and J.-L. Bougeret, AGU Geophys. Monogr. Ser., 119, 321, 2000.
</meta>

<meta name="creationDate">2016-03-30T15:52:00</meta>

<meta name="creator.name">Station de Radioastronmie de Nancay</meta>

<meta name="subject">jupiter</meta>
<meta name="subject">the-sun</meta>
<meta name="subject">solar-radio-emission</meta>
<meta name="subject">aurorae</meta>
<meta name="subject">planetary-magnetosphere</meta>
<meta name="subject">solar-wind</meta>
<meta name="subject">radio-astronomy</meta>
<meta name="subject">solar-system-astronomy</meta>

<meta name="contact.name">Laurent Lamy</meta>

<meta name="contact.email">contact.nda@obs-nancay.fr</meta>

<meta name="contact.address">
	Station de Radioastronomie Route de Souesmes, F-18330 Nancay, France
</meta>

<meta name="referenceURL">
	https://www.obs-nancay.fr/reseau-decametrique/
</meta>

<meta name="facility">Station de Radioastronomie de Nancay#SRN</meta>

<meta name="instrument">Nancay Decameter Array#NDA</meta>

<meta name="source">2000GMS...119..321L</meta>

<meta name="contentLevel">General</meta>
<meta name="contentLevel">University</meta>
<meta name="contentLevel">Research</meta>
<meta name="contentLevel">Amateur</meta>

EPNcore table definition

The EPNcore table should be defined in the RD using the epntap2 mixin. This ensures that your EPN-TAP service is compliant with the EPNcore specification. The epntap2 mixin will be updated as needed via the DaCHS debian package update. If you only plan to use the EPNcore mandatory parameters, your table definition section will be very simple:

<table id="epn_core" onDisk="true" adql="True" primary="granule_uid">
	<mixin spatial_frame_type="body">//epntap2#table-2_0</mixin>
</table>

This minimal table definition says:

• define an epn_core table
• write it on disk (i.e., do not keep it in RAM)
• activate ADQL for query
• use the granule_uid column for the primary key of the table
• use the epntap2 template table with only mandatory parameters, and with spatial_frame_type = "body"

If you plan to use some optional parameters, as defined in the EPNcore specification, the table definition will look like:

<table id="epn_core" onDisk="true" adql="True" primary="granule_uid">
	<mixin 
		spatial_frame_type="body"
		optional_columns="access_url access_format access_estsize thumbnail_url publisher bib_reference target_region feature_name"
		>//epntap2#table-2_0</mixin>
</table>

The extra optional_columns attribute tells the template engine to set up those extra columns, as they are defined in the epntap2 mixin.

If you plan to use custom columns of your own, you have to define them in the table definition element, as shown in the following example:

<table id="epn_core" onDisk="true" adql="True" primary="granule_uid">
	<mixin 
		spatial_frame_type="body"
		optional_columns="access_url access_format access_estsize thumbnail_url publisher bib_reference target_region feature_name"
		>//epntap2#table-2_0</mixin>
	<column name="receiver_name" type="text" ucd="meta.id" description="Receiver name used with the instrument." />
</table>

The column elements defines an extra column of the EPNcore table.

Data ingestion

The metadata ingestion is done through module called a Grammar in DaCHS jargon. Depending on the form of the metadata, different solutions are available. The Grammar module output is fed to a rowmaker module, which fills the table rows, with transformations if necessary. 

• Preprocessed metadata available as a CSV file: The data provider is pre-processing his data collection to build a CSV file, containing the EPNcore metadata using the adequate units and conventions. In this case, the csvGrammar shall be used.
• Individual data files available from the DaCHS server as FITS files: The data provider is mounting a remote volume (e.g., through NFS) with the data files. If the data format is FITS, we can use the fitsProdGrammar to load the FITS files header.
• Individual data files available from the DaCHS server as CDF files: The data provider is mounting a remote volume (e.g., through NFS) with the data files. If the data format is CDF, we can use cdfHeaderGrammar to load the CDF global attributes.
• The metadata is available in an external SQL database:  The data provider has access to an SQL database, containing the metadata (or data) he wants to load into his service. In this case, the odbcGrammar shall be used.   
• If all previous cases don't apply: The data provider should use a customGrammar to load the metadata into DaCHS, through a dedicated python script. 

We show below a simple example with CSV files available from the resource descriptor directory. 

<data id="import">
	<!--	Define where to retrieve the data	-->
	<sources>
		<!-- Pattern is used when there are multiple source files -->
		<!-- (here all the .csv files, and a data directory next to the q.rd file) -->
		<pattern>data/*.csv</pattern>
	</sources>

	<!-- we use the csvGrammar on the files defined in sources --> 
	<csvGrammar/>

	<!-- now we send the data to the epn_core table -->
	<make table="epn_core">

		<!-- Inserts the data of each row made by the grammar in its column	-->
		<rowmaker idmaps="*">
		<!-- idmaps="*" implies that any CSV columns with the same name as the epn_core column is mapped without processing --> 

			<!--	Insert non-varying data	-->
			<var key="target_name">"Mars"</var>
			<var key="service_title">"\schema"</var>
			[...]
			<!-- Bind the columns required by EPN-TAP	-->
			<apply procDef="//epntap2#populate-2_0" name="fillepn">
				<bind key="granule_uid">@granule_uid</bind>
				<bind key="granule_gid">@granule_gid</bind>
				<bind key="obs_id">@obs_id</bind>
				[...]
			</apply>
		</rowmaker>
	</make>
</data>

We list below a series of repositories using various grammar types: