...
Setting-up an EPN-TAP service means publishing a table in a DACHS server server (DACHS for this tutorial) describing data we want to displaydistribute. This table is referencing each file piece of information with a maximum amount of information attributes by filling parameters defined by the standards of EPN-TAP called epn_core view. The EPN-TAP standards have mandatory parameters (most of them could be left empty) and optional parameters, finally additional columns parameters could be defined to provide specific information. VESPA EPN-TAP is a way to interrogate registered services published in different DaCHS servers using keywords parameters or ADQL queries.
This tutorial will provide a very simple example to help beginners in understanding how they can set up an EPN-TAP service. In this purpose, we will set up a service called "Planets" providing information on the solar system planets. It will only document the method using a CSV (Comma Separated Values) file to fill the table.
...
First, a virtual machine hosting a DaCHS server must be settled set up. For simple services like this one, metadata must can be referenced in a CSV file containing useful featuresinformation to include, it is also possible to use a Python script to generate metadata. Then, a Resource Descriptor (RD) must be written in XML. This RD aims to define columns, read the CSV file and fill a table on DaCHS server. Finally, the service must be registered on VESPA portalthe IVOA registry which makes the link between EPN-TAP services and VESPA query interface.
Install the server on a virtual machine
First, it is necessary to have settled-up an EPN-TAP server hosting the . The method to set up a virtual machine hosting a DaCHS server . Method is described in this tutorial: EPN-TAP Server Installation for VESPA Data Provider Tutorial#TAPServerInstallationforVESPADataProviderTutorial-6Tutorial .AWStatsInstallationandConfigurationincludingApache .
Define granules
To publish your own service, you have to define granules. Each granule can link to a file (. Granules correspond to table rows, they represent the smallest piece of information accessible in the service. A granule can be one file (linked with the optional parameter access_url) or a set of parameters described into the table. A granule must have one unique identifier which is a primary key for the table (the mandatory parameter for this identifier is granule_uid). Each different type of granule must have an identifier (granule_gid parameter) so it is possible to group data by type.
In our example, each granule has the same type and corresponds to one planet. So, basically the granule_gid and granule_uid will be respectively "Planet" and the name of the planet. There is no need to define an access_url because for this example, there is no file linked for this example, each granule corresponds to a set of parameters.
Define the epn_core parameters
...
Metadatas which are varying from one granule to the other or cannot be post-treated in the resource descriptor must be referenced in a CSV file . The (if this is the method chosen). The first row has to be the column's names, then each row will set the values for a granule. This CSV file will further be read by the RD to fill the table in DaCHS. You can use your favourite programming language to create this CSV file.
For this example, we give the link to a hand-written CSV, you can download it and the associated Resource Descriptor on from github:
https://github.com/epn-vespa/DaCHS-for-VESPA/tree/master/q.rd_examples/planets
...
The Resource Descriptor (RD) is an XML-written file named q.rd which gives DaCHS the way to fill the table from the CSV file, it follows DACHS Standards for EPN-TAP. Other informations information on the RD building are is available here.
Structure of the Resource Descriptor
...
<resource schema="...">
<meta .../>
...
<meta .../>
<table ...>
<mixin .../>
<column .../>
...
<column .../>
</table>
<data id="import">
<sources .../>
<csvGrammar> <rowfilter procDef="//products#define"> <bind name="table">"\schema.epn_core"</bind> </rowfilter> </csvGrammar>
<make table="epn_core">
<rowmaker idmaps="*">
<var key="...">...</var>
...
<var key="...">...</var>
<apply procDef="//epntap2#populate-2_0" name="fillepn">
<bind name="...">@...</bind>
...
<bind name="...">@...</bind>
</apply>
</rowmaker>
</make>
</data>
</resource>
All the file is contained in the The tag <resource> encompasses the others. First, <meta> data are filled, then, <table> is defined containing <mixin> reference and <column> elements defining extra-columns. Into the tag <data>, data ingestion rules are set, the path of the sourcefile is defined into <sources> and the <csv grammar> is specified. The <make><rowmaker> content describes how the mandatory and added columns will be filled. <var> attributes set columns values while <bind> attributes in the <apply> tag link columns to its values and fill the table.
...
The first part is a set of meta tags with different attributes which defines global characteristics of the table. Meta tags aim to describe the service in the registry.
<resource schema="planets">
<meta name="title">Characteristics of Planets (demo)</meta>
<meta name="description" format="plain">
Main characteristics of planets. Data are included in the table, therefore most relevant parameters are non-standard in EPN-TAP. Data are retrieved from Archinal et al 2009 (IAU report, 2011CeMDA.109..101A) [radii] and Cox et al 2000 (Allen's astrophysical quantities, 2000asqu.book.....C) [masses, heliocentric distances, and rotation periods]. </meta>
<meta name="creationDate">2015-08-16T09:42:00Z</meta>
<meta name="subject">planet</meta>
<meta name="subject">mass</meta>
<meta name="subject">radius</meta>
<meta name="subject">period</meta>
<meta name="copyright">LESIA-Obs Paris</meta>
<meta name="creator.name">Stephane Erard</meta>
<meta name="publisher">Paris Astronomical Data Centre - LESIA</meta>
<meta name="contact.name">Stephane Erard</meta>
<meta name="contact.email">vo.paris@obspm.fr</meta>
<meta name="contact.address">Observatoire de Paris VOPDC, bat. Perrault, 77 av. Denfert Rochereau, 75014 Paris, FRANCE</meta>
<meta name="source">2000asqu.book.....C</meta>
<meta name="contentLevel">General</meta>
<meta name="contentLevel">University</meta>
<meta name="contentLevel">Research</meta>
<meta name="contentLevel">Amateur</meta>
<meta name="utype">ivo://vopdc.obspm/std/EpnCore#schema-2.0</meta>
...
Meta attribute "subject" is defined several times by different keywords defining data . "source" refers to the resource-related from UAT (Unified Astronomy Thesaurus). At least one of them must refer to a global topic listed in this page. In the context of VESPA, the 3 appropriate global topics are : "Exoplanet astronomy", "Solar physics" and "Solar system astronomy". The attribute "source" refers to the resource-related paper. Here, "contentLevel" takes the four values "General", "University", "Research", "Amateur" but it could take only some of these.
Table definition
To be well defined, the resource descriptor of our example planets_mixin_q.rd file must be modified by adding the meta attribute <meta name="subject">"Solar system astronomy"</meta> .
Table definition
Then, <table> definition starts, in every EPN-TAP services, the table <id> and <mixin> must respectively Then, <table> definition starts, in every EPN-TAP services, the table <id> and <mixin> must respectively take the values "epn_core" and " //epntap2#table-2.0".
spatial_frame_type attribute defines the type of coordinate system for the defined granules, it could take several values (listed in spatial_frame_type section of EPN TAP V2 parameter description, except the value "none") its choice will impact the coordinates definition (parameters c1, c2 and c3).
You may list predefined optional EPN-TAP parameters you have chosen to add in the The mixin "//epntap2#table-2_0" provides a standard definition of mandatory parameters and some optional ones. Mandatory parameters will be automatically present in the table and you may list predefined optional columns you want to add in optional_columns attribute. Then, additional columns could be defined manually in <column> tags.
Here, the spatial_frame_type is defined as celestial which means coordinate definition as is ICRS one (not relevant here so it will not be informed below). Optional columns time, <mixin spatial_frame_type="none" would be a better definition in this context). The tree optional columns time_scale, publisher and bib_reference have been added to the table in this example.
<table id="epn_core" onDisk="true" adql="True">
<mixin spatial_frame_type="celestial"
optional_columns= "time_scale publisher bib_reference" >//epntap2#table-2_0</mixin>
After mixin definition, you can start extra-parameters definition with the tag <column>. To do that, you should define the attributes name, type, tablehead, unit (if relevant, listed here), description, ucd (a set of keywords which defines the type of data, see ucd IVOA documentation ) and verblevel (a rate under 30 defining the columns importance). After extra-columns are set, the table definition is complete.
...
The <var> elements associate values to the fields columns which has not been filled automatically by the rowfilter or need post-processing :
...
After bind elements definition, the table construction is finished and the service could be published.
- Publish
Install and check tables
To publish install your service on your DaCHS server, you have to create a directory on your EPN-TAP server containing your q.rd RD and your CSV file into gavodachs directory : /var/gavo/inputs/{servicename}, where servicename is the same as <resource schema=".."> value into the RD. In this directory, the RD must be named q.rd.
For our example, in your "planets" work directory in sudo mode, type:connect to your server and, into the gavodachs directory, create the directory "planets" and a "data" subdirectory :
$ sudo mkdir
# mkdir/var/gavo/inputs/planets
#$ sudo mkdir
/var/gavo/inputs/planets/data
Then go to the directory in which you have downloaded the RD and the CSV of the example "planets" from github (here we assume its path is ~/planets) and copy these files into the directory previously created:
$ cd ~/planets
# cp planets_mixin_q.rd
~/planets$ sudo cp planets_mixin_q.rd/var/gavo/inputs/planets/q.rd
# cp~/planets$ sudo cp Masses2.csv
/var/gavo/inputs/planets/data/Masses2.csv
Then, open this go to your gavodachs resource directory
# $ cd /var/gavo/inputs/planets/
And Now, you have to check the syntax of your RD with the command gavo val :
/var/gavo/inputs/planets/$ sudo gavo val q.rd
It returns "q.rd – OK" if the syntax of your RD is correct.
When the syntax is correct. If it returns "OK", you can import your service on your DaCHS server with the command gavo imp, if no error is raised, "Rows affected : {...}" will appear, then gavo imp :
/var/gavo/inputs/planets/$ sudo gavo imp q.rd
If the service is correctly imported, the following message will appear
Making data planets/q#import
Starting /var/gavo/inputs/planets/data/Masses2.csv
Done /var/gavo/inputs/planets/data/Masses2.csv, read 8
Shipped 8/8
Then, you can restart the server to check the newly settled-up service with gavo serve restart :
...
/var/gavo/inputs/planets/$ gavo serve restart
The 3 steps (gavo val q.rd
...
, gavo imp q.rd
...
and gavo serve restart
...
These 3 steps ) are necessary each time the q.rd RD is modified.
After that, on your browser, type the address :
...
To make your own service available on VESPA portal, you may first follow this tutorial : Registering your VESPA EPN-TAP Server. It consists in configuration and publication of your DaCHS server in the IVOA Registry of Registries (RoR), and publication of your TAP service in order to make it available in . Finally, the VESPA team has to review this new service before making it available on VESPA Query Interface.
The service Planets may not be registered because it is already available in VESPA.
...