You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Article under developpement, not already finished and reviewed do not take it into account for now

Setting-up an EPN-TAP service mean to publish a table in a DACHS server describing data we want to display. This table is referencing each file with a maximum amount of information by filling parameters defined by the standards of EPN TAP called epn core view . The EPN TAP standards has mandatory parameters wich can (except a few of them) be left empty and optional parameters, finally other columns could be defined by users to provide specific information. VESPA is a way to interrogate registered services published in different DaCHS servers using keywords or ADQL queries.

This tutorial will provide a very simple example to help beginners to understand how they can set up an EPN-TAP service. In this purpose, we will set up a service called "Planets" providing informations on the solar system planets.

For this example, we want to display these data on our service :

namemean radius (km)

mean radius uncertainty


equatorial radius


equatorial radius



polar radius

















distance to








Steps to follow:

First, a virtual machine hosting a DaCHS server must be setteled up. For simple services like this one, metadata must be referrenced and a Comma Separated Values CSV file containing useful features must be created, other methods are available. Then a Ressource Descriptor in XML must be written. This resource descritor will read the csv file and fill the table on DaCHS server. Finally, the service must be registered on VESPA portal (not for this tuto).

  • Install the server on a virtual machine:

First, it is necessary to have settled-up an EPN-TAP server hosting the DaCHS server, following this tutorial: EPN-TAP Server Installation for VESPA Data Provider Tutorial#TAPServerInstallationforVESPADataProviderTutorial-6.AWStatsInstallationandConfigurationincludingApache .

  • Define granules :

To publish your own service, you have to define granules. Each granule can link to a file (with the optional parameter access_url). Each granule must have one unique identifier which is a primary key  for the table (the mandatory parameter for this identifier is granule_uid). Each different type of granule must have an identifier (parameter granule_gid).

In our example, each granule have the same type and correspond to one planet. So, basically the granule_gid and granule_uid will be "Planet" and the name of the planet.

  • Define the epn_core parameters

Once granules are defined, you have to browse the EPN TAP V2 parameters and their description to see what could be informed. In order to define clearly the organization of the final table, it is advised to fill a scheme with the EPN-TAP parameters on one side an what you decide to put into each in the other side available here in xls format. A large part of the mandatory parameters could be left empty, other parameters  you want to add could be defined as table columns.

  • Create a CSV file containing metadata

Metadatas which are varying from one granule to the other or cannot be post-treaded in the q.rd must be referenced in one column of a CSV file. The first row has to be the column's names, then each row will set the values for a granule. This CSV file will further be read by the resource descriptor to fill the tables in  DaCHS. You can use your favourite programming language to create this CSV file.

For this example, we give the link to the hand written CSV,  you can downolad it and the  associated Resource Descriptor on github:

  • Building a resource descriptor

The resource descriptor is an XML-written file which gives DaCHS the way to read the CSV (in case of the CSV method) and fill the metadata table, it follows DACHS Standards for EPN TAP.

Structure of the q.rd file

For this example, the q.rd file previously downloaded has this structure :

All the file is contained in the tag resource, after that meta tags are defined, then the creation of the table by columns definition from the csv file.

After that , into the tag data are defined the path of the sourcefile, the rowfilter, the tag make.

Into the rowmaker, the element var aim to define

<resource schema="...">

<meta ... />
<meta ... />

<table ...>

<mixin .../>

<column .../>
<column .../>


<data id="import">

<sources ... />

<csvGrammar> <rowfilter procDef="//products#define"> <bind name="table">"\schema.epn_core"</bind> </rowfilter> </csvGrammar>

<make table="epn_core">

<rowmaker idmaps="*">

<var key="...">...</var>
<var key="...">...</var>

<apply procDef="//epntap2#populate-2_0" name="fillepn">

<bind name="...">@...</bind>
<bind name="...">@...</bind>


It is most commonly divided in 3 parts, all the content is containes in one global tag <resource schema="...."> where the resource schema name must be the same as the name of the folder containing the q.rd . 

The first part is meta tags with diffecrent attributes which defines global characteristics of the table.

The meta attribute "subject" could and must be defined several times .

utype meta attribute has to be ivo://vopdc.obspm/std/EpnCore#schema-2.0

<resource schema="planets">
<meta name="title">Characteristics of Planets (demo)</meta>
<meta name="description" format="plain">
Main characteristics of planets. Data are included in the table, therefore most relevant parameters are non-standard in EPN-TAP. Data are retrieved from Archinal et al 2009 (IAU report, 2011CeMDA.109..101A) [radii] and Cox et al 2000 (Allen's astrophysical quantities, [masses, heliocentric distances, and rotation periods]. </meta>
<meta name="creationDate">2015-08-16T09:42:00Z</meta>
<meta name="subject">planet</meta>
<meta name="subject">mass</meta>
<meta name="subject">radius</meta>
<meta name="subject">period</meta>
<meta name="copyright">LESIA-Obs Paris</meta>
<meta name="">Stephane Erard</meta>
<meta name="publisher">Paris Astronomical Data Centre - LESIA</meta>
<meta name="">Stephane Erard</meta>
<meta name=""></meta>
<meta name="contact.address">Observatoire de Paris VOPDC, bat. Perrault, 77 av. Denfert Rochereau, 75014 Paris, FRANCE</meta>
<meta name="source"></meta>
<meta name="contentLevel">General</meta>
<meta name="contentLevel">University</meta>
<meta name="contentLevel">Research</meta>
<meta name="contentLevel">Amateur</meta>
<meta name="utype">ivo://vopdc.obspm/std/EpnCore#schema-2.0</meta>

Then, you start defining the table, in every EPN TAP services, the table id must be epn_core and he mixin must be //epntap2#table-2.0, spatial frame type must be chosen in EPN TAP parameters

You can list in the optional_columns tags, optional EPN TAP parameters you choose to inform.

   <table id="epn_core" onDisk="true" adql="True">

      <mixin spatial_frame_type="celestial"
      optional_columns= "time_scale publisher bib_reference" >//epntap2#table-2_0</mixin>

After mixin definition,  you can start to define your own columns  from the CSV file with the tag <column>, where you should define the attributesname, type, tablehead, unit (if relevant), description, ucd (defines the type of data, see ucd  IVOA documentation ), verblevel is a rate under 30 defining the columns importance. After columns definition, the <table> tag is complete.

      <column name="distance_to_primary" type="double precision"
tablehead="Distance_to_primary" unit="km"
description="Extra: Mean heliocentric distance (semi-major axis)"
 <column name= ... />
<column name= ... />



   <data id="import">
<rowfilter procDef="//products#define">
<bind name="table">"\schema.epn_core"</bind>

<make table="epn_core">
<rowmaker idmaps="*">
<var key="dataproduct_type">"ci"</var>
<var key="spatial_frame_type">"celestial"</var>

<var key="target_name" source="target_name" />
<var key="granule_uid" source="target_name" />
<var key="granule_gid">"Planet" </var>
<var key="obs_id" source="obs_id" />

<var key="distance_to_primary" source="distance_to_primary" />

<var key="creation_date">"2015-08-20T07:54:00.00" </var>
<var key="modification_date">"2017-12-15T17:54:00.00" </var>
<var key="release_date">"2015-08-20T07:54:00.00" </var>

<var key="service_title">"planets" </var>
<var key="bib_reference">""</var>
<var key="publisher">"LESIA" </var>

<apply procDef="//epntap2#populate-2_0" name="fillepn">
<bind name="granule_gid">@granule_gid</bind>
<bind name="granule_uid">@granule_uid</bind>
<bind name="obs_id">@obs_id</bind>
<bind name="target_class">"planet"</bind>
<bind name="time_scale">"UTC"</bind>

<bind name="target_name">@target_name</bind>
<!-- <bind name="access_format">""</bind> -->
<bind name="instrument_host_name">""</bind>
<bind name="instrument_name">""</bind>

<bind key="processing_level">5</bind>

<bind name="dataproduct_type">@dataproduct_type</bind>
<bind name="measurement_type">"phys.mass#phys.size.radius"</bind>

<bind name="service_title">@service_title</bind>
<bind name="creation_date">@creation_date</bind>
<bind name="modification_date">@modification_date</bind>
<bind name="release_date">@release_date</bind>


  • Publish and check the tables

You have to create a directory  containing your q.rd and your csv file in gavodachs directory : /var/gavo/inputs/{servicename}, the resource descriptor in this directory must be named q.rd.

For our example, in your "planets" work directory,  in sudo mode type:

# mkdir /var/gavo/inputs/planets
# mkdir /var/gavo/inputs/planets/data
# cp planets_mixin.rd  /var/gavo/inputs/planets/
# cp Masses2.csv /var/gavo/inputs/planets/data/Masses2.csv

then open this directory

# cd  /var/gavo/inputs/planets/

and check the grammar of your csv and publish your service on your DaCHS server:

# gavo val q.rd

# gavo imp q.rd

# gavo serve restart

You will have to follow these 3 steps each time you change your q.rd or csv file. It is advised to work in another directory for security.

After that, on your browser, type the adress  http://<<my_servername>>.<<my_domain>>:8000/  previsously setteled-up during your server installation.

Click on the service name, you can now see your table informations. It is possible to send ADQL queries. If you want to select the whole database, you can type:

SELECT * FROM planets.epn_core

You can now check up how the table apprears and check if the rows are correctly filled.

  • Register the service

We will not follow this step for the service "Planets", because this service planets is already available in VESPA.

To register your own service in the RoR, you may follow this tutorial : Registering your VESPA EPN-TAP Server when you have finished the build of your RD and published.

Once your service has been published in your DaCHS server,  you may register in in the IVOA Registry of Registries , RoR to make it available in VESPA Query Interface.

To go further ...

An other example with iks service

add a link to a thumnail

programming in python

  • No labels