VESPA EPN-TAP Library specification

EPN2020-RI

EUROPLANET2020 Research Infrastructure 

H2020-INFRAIA-2014-2015 

Grant agreement no: 654208

Document: VESPA-WP11-2-008-SS-v1.0(92)

VESPA EPN-TAP Library specification

Date: 2020-03-04

      List of IVOA Registries providing a RegTAP interface

          http://registry.euro-vo.org/regtap/tap

          http://gavo.aip.de/tap

          http://reg.g-vo.org/tap

          http://dc.zah.uni-heidelberg.de/tap

          http://gavo.aip.de/tap

Definition of terms

• Library - The software library providing access to the VESPA services using TAP.
• Client - The software hosting the Library, which is usually a visualization tool.
• User - The real user accessing the Client to do some scientific analysis.
• Scene - The configuration of the visualization panel (temporal range, spectral range, target...)

Scenario #1: Query all VESPA services known by the IVOA Registry (Query created by the client)

In the Client (AMDA, 3Dview, CASSIS, PlanetServer...), the User has set up a Scene, by selecting for example a time span and a target (target_name and target_class). He wants to discover data resources related to this Scene. The Client provides a "Find Data in VESPA" capability. When triggered, the Client internally executes the following steps:

• Step 1. The Client calls getEPNServices(), to retrieve an up-to-date list of all EPN-TAP services in the IVOA Registry. getEPNServices() issues a RegTAP query to the IVOA Registry, searching for TAP services with EPNcore data model, and returns a VOTable containing metadata related to these services. The Client may display the list of services (in a simplified version, including the name of the service and its description, for instance) and may allow the User to select which services he wants to query.
• Step 2. The Client calls sendQuery(TAPURL, schema_name,Query), to query each EPN-TAP service previously selected, using the access URL and schema_name provided by getEPNServices()  and specifying the search parameters as set in the Scene. sendQuery(TAPURL,schema_name,Query) builds  the ADQL query with the parameters given by the Client  and issues a TAP query to each service with this ADQL query. sendQuery(TAPURL, schema_name,Query) returns a  VOTable containing the selected rows for each EPN-TAP service.  
• Step 3. The Client displays the list of results in a form adapted with its scope and interface. The interface showing the results shall allow the User to either download the data, use them in the Client (if possible), or send them to SAMP-enabled software.

Scenario #1bis: Query all VESPA services known by the IVOA Registry (ADQL Query created by the client)

The difference between scenarios #1 and #1bis is the creation of the ADQL Query. In #1bis, the ADQL query is created by the Client whereas in #1, it is created by the Library 

In the Client (AMDA, 3Dview, CASSIS, PlanetServer...), the User has set up a Scene, by selecting for example a time span and a target (target_name and target_class). He wants to discover data resources related to this Scene. The Clientprovides a "Find Data in VESPA" capability. When triggered, the Client internally executes the following steps:

• Step 1. The Client calls getEPNServices(), to retrieve an up-to-date list of all EPN-TAP services in the IVOA Registry. getEPNServices() issues a RegTAP query to the IVOA Registry, searching for TAP services with EPNcore data model, and returns a VOTable containing  metadata related to these services. The Client may display the list of services (in a simplified version, including the name of the service and its description, for instance) and may allow the User to select which services he wants to query.
• Step 2. For each service previously selected, the Client creates an ADQL query and calls sendADQLQuery(TAPURL, schema_name,ADQLQuery), to query each EPN-TAP service previously selected, using the access URL and schema_name provided by getEPNServices()  and specifying the search parameters as set in  the Scene. sendADQLQuery(TAPURL,schema_name,ADQLQuery) builds  the ADQL query with the parameters given by the Client  , issues a TAP query to each service with this ADQL query, and returns a  VOTable containing the selected rows for each EPN-TAP service.  
• Step 3. The Client displays the list of results in a form adapted with its scope and interface. The interface showing the results shall allow the User to either download the data, use them in the Client (if possible), or send them to SAMP-enabled software.

Scenario #2: Query a single VESPA service known by the client and the IVOA Registry

In this scenario, we assume that the client  knows the iovID of the service he wants to query; this service is already registered in the IVOA Registry. In the Client (AMDA, 3Dview, CASSIS, PlanetServer...), the User has set up a Scene (idem scenario #1). The Client proposes to access  remote data from a specific VESPA service, that the Client can ingest automatically. The Client provides an "Add remote Data from VESPA" capability. When triggered, the Client internally executes the following steps:

• Step 1. The Client calls getEPNService(ivoID), to retrieve the IVOA registry record corresponding to the specific EPN-TAP service, using the known "ivoID" of this service. getEPNService(ivoID) issues a RegTAP query to the IVOA registry, searching for the service corresponding to this ivo-id, and returns a VOTable containing the description of this service. 
• Step 2. The Client calls  sendQuery(TAPURL,schema_name,Query), to query the specific EPN-TAP service, using the access URL provided by getEPNService(ivoID)  and specifying the search parameters as set in the Scene. sendQuery(TAPURL,schema_name,Query) builds an ADQL query corresponding to the parameters of the scene , issues a TAP query to the service with this ADQL query,and returns a VOTable containing the selected rows for the EPN-TAP service.  
• Step 3. The Client displays the results in a form adapted with its scope and interface.

Scenario #2bis: Query a single VESPA service known by the IVOA Registry

In this scenario, the TAP service URL and the Schema name are obtained using specific methods, where as in Scenario #2, a single method provides the metadata, and the Client has to extract the TAP service URL and Schema name on its own.
In the Client (AMDA, 3Dview, CASSIS, PlanetServer...), the User has set up a Scene (idem scenario #1). The Client proposes to access  remote data from a specific VESPA service, that the Client can ingest automatically. The Client provides an "Add remote Data from VESPA" capability. When triggered, the Client internally executes the following steps:

• Step 1. The Client calls getTAPURL(service_ivoid) and getEPNCoreTableName(service_ivoid), to retrieve the AccessURL and schema_name of the service service, using its known "ivoID". 
• Step 2. The Client calls sendQuery(TAPURL,schema_name,Query), to query the specific EPN-TAP service. sendQuery(TAPURL,schema_name,Query) builds an ADQL query corresponding to the parameters of the scene , issues a TAP query to the service with this ADQL query, and returns a VOTable containing the selected rows for the EPN-TAP service.  
• Step 3. The Client displays the results in a form adapted with its scope and interface.

Scenario #3: Query a single VESPA service unknown by the IVOA Registry

In the Client (AMDA, 3Dview, CASSIS, PlanetServer...), the User has set up a Scene (idem scenario #1). The Client proposes to test a new VESPA service not yet  in the Registry, and knows its accessURL and schema_name. The Client provides a "Query Custom VESPA service " capability. When triggered, the Client internally executes the following steps:

• Step 1. The Client calls sendQuery(TAPURL,schema_name,Query)  to query the specific EPN-TAP service, using the access URL and schema name provided by the User and specifying the search parameters as set in the Scene.  sendQuery(TAPURL,schema_name,Query) builds an ADQL query corresponding to the parameters of the scene , issues a TAP query to the service with this ADQL query,and returns a VOTable containing the selected rows for the EPN-TAP service.  
• Step 2. The Client displays the results in a form adapted with its scope and interface.

This scenario is similar to the "Custom Resource" page of the generic VESPA client: http://vespa.obspm.fr/planetary/data/epn/query/resource/

Scenario #4: Query a subset of VESPA services known by the IVOA Registry

In the Client (AMDA, 3Dview, CASSIS, PlanetServer...), the User has set up a Scene (as in scenario #1). The Client sets up a window in which users are able to select services they want to query, according to one or more keywords ( plain text). This is similar to the capability provided by TopCat.

• Step 1. The Client calls getEPNServices(keywords), to query the Registry (RegTAP interface) to search for EPN-TAP services, according to one or more keywords. getEPNServices(keywords) returns a VOTable containing metadata related to these services. The Client may display the list of services (the name of the service, its ivoID and its description, for instance) and may allow the User to select which services he wants to query.   
• Step 2. The Client calls sendQuery(TAPURL, schema_name,Query) to query each EPN-TAP service previously selected, using the access URL and schema_name provided by getEPNServices(keywords)  and specifying the search parameters as set in the Scene. sendQuery(TAPURL,schema_name,Query) builds  the ADQL query with the parameters given by the Client, issues a TAP query to each service with this ADQL query, and returns a  VOTable containing the selected rows for each EPN-TAP service.  
• Step 3. The Client displays the list of results in a form adapted with its scope and interface. The interface showing the results shall allow the User to select  the data, use them in the Client (if possible), or send them to SAMP-enabled software.

EPN-TAP Requests

The goal of an EPN-TAP query is to get the values of some parameters in a table. Therefore, the arguments of the query are the columns of this table. The arguments are used to filter the contents of the database. The response of the service contains only the rows of the table which correspond to these arguments.

An EPN-TAP Request is sent using HTTP GET or POST

The accessURL of the service can be found in the resources/capabilities/accessurl element ( see below )

for example, TAPURL = http://voparis-tap.obspm.fr/__system__/tap/run/tap

Add the part dedicated to the Query itself

Synchronous mode:

TAPURL/sync?QueryLanguage+query

Asynchronous mode:

TAPURL/async?QueryLanguage+query

Examples

       TAPURL = http://voparis-tap.obspm.fr/__system__/tap/run/tap

        HTTP GET Queries

           TAPURL/sync?request=doQuery&lang=ADQL&query=select top 100 * from apis.epn_core

          returns the first hundred granules from APIS

           TAPURL/sync/request=doquery&lang=adql&query=select * from apis.epn_core where time_min > '2455197.5' and time_max < '2455927.5'  

           returns from APIS the list of granules from 01/01/2010 (  2455197.5 ) to 01/01/2012 ( 2455927.5 )

        HTTP POST Queries

        this query is sent with the Unix curl command

         curl --data "REQUEST=doQuery&LANG=ADQL&QUERY=select * from amdadb.epn_core where instrument_host_name='Pioneer_10'"
                http://cdpp-epn-tap.cesr.fr/__system__/tap/run/tap/sync

         returns the list of granules which contain data from the Pioneer 10 Spacecraft

Example of VOResource element

<resources>
            <status>active</status>
            <publisher>IA2</publisher>
            <updated>2013-12-05T08:30:00Z</updated>
            <contentlevel>Research</contentlevel>
            <description>
                The Cosmic dust catalog is an internal resource of the SBDN, since we have internally developed original services to access this catalogs. NASA's Cosmic dust catalog 15 and 18 have                   been joined to obtain this service. 467 (from catalog 15) plus 957 (from catalog 18) dust grains with their main characteristics, images and X-ray spectra are listed. Not only cosmic dust                   particles are listed, but also terrestrial contamination (natural), terrestrial contamination (artificial) and aluminium oxide spheres.
            </description>
            <contributors>INAF-IAPS</contributors>
            <creator>IA2</creator>
            <provenance>ivo://esavo/registry</provenance>
            <referenceurl>
                 http://www.iasf-roma.inaf.it:8080/web/sbdn/cosmic-dust-catalog
            </referenceurl>
            <created>2013-02-20T08:00:49Z</created>
            <contactemail>ia2@oats.inaf.it</contactemail>
            <subjects>dust catalogue</subjects>
            <capabilities>
                  <accessurl>http://ia2-tap.oats.inaf.it:8080/epntap</accessurl>
                  <standardid>ivo://ivoa.net/std/TAP</standardid>
                  <datamodel>EpnCore-1.0</datamodel>
            </capabilities>
            <contactname>IA2</contactname>
            <title>INAF-IAPS RDB NASA dust catalogue TAP service</title>
            <shortname>nasadustcat</shortname>
            <identifier>
                ivo://ia2.inaf.it/hosted/iaps/epn/tap/nasadustcat
            </identifier>
            <type>CatalogService</type>
</resources>