Page History

List of EPN-TAP parameters
v2, 3/8/2015 (SE)
v3, 29/8/2015 (SE, integrating comments by PLS and BC)
v4, 7/9/2015 (SE, integrating comments by PLS)
v5, 7/10/2015 (SE, integrating comments by PLS: time_scale UCD)
v6, 7/11/2015 (SE, integrating comments by PLS/BC)
v7, 14/12/2015 (SE, added point 8, clarification of an older idea)
v8, 24/12/2015 (SE, review/integration of comments/corrections)
v9, 5-7/01/2016 (SE/BC, including comments from Chiara Marmo and Michel Gangloff)

Main evolution relative to v1

1) The previous notion of dataset is deprecated

This was complex to handle in the database, and in general not relevant for the services.

2) Grouping of products is rationalized in version 2

A granule is still a record/line in the epn_core view database, and corresponds to the smallest data unit described by the service.

A "product" is typically a data file, or a service output, that can be reached through a URL

In version 2 both concepts coincide, while in v1 the same granule could be composed of several products related to the same initial data (an "observation")

In EPN-TAP v2, granules are referenced by 3 parameters:

• granule_uid provides a unique ID for the granule in the service, ie for each line in the epn_core view. It is equivalent to the previous index parameter (index is a reserved word in many database languages and should not have been used in the first place)
• granule_gid is related to a type of product: it is identical for all granules containing the same type of information for different observations (e.g., calibrated files). An explicit string is recommanded in this field, with suggested standard values (e.g., preview/native/calibrated/geometry/projection…).
• obs_id is related to an observation: it is identical for all granules related to the same observation, containing different type of data (e.g.: raw and calibrated data, associated geometry, etc). In many EPN-TAP v1 services, such products were described together on the same line with a unique index parameter.

These 3 parameters can be arbitrary alphanumeric strings — see example application to APIS service below.

In practice, different products related to the same observation are no longer described together on a single line of the epn_core view, but on successive lines associated by the same obs_id, each with a different granule_gid (and a specific granule_uid, see Table 2 below).

Each line in the epn_core view must describe only one product (plus a thumbnail wherever relevant). The notion of "main product" (which was more or less explicit in v1) is therefore deprecated, and the epn_core view in v2 includes more lines than in v1. Although less compact that the previous table presentation, this list presentation is much more efficient for machine-handling, and easier to design.

3) The notion of table/service parameters is deprecated

This was not supported by TAP. Therefore, even constant parameters must be replicated in every line of the epn_core view.

The corresponding parameters may be duplicated in the registry declaration though, to provide a fast description of services.

4) Footprints can be provided through s_region

s_region is a parameter in the TAP standard of IVOA, and ADQL allows for powerful query functions such as intersections.

We have to study a possible extension of its use from regions of the sky to body-related coordinates (on ellipsoids or 3D shape models) .

The use in the context of GIS also has to be studied.

5) Better handling of the evolution of services

creation_date and modification_date are now mandatory parameters for every granule. The latter is intended to allow for fast mirroring of services.

From our first discussions, the best option may be to use an ISO 8601 string with format "2013-11-17T10:41:00.00+01:00" (we don't want any space in the string) — TBC with ADQL capacities, in particular the support of time zones.

6) Support of coordinated observations

The target_time parameter now provides observation time in the reference frame of the target. This is intended to facilitate the cross-correlation of observations from different locations, e.g., telescopic observations in support of space missions, or multi-spacecraft campaigns.

7) Axes ranges

All parameters defining a range are now introduced with a min and max value.

All floating point parameters are now in double precision (to prevents stupid errors).

8) Thematic extensions

Some science fields will require optional parameters, which need to be used consistently between services addressing the same field. Such extensions have to be designed by sub-groups involved in the corresponding data services, either as providers or users. This includes:

• Lab spectroscopy: parameters to describe mineralogical samples (and possibly other samples)
• Orbital/rotational parameters and physical properties of Solar System bodies (TBC, but requested by some)
• Planetary modelling run results
• Exoplanets / planetary systems properties

+ See notes below the table.

(1): depending on context (as given by spatial_frame_type)

...

Other modifications

...

• UCDs: to be reviewed against PDS4 and IPDA, and completed

• Processing levels: to be reviewed against PDS4 (again)

• min vs max :
if only one value available, it must appear in both fields

• Optional parameters: they come in sets that are logically related; if one is present, the related ones must be present also (e.g., 3 access_* parameters)

• Granule_gid : any general indication to providers? I.e.: preview, native, calibrated, geometry… 
A client should be able to display the values from a service, TBC

...

• Mandatory :
  processing_level -mandatory
  publisher -mandatory
  service_title –mandatory
• Optional - TBC
  spatial_coordinate_description  (implicitly body-fixed or J2000)
  spatial_origin  (implicitly body center or SS barycenter? Or observer location)
  time_origin  (implicitly: observer)
  time_scale (implicitly : UTC – no other values allowed in data services? [only in computational services, e.g. ephemeris])
  Same values to be used in registry declaration 

Example of V1 to V2 conversion with APIS database:

EPNcore Table v1

(was not really compliant…)

(refreshed/completed April 2019, Oct 2020, June 2021, Oct 2021) (SE) 

You can use this file to keep track of your service parameters: EPN-TAP_parameters_List_template.xlsx

EPN-TAP

• EPN-TAP is a VO access protocol dedicated to Planetary Science data. It is based on the TAP mechanism from IVOA, completed with sets of parameters and associated lists of values. In this regard, it is similar to ObsTAP but with a different scope.
• EPN-TAP version 2 is a major update of the protocol to accommodate larger services and simplify setup and use of data services. All parameters are described here.

Parameters which must be provided

are now clearly identified - those are not only mandatory, they also must provide a value.
They are mostly related to service description and granule identification.

+ See notes below the table.

Thematic extensions

Some science fields will require optional parameters, which need to be used consistently between services addressing the same field. Such extensions have to be designed by sub-groups involved in the corresponding data services, either as providers or users. This includes:

• Lab spectroscopy: parameters to describe mineralogical samples (and possibly other samples)
• Solar System objects: covers orbital/rotational parameters, physical properties, and taxonomy
• APIS: for consistency with APIS service. Contains parameters for observing programs (most parameters are actually included in other extensions)
• Contributive works / observing programs: enlargement of APIS extension to other data
• Exoplanets / planetary systems properties
• Map extension (to be enlarged)
• Events: covers the VOevent standard and other types of events
• Particle spectroscopy (to be finalized)
• Results of planetary 3D modelling run (in progress)
• Bibliographic entries? May be manageable otherwise, through bibcode / doi interpretation

Support file

You can use this file to keep track of your service parameters: EPN-TAP_parameters_List_template.xlsx

(1): depending on context (as given by spatial_frame_type), see table below

Longitude and RA range from 0. to 360; Latitude and Dec range from -90. to +90.

For spatial_frame_type = "none": no value is provided, UCD are empty strings (""), and no unit is provided

(2) Spatial resolution parameters have the same unit as spatial coordinate parameters. The associated UCD combine either pos.resolution (if linear) or pos.angResolution (if angular) with secondary stat.min or stat.max

c1: only body and celestial are angular; c2: only cartesian is linear; c3: only spherical is angular

(3): Any contour type that works with ADQL's geometry operators (CONTAINS, INTERSECTS...) is legal here

(4): Timestamps are provided as ISO-8601 String as specified by DALI. On VOtable output, xtype="timestamp" attribute is required.

Example table for IDs:

Syntax

• multivalued lists = first entry#second entry#…#last entry, or scalar (with no #)
Values separator = #
No quotes around the list

This can be parsed by ADQL/RegTAP function ivo_hashlist_has like this:
  select * from vvex.epn_core where 1 = ivo_hashlist_has(lower(target_name),'Venus')

Where the lower function is mandatory to handle values possibly containing upper cases (this is implicit on the 2nd argument)

Beware that only complete elements between separators will be found. The provider has to split the string according to expected searches, e.g.:
    Composite Infrared Spectrometer#CIRS
not    Composite Infrared Spectrometer (CIRS)

Parameters supporting multivalued lists include:
dataproduct_type (only when present in the same file; best avoided when possible)
target_name (for different targets only, but only one target can be described in the granule; use alt_target_name for other names of the same target)
alt_target_name
target_class (in association with target_name)
instrument_host_name (e.g. acronym and full name)
instrument_name (e.g. acronym and full name)
measurement_type (when present in the same file)
processing_level (when present in the same file)
bib_reference

• NULL and special values:
A standard query on a parameter will not return granules with NULL/void value. E.g. target_name LIKE '%toto%' will only select granules with this value (standard ADQL behavior).
NULL/void has to be tested specifically (e.g., when it means "I don't know") using the IS operator (IS is used only to test the NULL value in ADQL): 

target_name LIKE '%toto%' OR target_name IS NULL

Syntax IS NULL stands for both strings and numerical parameters (the = operator is accepted in this context only by latest DaCHS servers)

No inf, inf, or NaN value in ADQL? At least Inf/-Inf should be there, as per DALI.

• UCDs: the above table has been reviewed against the UCD documents, including latest discussions (4/2019). Review against PDS4 and IPDA to be performed.

2018 discussions / conclusions have been included here: https://wiki.ivoa.net/twiki/bin/view/IVOA/UCDList1dot42017June2018FebRFM

• *_min vs *_max parameters: 

If only one value is available, it must appear in both fields

• Optional parameters: some of these come in sets that are logically related; if one is present, the related ones must be present also (e.g., 3 access_* parameters)

• Granule_gid: any general indication to providers? I.e.: preview, native, calibrated, geometry… 
A client should be able to display the values present in a service (feasible in TOPCAT)

• Reshuffle previous "service parameters":

• Mandatory :
  publisher - make it mandatory???
  add publisher_did as in Obscore? (for DaCHS/registry; provides unique ID of service for this publisher/server), with UCD = meta.ref.uri;meta.curation 
• Optional 
  spatial_coordinate_description  (default = none)
  spatial_origin  (default = body center or SS barycenter? Or observer location)
  time_origin  (default = observer)
  time_scale (default = UTC – no other values allowed in data services? [only in computational services, e.g. ephemeris])
  Same values to be used in registry declaration 

• Call-back parameters / reference

Currently using service_title (= schema name) + granule_uid. 

May use ivoID in the future.

• Other parameters

The most recent extra parameters often have names starting in prefix_*, where prefix identify the scope or context (e.g., spase_, vims_, image_, etc). Seems to be a good practice.

Accref is introduced by the EPN-TAP localfile mixin, but not used - in principle not included in TAP response, be may be present anyway. It may be better to hide it also in the portal.

• Parameters introducing error bars/uncertainties

Some parameters providing a scalar value X in the EPN-TAP table may be associated with an error bar in a related parameter. This is currently (4/2019) entered as:

In Basecom: Xerr (to be changed when upgrading to mixin version)
In DynAstVO: X_error
In Exoplanets: X_error_min; X_error_max
In planets: X_uncertainty
TNOsarecool: X_sigma_plus, X_sigma_minus (to be changed?)

The associated UCDs start with stat.error; or stat.error;stat.min; & stat.error;stat.max;

• Support for PDS3 detached labels (proposal)

Solution with datalink seems OK: data files under access_url and detached labels provided under datalink_url in a link table - although no attempt made to read them from the portal yet (use VIR unpublished service to test this).

• Utypes

Need to clean up current doc (2.0). Utypes are = DM fields. They are supposedly used to identify the meaning of parameters and help e.g. tools to grab required quantities - This will not work in some areas though, e. g. with spectral tools as they currently use UCD instead of Utype for this purpose (not many tools appear to actually rely on Utype in fact). See discussion here for usage (a bit old?): http://www.ivoa.net/documents/Notes/UTypesUsage/20130213/NOTE-utypes-usage-1.0-20130213.html

To handle this in practice:

• Associate each parameter to a specific Utype in EPNCore - all names need to start with the epncore: prefix/namespace.
• Then map epncore Utype to other DM (find equivalent parameter, or trace back the original templates of EPNCore parameters - often from ObsCore)
• Reuse Utype from other DM each times it makes sense - TBC: the epncore: namespace is still required (even when using Utypes from other DM)
  This allows tools to handle EPNCore parameters like existing parameters from other DM, i.e., with no specific implementation
  Pb is that small differences in the use of parameters may preclude reusing the same Utype (TBC: does that applies to units also?)
• Cross our fingers: known Utype (from other DMs) may be usable in existing tools (e.g. a tool supporting Provenance would grab equivalent info in EPN-TAP services transparently)
  Unclear if the use of the namespace makes it more complicated in tools.

Example of V2 with APIS database:

EPNcore Table v2