EPN-TAP V2.0 parameters

List of EPN-TAP parameters
v2, 3/8/2015 (SE)
v3, 29/8/2015 (SE, integrating comments by PLS and BC)
v4, 7/9/2015 (SE, integrating comments by PLS)
v5, 7/10/2015 (SE, integrating comments by PLS: time_scale UCD)
v6, 7/11/2015 (SE, integrating comments by PLS/BC)
v7, 14/12/2015 (SE, added point 8, clarification of an older idea)
v8, 24/12/2015 (SE, review/integration of comments/corrections)
v9, 5-7/01/2016 (SE/BC, including comments from Chiara Marmo and Michel Gangloff)
v10, 21/02/2016 (SE, from model implementation of IKS)

New in v2

• The "laundry list" format makes services easier to design and to query
• Allows grouping of results from several services at once
• Supports multispacecraft observations
• Speeds up mirroring of services (support for partial updates)
• Better support of footprints, and better interface with GIS

Main evolution relative to v1

1) The previous notion of dataset is deprecated

This was complex to handle in the database, and in general not relevant for the services.

2) Grouping of products is rationalized in version 2

A granule is still a record/line in the epn_core view database, and corresponds to the smallest data unit described by the service.

A "product" is typically a data file, or a service output, that can be reached through a URL

In version 2 both concepts coincide, while in v1 the same granule could be composed of several products related to the same initial data (an "observation")

In EPN-TAP v2, granules are referenced by 3 parameters:

• granule_uid provides a unique ID for the granule in the service, ie for each line in the epn_core view. It is equivalent to the previous index parameter (index is a reserved word in many database languages and should not have been used in the first place)
• granule_gid is related to a type of product: it is identical for all granules containing the same type of information for different observations (e.g., calibrated files). An explicit string is recommanded in this field, with suggested standard values (e.g., preview/native/calibrated/geometry/projection…).
• obs_id is related to an observation: it is identical for all granules related to the same observation, containing different type of data (e.g.: raw and calibrated data, associated geometry, etc). In many EPN-TAP v1 services, such products were described together on the same line with a unique index parameter.

These 3 parameters can be arbitrary alphanumeric strings — see example application to APIS service below.

In practice, different products related to the same observation are no longer described together on a single line of the epn_core view, but on successive lines associated by the same obs_id, each with a different granule_gid (and a specific granule_uid, see Table 2 below).

Each line in the epn_core view must describe only one product (plus a thumbnail wherever relevant). The notion of "main product" (which was more or less explicit in v1) is therefore deprecated, and the epn_core view in v2 includes more lines than in v1. Although less compact that the previous table presentation, this list presentation is much more efficient for machine-handling, and easier to design.

3) The notion of table/service parameters is deprecated

This was not supported by TAP. Therefore, even constant parameters must be replicated in every line of the epn_core view.

The corresponding parameters may be duplicated in the registry declaration though, to provide a fast description of services.

4) Footprints can be provided through s_region

s_region is a parameter in the ObsCore standard of IVOA, and ADQL allows for powerful query functions such as intersections. s_region introduces an STC-S string providing description of the observed area on a sphere.

We have to study a possible extension of its use from regions of the sky to body-related coordinates (on ellipsoids or 3D shape models) .

The use in the context of GIS also has to be studied.

5) Better support of evolving services

creation_date and modification_date are now mandatory parameters for every granule. The latter is intended to optimize mirroring of services, by identifying the part to be updated/copied.

These parameters must be provided as ISO 8601 strings with format "2013-11-17T10:41:00.00+01:00" (with no space) where the indication of time zone is mandatory. The associated data type must be "date". ADQL supports this format and filtering using time zone is possible.

6) Support of coordinated observations

The target_time parameter now provides observation time in the reference frame of the target. This is intended to facilitate the cross-correlation of observations from different locations, e.g., telescopic observations in support of space missions, or multi-spacecraft campaigns.

7) Axes ranges

All parameters defining a range are now introduced with a min and max value.

All floating point parameters are now in double precision (to prevents stupid errors).

8) Thematic extensions

Some science fields will require optional parameters, which need to be used consistently between services addressing the same field. Such extensions have to be designed by sub-groups involved in the corresponding data services, either as providers or users. This includes:

• Lab spectroscopy: parameters to describe mineralogical samples (and possibly other samples)
• Orbital/rotational parameters and physical properties of Solar System bodies (TBC, but requested by some)
• Planetary 3D modelling run results
• Exoplanets / planetary systems properties
• Bibliographic entries? May be manageable otherwise, through bibcode / doi interpretation

9) Parameters which must be provided

are now clearly identified - those are not only mandatory, they also must provide a value.

They are mostly related to service description and granule identification.

+ See notes below the table.

(1): depending on context (as given by spatial_frame_type)

Beware that datatypes apply to the epn_core view, not to the q.rd file where they can be different

Example table:

Other modifications, to be reviewed

• string delimiter for enumerated lists = #first entry#second entry#…#last entry#
TBC with PDS, exoplanets, NSSDC…  (requires a character which never appears in target names, etc)

• Defaut values :
NULL/void: will never return an answer to a query using this parameter (TBC, seems ADQL-related. To be corrected if it is a limitation of the client)
For float / double : -inf for *_min +inf for *_max – still TBC (NaN won't do). To be tested on a real case.
For strings: not needed? (i.e. NULL/void is OK?)

• UCDs: to be reviewed against PDS4 and IPDA, and completed

• Processing levels: to be reviewed against PDS4 (again)

• min vs max :
if only one value available, it must appear in both fields

• Optional parameters: they come in sets that are logically related; if one is present, the related ones must be present also (e.g., 3 access_* parameters)

• Granule_gid : any general indication to providers? I.e.: preview, native, calibrated, geometry… 
A client should be able to display the values from a service, TBC

• access_format: do we really use mime-types?

• Reshuffle previous "service parameters":

• Mandatory :
  processing_level -mandatory
  service_title –mandatory
  publisher -mandatory???
• Optional - TBC
  spatial_coordinate_description  (default = body-fixed or J2000)
  spatial_origin  (default = body center or SS barycenter? Or observer location)
  time_origin  (default = observer)
  time_scale (default = UTC – no other values allowed in data services? [only in computational services, e.g. ephemeris])
  Same values to be used in registry declaration 

• Support for PDS3 detached labels (proposal)

access_format = "PDS3label" (it has to be here: we need to know that there is a detached label)
then access_url points to the label, and data_access_url points to the file (param is mandatory in this case - although the data file name is in the label, it can be in another directory)
A script can then recover both files and do something with them. This mechanism can be extended to other formats with detached labels (ENVI…).

Example of V1 to V2 conversion with APIS database:

EPNcore Table v1

(was not really compliant…)

EPNcore Table v2