EPN-TAP V2.0 parameters

List of EPN-TAP parameters
v2, 3/8/2015 (SE)
v3, 29/8/2015 (SE, integrating comments by PLS and BC)
v4, 7/9/2015 (SE, integrating comments by PLS)
v5, 7/10/2015 (SE, integrating comments by PLS: time_scale UCD)
v6, 7/11/2015 (SE, integrating comments by PLS/BC)
v7, 14/12/2015 (SE, added point 8, clarification of an older idea)
v8, 24/12/2015 (SE, review/integration of comments/corrections)
v9, 5-7/01/2016 (SE/BC, including comments from Chiara Marmo and Michel Gangloff)

Main evolution relative to v1

1) The previous notion of dataset is deprecated

This was complex to handle in the database, and in general not relevant for the services.

2) Grouping of products is rationalized in version 2

A granule is still a record/line in the epn_core view database, and corresponds to the smallest data unit described by the service.

A "product" is typically a data file, or a service output, that can be reached through a URL

In version 2 both concepts coincide, while in v1 several products could be part of the same granule when they are related to the same initial data ("observation")

granule_uid is equivalent to the previous index parameter and provides a unique ID for the granule in the service, ie for each line in the epn_core view (index is a reserved word in many database languages and should not have been used in the first place)

granule_gid is related to a type of product: it is identical for all granules containing the same type of information for different observations (e.g., calibrated files). An explicit string is recommanded in this field (e.g., native/calibrated/geometry).

obs_id is related to an observation: it is identical for all granules related to the same observation, containing different type of data (e.g.: raw and calibrated data, associated geometry, etc). In many EPN-TAP v1 services, such products were described together on the same line with a unique index parameter.

As a general rule, different products related to the same observation are no longer described together on a single line of the epn_core view, but on successive lines associated by the same obs_id, each with a different granule_gid (and a specific granule_uid, see Table 2 below).

Each line in the epn_core view must describe only one product (plus a thumbnail wherever relevant). The notion of "main product" (more or less explicit in v1) is therefore deprecated, and the epn_core view in v2 includes more lines than in v1. Although less compact that the previous table presentation, this list presentation is much more efficient for machine-handling.

From our initial discussions, all such IDs can be alphanumeric strings, TBC (in particular for granule_uid, but this seems OK with ADQL)

3) The notion of table/service parameters is deprecated

This was not supported by TAP. Therefore, even constant parameters must be replicated in every line of the epn_core view.

The corresponding parameters may be present also in the registry declaration though, to provide a fast description of services.

4) Footprints can be provided through s_region

s_region is a parameter in the TAP standard of IVOA, and ADQL allows for powerful query functions such as intersections.

We have to study a possible extension of its use from regions of the sky to body-related coordinates (on ellipsoids or 3D shape models) .

The use in the context of GIS also has to be studied.

5) Better handling of the evolution of services

creation_date and modification_date are now mandatory parameters for every granule. The latter is intended to allow for fast mirroring of services. From our first discussions this is best formatted using an ISO string, with the form "2013-11-17T10:41:00.00+01:00" (TBC with ADQL capacities, in particular support of time zone - we don't want any space in the string).

6) Support for coordinated observations

The target_time parameter now provides observation time in the reference frame of the target. This is intended to facilitate the cross-correlation of observations from different locations, e.g., telescopic observations in support of space missions, or multi-spacecraft campaigns.

7) Axes ranges

All parameters defining a range are now introduced with a min and max value.

All floating point parameters are now in double precision (to prevents stupid errors).

8) Thematic extensions

Some science fields will require optional parameters, which need to be used consistently between services addressing the same field. Such extensions have to be designed by sub-groups involved in the corresponding data services, either as providers or users. This includes:

Lab spectroscopy: parameters to describe mineralogical samples (and possibly other samples)
Orbital/rotational parameters and physical properties of Solar System bodies (TBC, but requested by some)

+ See notes below the table.

Name, v2	Name, v1	Class	Unit	Description	UCD
EPNCore mandatory parameters
granule_uid	index	String		Internal table row index Unique ID in data service, also in v2. Number only?	meta.id
granule_gid	_	String		Common to granules of same type (e.g. same map projection, or geometry data products). Can be alphanum.	meta.id
obs_id	_	String		Associates granules derived from the same data (e.g. various representations / processing levels). Can be alphanum., may be the ID of original observation.	meta.id
—	resource_type	String		Can be dataset or granule	meta.code.class
_ (TBC)	dataset_id	String		Dataset identification & granule reference	meta.id
dataproduct_type	dataproduct_type	String		Organization of the data product, from enumerated list	meta.code.class
target_name	target_name	String		Standard name of target (from a list depending on target type), case sensitive	meta.id;src
target_class	target_class	String		Type of target, from enumerated list	meta.code.class;src
time_min	time_min	Double	d	Acquisition start time (in JD). UTC at time_origin (default is observer's frame)	time.start
time_max	time_max	Double	d	Acquisition stop time (in JD). UTC at time_origin (default is observer's frame)	time.end
time_sampling_step_min	time_sampling_step_min	Double	s	Min time sampling step	time.interval;stat.min
time_sampling_step_max	time_sampling_step_max	Double	s	Max time sampling step	time.interval;stat.max
time_exp_min	time_exp_min	Double	s	Min integration time	time.duration;stat.min
time_exp_max	time_exp_max	Double	s	Max integration time	time.duration;stat.max
spectral_range_min	spectral_range_min	Double	Hz	Min spectral range (frequency)	em.freq;stat.min
spectral_range_max	spectral_range_max	Double	Hz	Max spectral range (frequency)	em.freq;stat.max
spectral_sampling_step_min	spectral_sampling_step_min	Double	Hz	min spectral sampling step	em.freq.step;stat.min (not in list)
spectral_sampling_step_max	spectral_sampling_step_max	Double	Hz	Max spectral sampling step	em.freq.step;stat.max (not in list)
spectral_resolution_min	spectral_resolution_min	Double	Hz	Min spectral resolution	spect.resolution;stat.min
spectral_resolution_max	spectral_resolution_max	Double	Hz	Max spectral resolution	spect.resolution;stat.max
c1min	c1min	Double	(1)	Min of first coordinate	pos;stat.min
c1max	c1max	Double	(1)	Max of first coordinate	pos;stat.max
c2min	c2min	Double	(1)	Min of second coordinate	pos;stat.min
c2max	c2max	Double	(1)	Max of second coordinate	pos;stat.max
c3min	c3min	Double	(1)	Min of third coordinate	pos;stat.min
c3max	c3max	Double	(1)	Max of third coordinate	pos;stat.max
s_region		Vector?		TAP-like footprint, assume spatial_coordinate_description – this one TBC (needs another param for GIS interface?)
c1_resol_min	c1_resol_min	Double	(1)	Min resolution in first coordinate	pos.resolution;stat.min (not in list)
c1_resol_max	c1_resol_max	Double	(1)	Max resolution in first coordinate	pos.resolution;stat.max (not in list)
c2_resol_min	c2_resol_min	Double	(1)	Min resolution in second coordinate	pos.resolution;stat.min (not in list)
c2_resol_max	c2_resol_max	Double	(1)	Max resolution in second coordinate	pos.resolution;stat.max (not in list)
c3_resol_min	c3_resol_min	Double	(1)	Min resolution in third coordinate	pos.resolution;stat.min (not in list)
c3_resol_max	c3_resol_max	Double	(1)	Max resolution in third coordinate	pos.resolution;stat.max (not in list)
spatial_frame_type	spatial_frame_type	String		Flavor of coordinate system, defines the nature of coordinates	meta.code.class;pos.frame
incidence_min	incidence_min	Double		Min incidence angle (solar zenithal angle)	pos.posang;stat.min
incidence_max	incidence_max	Double		Max incidence angle (solar zenithal angle)	pos.posang;stat.max
emergence_min	emergence_min	Double		Min emergence angle	pos.posang;stat.min
emergence_max	emergence_max	Double		Max emergence angle	pos.posang;stat.max
phase_min	phase_min	Double		Min phase angle	pos.phaseang;stat.min
phase_max	phase_max	String		Max incidence angle	pos.phaseang;stat.max
instrument_host_name	instrument_host_name	String		Standard name of the observatory or spacecraft	meta.id;instr.obsty
instrument_name	instrument_name	String		Standard name of instrument	meta.id;instr
measurement_type	measurement_type	String		UCD(s) defining the data	meta.ucd
creation_date		ISO String?		Date of first entry of this granule
modification_date		ISO String?		Date of last modification (used to handle mirroring)
Optional parameters
access_url	access_url	String		URL of the data file, case sensitive. If present, next 2 parameters must be present.	meta.ref.url;meta.file
access_format	access_format	String		File format type (mime type in lowercases)	meta.code.mime
access_estsize	access_estsize	Integer	kB	Estimate file size in kB	phys.size;meta.file (TBC)
_ replaced by another granule group	preview_url	Integer		URL of a preview image (std format with adequate resolution for user's purpose)	meta.ref.url;meta.file
_ replaced by another granule group	native_access_url	String		URL of the data file in native form, case sensitive	meta.ref.url;meta.file
_ replaced by another granule group	native_access_format	String		File format type in native form	meta.id;class or meta.code.mime (if we use MIME type)
thumbnail_url	_	String		URL of a thumbnail image with predefined size (png ~200 pix, for use in a client)	meta.ref.url;meta.file
file_name	file_name	String		Name of the data file only, case sensitive	meta.id;meta.file
species	species	String		Identifies a chemical species, case sensitive	meta.id;phys.atmol
target_region	target_region	String		Type of region of interest	meta.id;class
feature_name	element_name	String		Secondary name (can be standard name of region of interest)	meta.id;pos
bib_reference	reference	String		Bibcode, doi, or other biblio id, URL…	meta.bib
ra	ra	Double		Right ascension	pos.eq.ra;meta.main
dec	dec	Double		Declination	pos.eq.dec;meta.main
solar_longitude_min	solar_longitude	Double		Min Solar longitude Ls (location on orbit / season)	pos.posangle (TBC)
solar_longitude_max	solar_longitude	Double		Max Solar longitude Ls (location on orbit / season)	pos.posangle (TBC)
local_time_min	local_time_min	Double	h	Local time at observed region	time.phase;stat.min?
local_time_max	local_time_max	Double	h	Local time at observed region	time.phase;stat.max?
target_distance_min	target_distance (no min/max)	Double	km	Observer-target distance	pos.distance;stat.min
target_distance_max		Double	km	Observer-target distance	pos.distance;stat.max
target_time_min	_	Double	d	Observing time in target frame	time.start ??
target_time_max	_	Double	d	— TBC if we want both	time.end ??
particle_spectral_type	particle_spectral_type	String
particle_spectral_range_min	particle_spectral_range_min	Double
particle_spectral_range_max	particle_spectral_range_max	Double
particle_spectral_sampling_step_min	particle_spectral_sampling_step_min	Double
particle_spectral_sampling_step_max	particle_spectral_sampling_step_max	Double
particle_spectral_resolution_min	particle_spectral_resolution_min	Double			spect.resolution;stat.min
particle_spectral_resolution_max	particle_spectral_resolution_max	Double			spect.resolution;stat.max
Relative to service / Table header (mandatory or not?)
processing_level	processing_level	Integer		CODMAC calibration level in v1 To be replaced by PDS4 values in v2?	meta.code;obs.calib
publisher	publisher	String		Resource publisher	meta.name
service_title	service_title (was initially "title")	String		Title of resource	meta.title
spatial_coordinate_description	spatial_coordinate_description	String		?	?
spatial_origin	spatial_origin	String		Defines the frame origin	meta.ref;pos.frame
time_origin	time_origin	String		Defines where the time is measured	?
time_scale	?	String		Always UTC in data services (may be relaxed in computational services such as ephemeris)	time.scale

(1): depending on context (as given by spatial_frame_type)

Example table:

File name-type	granule_uid	granule_gid	obs_id
A-Raw	1	native (or 1)	1 (or A)
A-Calib	2	2	1
A-geom	3	geometry (or 3)	1
A-proj	4	4	1
B-Raw	5	1	2 (or B)
B-Calib	6	2	2
B-geom	7	3	2
B-proj	8	4	2

Other modifications

• string delimiter for enumerated lists = #first entry#second entry#…#last entry#
TBC with PDS, exoplanets, NSSDC… (requires a character which never appears in target names, etc)

• Defaut values :
NULL/void: will never return an answer to a query using this parameter (TBC, seems ADQL-related. To be corrected if it is a limitation of the client)
For float / double : -inf for *_min +inf for *_max – still TBC (NaN won't do). To be tested on a real case.
For strings: not needed? (i.e. NULL/void is OK?)

• UCDs: to be reviewed against PDS4 and IPDA, and completed

• Processing levels: to be reviewed against PDS4 (again)

• min vs max :
if only one value available, it must appear in both fields

• Optional parameters: they come in sets that are logically related; if one is present, the related ones must be present also (e.g., 3 access_* parameters)

• Granule_gid : any general indication to providers? I.e.: preview, native, calibrated, geometry…
A client should be able to display the values from a service, TBC

• Reshuffle previous "service parameters":

Mandatory :
processing_level -mandatory
publisher -mandatory
service_title –mandatory
Optional - TBC
spatial_coordinate_description (implicitly body-fixed or J2000)
spatial_origin (implicitly body center or SS barycenter? Or observer location)
time_origin (implicitly: observer)
time_scale (implicitly : UTC – no other values allowed in data services? [only in computational services, e.g. ephemeris])
Same values to be used in registry declaration

• All floating point parameters are now in double precision – this prevents stupid errors…

Example of V1 to V2 conversion with APIS database:

EPNcore Table v1

index	resource_type	dataset_id	access_url	access_format	preview_url	native_access_url	native_access_format
23801	granule	original_data	o5g202x4q_x2d.jpg	jpg	o5g202x4q_x2d_small.jpg	o5g202x4q_x2d.fits	fits
23802	granule	processed_data	o5g202x4q_proc.jpg	jpg	o5g202x4q_proc_small.jpg	o5g202x4q_proc.fits	fits
23803	granule	cylindric_proj	o5g202x4q_cyl.jpg	jpg	o5g202x4q_cyl_small.jpg
23804	granule	polar_proj_north	o5g202x4q_pol_n.jpg	jpg	o5g202x4q_pol_n_small.jpg
23805	granule	polar_proj_south	o5g202x4q_pol_s.jpg	jpg	o5g202x4q_pol_s_small.jpg

EPNcore Table v2

granule_uid	granule_gid	obs_id	access_url	access_format	thumbnail_url
o5g202x4q_x2d	original_data	o5g202x4q	o5g202x4q_x2d.fits	fits	o5g202x4q_x2d_small.jpg
o5g202x4q_x2d_prev	original_data_preview	o5g202x4q	o5g202x4q_x2d.jpg	jpg	o5g202x4q_x2d_small.jpg
o5g202x4q_proc	processed_data	o5g202x4q	o5g202x4q_proc.fits	fits	o5g202x4q_proc_small.jpg
o5g202x4q_proc_prev	processed_data_preview	o5g202x4q	o5g202x4q_proc.jpg	jpg	o5g202x4q_proc_small.jpg
o5g202x4q_cyl	cylindrical_projection	o5g202x4q	o5g202x4q_cyl.jpg	jpg	o5g202x4q_cyl_small.jpg
o5g202x4q_pol_n	polar_projection_north	o5g202x4q	o5g202x4q_pol_n.jpg	jpg	o5g202x4q_pol_n_small.jpg
o5g202x4q_pol_s	polar_projection_south	o5g202x4q	o5g202x4q_pol_s.jpg	jpg	o5g202x4q_pol_s_small.jpg

Page tree

EPN-TAP V2.0 parameters

Main evolution relative to v1

Other modifications

Example of V1 to V2 conversion with APIS database: