First thoughts about the next evolution of the protocol.
Evolution
• Add a different mixin for next minor versions (2.1 and above), preserve v2.0. Send new versions to Markus for inclusion in DaCHS (also some fixes required in current version)
• Adapt search interface to support all versions 2 separately - should only require minor changes in queries and displays
So that all v2 can run in parallel. The latest one becomes mandatory for new services, but upgrades will not require to regenerate all services.
New minor versions therefore correspond to a required change in the mixin, ie a change in the way parameters are handled (a simple extension of possible values wouldn't usually trigger a new version, unless it seriously affects existing services).
Possible updates
So far, we've considered:
- Using ISO strings to handle time_min & time_max (instead of Julian days, which are not recommended in IVOA) - TBC, this is actually discussed (at some point, Markus advised against time stamps because of poor ADQL support)
- Providing spectral resolution as abs(lambda / d(lambda) ) = abs( freq / d(freq) ) (instead of d(freq) in v2.0) - discussion finalized with SSHADE and other spectral services (Berlin mostly)
- Fixing target_class list - need to improve interplanetary medium vs solar wind vs interplanetary dust? There seems to be a mix up between class, name and feature/region in this case, TBC.
- Adding "none" as a possible value for spatial_frame_type (include this in mixin, it impacts q.rd files significantly).
- Is there something new in recent versions of DaCHS? Parameters now seem to require type conversion before numerical operations - TBC
- Add a parameter to distinguish format from dataproduct_type (ex: dynamic spectra provided as plots vs actual data). Could be "granule_type" (not favorite…) with a limited list of values: data, preview (or plot…), possibly also associated_data, label, etc. Associated granules with different "granule_type" must have the same obs_id (granule_gid can't be used for this, because it is a free string).
- Define possible usage for new parameter datalink_url: can link to a doc, a web service, an internal service on the server (cutout).
- Find a way to include COOSYS in service table even when this is varying inside the table… - or at least when this does not vary.
- Make time_refposition and time_scale mandatory, so the DaCHS can build TIMESYS (defaults = UTC, TOPOCENTER)
- Need to add keywords in the registry to identify the science theme - to be taken from the IAU thesaurus preferably, or new value if nothing exist. Several such values can be added. Define a list of EPN-TAP services
- One of these keywords is "Event" to tag VOevents - this will make it possible to filter EPN-TAP services gathering/archiving VOevents (we expect lots of them, and we want to be able to prefilter them very easily when looking for data - not a nerd thing, please) - this filtering will be handled in the portal.
- Replace service_title = schema name by a parameter providing the ivoID of the service (same objective: to refer to original service in derived tables) - see EPN-TAP ticket ending 20/6/2018.
- Fix accref usage in DaCHS / mixin: accref is generated when the service is built from local files, and is apparently used in some situations (std name for other protocols). Really consider replacing access_url by accref. (no, in practice : accref is used internally to DaCHS in some situations, better not mess with it).
- Replace cube type by other_cube? This name is severely misleading.
- Accommodate PDS4 processing levels (in particular "partially calibrated") and leave room for other nomenclatures. Make processing_level a string? Or duplicate (one CODMAC/EPNCore and one native?)
- Add reference RA/DEC (sky) or lon/lat (surfaces) parameters for each granule whenever relevant (to make handling in Aladin easier, in particular for MOCs)
- Possibly split coordinates in independent parameters, according to spatial_frame_type => will give lon, lat, alt, x, y, theta, phi, R (with no duplication? TBC if consistent). This seems required to mix granules from different services, and with different types of coordinates (see draft from Baptiste to compare with PDS4 parameters, 2019). This would also make it possible to provide several coordinates (although they would have to be defined…) - already done for RA-Dec in practice. Beware that this is tricky: some services use sets of coordinates with different meanings, e.g. VVEx service provides body lon/lat in C1/C2, but s/c pointing direction in RA/DEC, ie different quantities using different coordinates.
- Markus encourages to add a publisher_did parameter.
- Think about possible evolution of s_region - do we stick to this (nearly deprecated) standard or do we shift to DALI-like regions or anything else (MOC…)? (pros: now widely supported in TAP and tools, and fits into tables; cons: ingestion in services is always a mess, e.g. spoly vs STC-S issues). MOC don't fit in tables, so the usage is different. evol : DALI regions for s_region, MOC are introduced in the coverage parameter.
- Accept (impose?) new parameter to accommodate MOCs (assessment study by Markus, May 2020). Pros: OK in an EPN-TAP table on use case; solves the direction issue of s_region; resolution is high enough even for Mars orbital (order up to 29); will work on any convex body; ADQL queries similar to polygon ones; quick. Cons: any areal application will be wrong on non-spherical bodies; won't work on peanuts (67P like); don't mix with polygons in s_region…; defined only on the sky - to be used in conjonction with spatial_coordinate_description values (this param should always be present in ADQL queries involving coord or regions).
spatial_coordinate_description value associated to spatial_frame_type = "none"?
Add a data_doi parameter? Even if available only at dataset level, this would come handy when dealing with results from mixed services.
- Make alt_target_name a mandatory parameter? That would greatly help the portal to handle multiple denominations of small bodies.
- Possibility to associate a bib ref to individual parameters? Something like parameter_reference. Alternatively, could be provided through datalink.
To be corrected in current mixin: file_name is not an url + some typos identified + correct units in coordinates
Markus is involved in this update activity, and willing to help.Usage of instrument_name / instrument_host_name to be clarified. instrument_host_name may be used for either mission or spacecraft / observatory or telescope; instrument_name is for instrument only (and detector_name is also available). This should be consistent with both ObsCore and Spase, TBC (Spase definitions would be: hosts are associated to a location; instruments to coverages). To prepare this, first fix existing services.
- Clarification for simulations related to an instrument: instrument_host_name could be "simulation#Cassini" when simulating observations from a spacecraft; instrument_name could be "express_v3#VIMS", especially if specific instrumental response is included. The reason is to inform the user that simulations exist when he's looking for actual observations from an instrument - this is difficult otherwise (in addition, measurement_type still provides the physical quantity, with ";meta.modelled" appended). Still TBC, but looks OK.