This page focuses on VESPA. Other access modes to planetary data predate VESPA and are still in use.
VO standards used in VESPA
General IVOA standards used by VESPA
Figure 1 identifies the IVOA standards used in VESPA
- VESPA uses a single protocol to access most data services: EPN-TAP, which is based on the TAP mechanism associated with a list of parameters describing the data, standards units to store the numerical parameters, and reference lists of values for strings parameters. This is protocol of choice to access most data in this field, which are very often available as a list of files in a dataset, or as catalogs of object properties.
EPN-TAP is used to access all possible data types; a specific client (VESPA portal / user interface) converts from EPN-TAP standard conventions to user parameters. In this sense, the set (VESPA portal and EPN-TAP) constitutes a middleware between the user and the data.
EPN-TAP provides access to granule level as defined by data providers, not to the underlying / subgranule structure (e.g., file structure). However, post-processing may be applied to a data product retrieved via VESPA (e.g. script extracting spectra from a cube, see CRISM use case or APERICubes).
Some useful extensions need to be defined or finalized, including:
- Projections/cartography (coord frame nomenclature still TBC)
- Lab spectroscopy (sample description)
- Solar System objects properties (both dynamic and physical)
- Collaborative work
EPN-TAP v2 doc ~ finalized Aug 2019, to be submitted to IVOA as a Working Draft (WD).
Access to on-line codes
However, on-line codes also need specific access.
- A draft protocol (EPN-Ping) was studied in EPN2020, using the EPN-TAP parameters to pass input arguments to such codes - proved difficult to implement, deprecated. But the general principle still stands.
- Datalink provides an alternative way to connect web services from a data service - assed with SPICAM and MCD services + used to call ephemeris from various services (HST, etc). Another possible usage is extraction of spectra from RoI in spectral cubes (using e.g. APERICubes)
- In EPN2024, implement OPUS platform from CTA.
- It is currently impossible to retrieve all EPN-TAP services with a single request, e.g. from TOPCAT (see Searching for EPN-TAP Services in the Registry). Conversely, the registry contains gremlins and old drafts which must be removed. The declaration procedure needs to be clarified entirely, and implemented on existing services.
- Need to identify EPN-TAP version in the registry - required to prepare support of EPN-TAP 2.1 in the portal.
Required for VESPA development
Solar System data description and handling require specific capabilities that are not always implemented for astronomy. The most important are listed here.
• The STC standard does not seem practical to accommodate the variety of coordinate frames in use in the Solar system.
- Body-fixed frames are of particular interest, both because of their ubiquity and because we want to develop the connection between the VO and GIS. This has to progress in parallel with similar requirements to support planets in OGC context.
- Although IAU provides the characteristics of reference coordinate systems on solar system bodies, they do not include a version number to identify what system is actually used in a dataset.
- Important parameters defining a body-fixed coordinate system include: body of interest, prime meridian and rotational parameters, control point system, date or version; plus possibly planetographic/centric nature, ellipsoid body radii, origin of vertical scale (altitude counted from ellipsoid or shape model, and altitude vs distance from center)
- Current fits kw do not allow projecting images on ellipsoids, even less on 67Ps (see Chiara's geofits extension, implemented in GDAL > 3)
We stick to the principle to provide a description of physical quantities via UCDs rather than in Utypes - VO tools use UCDs to understand the data, and this proves more flexible & easy than Utypes.
Although Solar System specific UCDs have been proposed in the recent year, the work is still in progress. Proposals for new UCDs are regularly forwarded to the semantics group of IVOA.
- In particular, spectroscopic and photometric measurements in the Solar system are commonly performed in reflected light and require specific description:
Unresolved sources may be measured as incident flux.
Resolved sources are either measured in radiance or reflectance (= I/F).
Lab measurements are provided in slightly different scales.
Flux can be described as phot.flux.density;em.wl
Radiance is best referred to as phys.luminosity;phys.angArea;em.wl in UCD 1.23, which is uncomfortably complicated. More appropriate would be phys.radiance (note: phot.radiance was proposed in v 1.3 but seems TBC for the thermal range)
Observed reflectance is usually provided as I/F (= radiance factor), which is the ratio of radiance to the reflected solar flux at the target distance (provided as sr-1). The only current match is phys.albedo, which is not very specific and implies a spectrally integrated quantity. More appropriate would be phys.reflectance
For lab measurements, standard quantities are reflectance/pi or reflectance/cosine of incidence angle, TBD
Normalized spectra may also need specific descriptors
Various kinds of albedos could be identified by specific UCDs
- Another specific area is the description of illuminated conditions
Initially, only phase angle is described in the UCD doc. Incidence (or "solar zenithal angle" in some contexts, but this may be misleading) and emergence angles at least are required (and possibly azimuth).
Obvious choices are pos.incidenceAng, pos.emergenceAng, pos.azimuthAng
- Other opens issues are related to in-situ measurements, and samples description (e.g., measurements in the lab).
- Other UCDs of more general use would be useful by EPN-TAP but are currently undefined:
pos.resolution (deprecated and replaced by pos.Angresolution) was useful to provide resolution on a linear distance axis, for instance on a planetary surface.
time.update could be used to provide the date of last update in a service (e.g. to speed up mirroring actions)
+ see UCDs used to describe EPN-TAP parameters.
VESPA contributions to the IVOA vocabulary are discussed elsewhere: https://voparis-gitlab.obspm.fr/vespa/ivoa-standards/semantics/veps
Daily and seasonal variations cycles are commonly studied, and may require specific UCDs (is it in Characterization?)
Use of scenarios must be considered (e.g. simulated Martian years for atmospheric studies)
Time itself need to be related to the target whenever coordinated observations are involved - this is best way to handle multispacecraft observations
Several problems have been spotted so far with ADQL requests in this context:
- Case variations must be supported to allow the use of IAU standards, e.g. in target names - otherwise, we are forced to request data providers to violate IAU naming conventions (e.g. write mars instead of Mars) - this should be in ADQL 2.0, TBC. If yes, we have a requirement to use only servers implementing ADQL 2.0
- Multivalued lists are difficult to handle. The obvious way is to search a string with 'like', but this yields many false alarms (e.g. Io contained in Iolanda), which can be solved by imposing a list separator.
- fixed through RegTAP functions, but a combination of #list search and LIKE would be welcome.
- Usage of NULL values - when the parameter contains no value, we do not necessarily want it to filter out results, TBC. It is possible to impose non-filtering of occasional NULL values using and extra OR query.
- The s_region parameter is handy for searches on the sky, and can be enlarged to searches on a sphere. What about ellipsoids or 3D shapes? Is it consistent with use by GIS?
- Assessment of s_region to cross-match several datasets (see OMEGA vs HRSC tuto); still TBC for region including a pole.
- Searches with s_region require to specify a coordinate frame. We currently have to use 'UnknownFrame', but we need at least a generic 'BODY-FIXED' frame to handle longitudes/latitudes (east-handed, 0-360°), and possibly to specify the body considered.
- More generally, use of ADQL regions is handy provided that the coordinate system is an input parameter (as opposed to assuming RA-DEC).
- Cartesian frames should also be supported?
- Need to assess the use of DALI-like contours - again the sense of rotation may be an issue here.
- Same for MOCs to provide space/time coverage - check if usable in ADQL/TAP context
Tools / special requirements
Open issues are related to:
- No initial support for reflectance or radiance in any spectral tool - related to missing UCDs. Implemented in CASSIS 5
- Direct EPN-TAP support in tools to be considered when the library is ready.
- addressed through VESPA developments; implemented in CASSIS, 3DView + TOPCAT and Aladin support TAP queries.
- 1D spectra: Working with IRAP for CASSIS - done.
- 1D-2D spectra : Specview used with APIS service, allows local summing of 2D spectra. Does not seem maintained though. I/O could be upgraded, in particular full support to VOtable in input + eps/pdf in output.
- now also handled in CASSIS to some extent.
- Need for general purpose arithmetics on vectors (profiles and spectra), on the fly: mean/sum/stddev inside a RoI, a series of columns, etc…
- Supports the LineList protocol (VAMDC related) which is more adapted than SLAP for our purpose? - this is less an actual protocol than a function in SpecView. SLAP2 being defined for VAMDC, TBC
- VOspec powerful but only accepts known quantities (therefore no reflectance). Plus, input dialogue is blocking. Apparently deprecated by ESA, not publishable as open source.
- SPLAT-VO is handy as a plotter at least. - confirmed, also supports fitting of noisy data. Nicely complements CASSIS.
- Spectral aggregates (e.g. Virtis-H, échelle spectrometer, but also multi-channel instruments) : special format needed?- see how SED tools can be used in this context. A specific format seem required, to store several spectral parts in a single file. Scripting addition to CASSIS actually works.
- "Slices" or series of profiles? No std way to store the data…
- Images: projections on ellipsoids and funny surfaces (3D shapes) - support of several datasets in MATISSE
- Working with CDS to handle planetary images in Aladin - see use cases
- Spectral cubes: APERICubes demonstrator - should also support 2D cubes (VIRTIS-H); GIS might be the best solution. Pb is that there is no standard way to store these data, so it really depends on dataset.
- PDS3 handling: APERICubes bottom layer (converts PDS3 in fits subsets, either images or spectra - requires GDL/IDL session on server side)
- Format conversions: via GDAL (GIS + fits formats, some PDS; scriptable + in QGIS) and ImageJ (standard image formats; now with SAMP interface)
Baptiste, shall we add VOEvent or open a page for VOEvent adaptation to PSWS (and VESPA?) needs ?
Indeed there is very little info on the wiki currently - needed for the report at least