The goal of this page is to identify a roadmap for the evolution of VESPA main search interface.

The VESPA portal is the main EPN-TAP client, which allows the user to discover and query EPN-TAP data services. New functions to be implemented during the course of the project will be discussed here. As a matter of principle, this interface is expected to be more versatile than generic TAP clients and embedded EPN-TAP clients/libraries, and more user friendly.

The older roadmap page from EPN2020 is available here.

New ideas from study by SpaceFrog / Nicolas Manaud. See status in Dec 2023 here.

Initial status (4/2020)

The VESPA portal is intended to query all available EPN-TAP services in one go, or specific services individually in more details.

Two versions are used:

The code of the VESPA portal is also distributed for local implementation, e.g. on Virtual Machines. It is currently on a svn repo in Paris and only available on demand (general security policy with interfaces).

The main developer is Cyril Chauvin at ObsParis/DIO.

An helpdesk link on the first page is intended to send comments to the development team (

At the onset of EPN-2024, the portal page is split in 3 areas:  

  • A query form on the left side, always visible and active, with several query modes
  • A query results area in the middle - this switches from a list of services to a table providing results in one selected service
  • A housekeeping area on the right side providing links to VO tools, links to examples and help page, and space to plot data thumbnails 

The portal uses a local registry of EPN-TAP services, rather than the IVOA registry. This is intended in particular to fulfill a requirement of the ANO5 service (French level), which is to certified data quality. The local registry only lists services which have been reviewed by the VESPA team. Only those are listed and queried together in default mode.

The current layout results from many interactions and user feedback during EPN-2020. 
• A general assessment of usability will be conducted by SpaceFrog Design, based on User Experience analysis (see VESPA UX/UI).

Main evolution to support future EPN-TAP versions

At the onset of EPN2024, all EPN-TAP services are in v2.0.

A need to implement (relatively) minor modifications will result in the release of EPN-TAP v2.1 in the mid-term. Both versions must be supported simultaneously.

Comments in blue below are requests from various review panels (EC, internal, ANO5 in France after labeling).

Additional functions to be implemented

Global layout / ergonomy

We'll study the evolution of the current layout to improve user experience.

• The most ergonomic model we've identified is the MAST interface, e.g.:

(query panel + results with footprints exposed on the same page, with real-time changes; tab system for successive requests). However, a major difference is that target or coordinates are always a parameter for Astronomy services, and all these data have spatial extension in the same reference frame.

• ESAsky is nicely done and quicker than MAST - again this is more visu on the sky, but there is a dedicated solar system object search based on ephemerides (not on catalogues):

• Another interesting model is the Gaia search interface:

• TapHandle also has interesting functions to query individual TAP services:

In particular, url and s_region can be SAMPED or displayed directly. The SAMP mechanism seems to be more flexible.

• Finally, some PDS interfaces have very effective interfaces, also to be studied (PILOT, OPUS, etc, as well as the PSA modern interface)

Fixes required on Query form

*** : remaining priorities in Dec 2023

  • Reorganize query panels: add access_format to main panel (search with LIKE) (OK in service pages)
  • Reorganize query panels: add service_title to Data Reference panel (search with LIKE?) 
  • Also add file_name? (not always there, but same kind of request as granule_uid)
  • Implement all mandatory parameters in standard query mode panel
  • *** Update value lists for parameters: in particular "none" for spatial_frame_type 
  • Need to implement all common optional parameters in the "optional" panel - long standing issues, and a blocking one (=> the optional has disappeared, OK)
  • The standard mode should detect EPN-TAP version of service, to handle both EPN-TAP 2.0 and 2.1 (when 2.1 is ready…)
  • Check with complete spoly in s_region (beware this is printed as STC-S after an ADQL query and in the output VOtable) see if INTERSECTS works - urgent (the new sandbox DaCHS supports it…). Check if there is an issue with longitude convention and s_region
  • If needed, check if sPoly can be generated on the fly from STC-S string in s_region (see simpleSTCSToPolygon function in module tapstc)
  • Implement spatial queries as in support.epntap ticket 31/5/2016 (with INTERSECTS, CONTAINS… + other modes, check that ticket).
    2019: the full query is more complicated - it requires to upload a table on a data server, so it has to go through TOPCAT. Basic INTERSECTS, etc (on a single region)… may go through the portal, but the Coord Syst should be checked in the same query. (=> essentially obsolete with broader usage of MOCs)
  • Fix queries on string parameters: always use LIKE instead of =, use lower() on both arguments: where lower(instrument_name) like lower('%CIRS%') ; or provide options
  • Case support still to be implemented on processing_level, dataproduct_type, + target_class - ?? (obsolete)

  • Add a list of measurement_type values present in connected services (but allow for other values in the query) — easier in service pages

  • do this also for instrument_name & instrument_host_name? — easier in service pages
  • *** On alt_target_name, bib_reference, it would be better to use a combination of hashlist and LIKE. Using a simple LIKE for the time being.

  • Implement searches with LIKE (instead of =) for all other string parameters

  • On first and next pages: the list of tools is sometimes understood as the complete list of tools connected to VESPA - we have to fix this.
    Plus, some people want a short indication of what these tools are for… (maybe we can add a link to a special help page, or a help bubble, under the subtitle "Plotting tools"?)
    Plus-plus, links to VO support services would be nice (ssodnet, Miriade, Spanish filter service… )

    => see design proposals by Nicolas M
  • *** We need a function to retrieve all parameters used in declared services (with indication of service where they appear). In some case, existing values may be also required, TBC (this is more a review tool than interface related). Test service in 2018, not finalized (and not published)
  • Need to reflect changes in EPN-TAP v2 PR version (21/7/2021) - in particular value lists

Additional functions on Query form

  • Add a button to convert W longitudes inputs to E longitude (EPNcore std) - this would save time is some situations (eg, query based on W-handed coordinates). This only stands when Spatial_Frame_Type = body
  • When Spatial_Frame_Type = celestial, use conversion system similar to Vizier (interpret all possible formats for RA-DEC, e.g., deg and hh:mm:ss)
  • Test coordinate conversion services, such as Treps or WebgeoCalc, or GDAL. To be used ~ like the name resolver on the interface? - This is more intended for workflows.
  • When ready, use the Observatory list as a resolver in the Instrument panel
  • In Custom/Direct query mode: add a possibility to query views which are not xxx.epn_core? That would provide a general TAP / ObsTAP interface for free. TBC — much too demanding, dropped
  • Add link page to other web-based VO clients? Would provide the possibility to get data from astronomy servers through other protocols and share them in the local Hub. This includes Taphandle, Vizier, MAST, etc… - implemented in the general web site
  • Don't do PDAP query in custom mode (=> will be dropped with PDAP deprecation)

  • Support queries on multiple targets? Mars#Venus could be translated as target_name= lower('Venus') OR target_name= lower('Mars') - To be investigated — no… this is a very simple ADQL query

Re-organization of services Query results 

We currently have one box per service, which may become impractical when we have many services


  • add last modif date in box? Not the one from the registry though, it will never be updated (should be the most recent modification_date in service in v2 - where do we find this?)
  • Add some prefiltering function to only query/display services in a given area (eg: small bodies, surfaces…). This info has to be added in the local registry
  • => *** Tag services with science theme (at least in the local registry), and add a query mode restricted to those. May include: surfaces / atmospheres / plasma-radio / exoplanets / solar / dynamics / various. The request will only be sent to those
  • *** Important: add provider institute logo in green boxes - can be found in registry declaration (or Gitlab).
  • We also have to separate creator of service and creator of content - required by many data providers. See if something fits in RDA recommendations, propose to IVOA vocabularies.

Individual service page (custom service)

  • *** Add a function to retrieve the existing values of all parameters, or min/max values for numerical fields, TBC (feasible to some extent in ElasticSearch portal)

Evolution of  Results from service page (table)

We currently plot only the first 10-1000 results, but the VOtable transmitted from this page contains all results


  • "Download selection" button is implemented (now also on production version) - provides a zip file with results. Check if this requires protection from funny requests.
  • Fix round-off values after conversion - only keep a reasonable number of figures (e.g. in spectral range, not only)
  • Add time out or message if pb (depending on size of retrieved data set?) - currently returns an error
  • JSON output needed (for surfaces). Partially implemented in DaCHS (&FORMAT=json in request). Additional formatting may be required (see with Chiara Marmo). — she actually wanted geojson
  • Need for a data merging function: add a "Merge and Send" function in menu "Data Selection" - would call a script to join tables based on first column, possibly with resampling (STILTS or CASSIS can resample). Most useful for spectra, requested by Grenada service.
  • add a button to show / hide duplicates - i.e., granules with identical values for the parameters displayed on this page (this would allow filtering multiple observations of the same target in the TNO service, for instance). default = hide (suggestion from Florence)
  • Result VOTable should not be systematically named sync, should at least include an extension (settable in DaCHS? - Json output is called sync.json - if not, change in client?) - now called sync.xml

  • Convert spectral resolution & sampling step from Hz to µm and back - involves 2 parameters, can be done with TAP request (see mail dated 1/8/2017) - obsolete with change of spectral_resolution
  • Add conversion to cm-1 for spectral range (same conversion as in the query form)
  • Add either info to data service (server URL + schema - this is almost invisible otherwise) or link to this in Taphandle, on option (button or info panel)
  • Check why ds9 does not receive fits images from VESPA (2019: OK from HST_planeto, but HTML error in ds9 - seems to be on their side)
  • Can we use a js9 based application to handle 2D spectra from APIS (instead of dying SpecView)? - does not seem to handle these files 
  • Implement support for datalink, similar to TAP handle
  • Connect Planetary Cesium Viewer as well as Mizar to display bounding boxes - should even plot s_region in fact - this requires a CRS in the geojson output, TBD — obsolete with MOC usage
  • Make subgranule_url module (in CRISM service) work like in TAPhandle - obsolete, solution never reused
  • Support for multiple table resources: pass them one by one when SAMPing (required to support VizieR service) — OK from Vizier_planets service to TOPCAT, but some limitations on TOPCAT side
  • SAMP buttons: preserve possibility to select the data type (e.g., to send spectra in TOPCAT: choose table instead of spectrum), but format should be handled automatically (TOPCAT makes a difference between fits and VOtable). 

Integrated service answers

At the onset of EPN2024, this is implemented in a common table merging the first answers from all services. This is an additional box in the Query results page (no table display):

  • a specific parameter is added to identify the service of origin for each granule - currently the service name, but this relies on services compliancy TBC (ivoID would be better)
  • Include TAP query in the result VOtable header
  • there is currently no associated table display (displaying Results from all services)? — it is demanding to cross services on a single page, and little added value; can be done in TOPCAT
  • But still… can we drop a EPNCore VOTable and see it in service mode ?
  • check whether supporting coordinated observations require any special function. This must rely on v2 parameters target_time_min / max + target_name

This function may require to build a dynamic db merging all results - but workaround in 2019 test version
Another solution is to join the individual result tables with stilts, or even during the TAP query - done in the portal in 2019 test version (with astropy)

Use cases to consider for service interoperability:

Dec 2023: Those look like use cases for Jupyter notebooks, rather than for the portal

0) Interesting user request: how do we retrieve data on all Jupiter satellites?

An answer is to query quaero with: (: required after system, not =)

1) Retrieve spectra of asteroids located 3-4 AU from the Sun

  • This assumes the existence of a service providing orbital parameters of asteroids (now OK with MPC)
  • Get distances from this, and use this parameter to filter results from the spectral service (e.g. M4ast)
  • The only common key is target_name - but this still has to be made consistent!

2) Compare observations and simulations

  • Send a query to SPICAM profiles service, grab profiles of interest
  • Get the exact observational parameters (season, local time, location)
  • Send another query using these parameters to a simulation service (MCD)

In both cases, this involves selecting granules of interest and 1 (or several) parameters in the result table, and sending a new query from VESPA to all services.
The second case is more complicated (several parameters) and may require a pipeline (TOPCAT, Jupyter nb?…).
We need to provide at least guidelines on how to do this with real examples.

Ideally, I'd like to click and select parameters in the SPICAM results page to retrieve their values and insert them in a new query.

1) Query all services with
dataproduct = profiles
location : as below

WHERE c1max >= 0.0 AND c1min <= 40.0 AND c2max >= 0.0 AND c2min <= 30.0 AND (dataproduct_type LIKE '%pr%') AND (lower(target_name)= lower('Mars') OR lower(target_name)= lower('4'))

+ add range in solar longitude and local time (not currently possible!)

2) display results from SPICAM and MCD
not currently possible: missing a button in All results to open a service results page in another window. Fixable in the interface ?

3) If selection is accurate enough:
send corresponding profiles to TOPCAT, plot together and compare

2b) Alternatively:

focus on SPICAM results, and enter magic mode.
Clicking Solar longitude would select both min /max values of current line and include them as range in a new query.
Idem for other parameters.

+ Select service #2 (here: MCD)
+ Issue query and plot results in another window

Alt: send new query to all services, plot all results together.

User feedback (through JCM)

- Facet search to be implemented, with number of results updated in real time. Avoid displaying situations with 0 answers

- OPUS is definitely a good model, especially search system & table vs thumbnails views

- Query tab appears too complicated: parameter names are no familiar, possible values are uncertain. Remove secondary parameters from the top panel (only processing level is left there)
In fact, parameters names could be replaced by more familiar ones (target, instrument, observatory/spacecraft…) and could appear only in the contextual help box (already done for Location tab).

- Processing level should be limited to 0-6 range, TBC (no negative values at least)

All this is handled in SpaceFrog study.

  • No labels