Page tree
Skip to end of metadata
Go to start of metadata

As of Dec 2017, the VESPA portal is supporting the 2 existing PDAP servers, at PSA and DARTS (see comment about NASA below).

However, these servers only accept limited queries, and the PDAP protocol seems intrinsically unable to handle detailed queries as EPN-TAP does (no wonder - this is why we developed EPN-TAP in the first place).

This page gathers pros and cons about continuing support of PDAP in the VESPA portal.

1. Initial assessment of PDAP in Europlanet

An initial assessment of PDAP was performed during the FP7 contract in the IDIS activity (Feb 2011). This relied in particular on a PDAP client developed and installed at PADC (ObsParis), and focused on the capacity of PDAP to retrieve individual data products in data archives (including, but not limited, to space missions), based on operational and observational parameters (what EPN-TAP is now doing).

The assessment document (PDAP_assessment_v5.pdf) was discussed during an IPDA meeting in Oct 2011; some items were agreed upon and implemented in a later update of the PDAP protocol document, other were not. In particular, there was no intention to enlarge PDAP to support data not included in space archives (e.g. telescopic ones, but also derived data from space missions), and a clear will to stick to PDS3 references (dictionary, parameters, etc). Besides, no solution was in view to read PDS3 data on-line.

Altogether, the IDIS team concluded that PDAP would never be flexible enough to allow the type of search sought by Europlanet, and decided to develop a restriction of IVOA's TAP protocol instead (EPN-TAP), similar to ObsTAP for Astronomy. This was later refined and improved by VESPA in Europlanet2020 (EPN-TAP v2).

2. Current implementation of PDAP in VESPA

• The initial PDAP client in PADC is still accessible at the time of writing (12/2017), but no longer supported:

• The VESPA portal has included from the start a similar function. The EPN-TAP query entered in the query form is translated to a minimal PDAP request. This was initially passed to the 2 existing servers supporting the protocol (DARTS at Jaxa and PSA at ESA), the results of which are listed at the bottom of the service result page after all EPN-TAP services.

Given the answer to such queries, the VESPA PDAP client capacities were reduced progressively to avoid constant errors. PDAP queries were limited to dataset (not dataproducts); the PSA service was deactivated at some point, then reactivated in Dec 2018 in the updated VESPA portal, after an update of the PDAP service at ESA. We now consider removing this function altogether because DARTS is returning mostly errors or impractical results.

• Later analyses of the situation led to the conclusion that PDAP is only used as low-level tool for automatic updates/mirroring of datasets between archives. Practically, PDAP seems to be only able to search by dataproduct_name, instrument_name or time range. The current capacities of PDAP therefore cannot fulfill the functions provided by EPN-TAP, and will not do so until a big development is made to define missing topical extensions.

EPN-TAP now covers mirroring functions efficiently, with the addition of the modification_date parameter in 2014, and the File grabbing interface (fgi) (scripts locating files distributed in an EPN-TAP service) in July 2017.

Examples of functioning from the VESPA portal:

(as of Jan. 2020)

  • Query with a target name => both PSA and DARTS reply with a list of datasets containing the target. This can be send to TOPCAT via SAMP or downloaded as VOTable.
  • Query with no target name => only the PSA replies with a similar list. DARTS is in error.
  • Query with instrument_host_name and instrument_name => PSA answers correctly, DARTS provides no answers.
  • Other parameters result in errors from DARTS. No error is returned by the PSA service but these parameters are ignored (not used to filter the data).
  • In all cases, NASA PDAP interface only returns an incorrect VOtable which is not interpreted as valid xml by any software. This interface is not maintained and will not be fixed, and therefore PDAP access to NASA archives cannot be implemented in the portal.

3. Future support of PDAP in VESPA

• Given the high score of errors in PDAP queries, the non-working NASA access, and the practical impossibility to locate a file corresponding to given conditions, maintaining the PDAP client in VESPA is now considered problematic - it mostly projects a degraded picture of both PDAP and VESPA capacity access to space data archives.

In consequence, we propose to drop PDAP support in the VESPA portal until possible improvements are implemented both in the protocol and the servers.

  • No labels

1 Comment

  1. Later remark: when trying to connect the PDS PDAP service from the portal, we find out another kind of problem. When using RETURN_TYPE= VOTABLE (or no RETURN_TYPE) the result is a badly formatted XML file (VOTABLE v1.1, with missing = in the first line). When using RETURN_TYPE= any other value, the return is a funny XML table (not a VOTABLE).

    Hence: at the present time, the VESPA portal simply can't get a usable answer from the NASA PDAP interface.

Write a comment...