Individual Repository for VESPA Service Resource Descriptor in DaCHS

EPN2024-RI

eEUROPLANET 2024 Research Infrastructure 

H2020-INFRAIA-2019-1 

Grant agreement no: 871149

Document: VESPA-WP6-2-040-TN-v1.2(42)

DOI: 10.25935/7JJ8-V683

Individual Repository for VESPA Service Resource Descriptor in DaCHS

Date: 2021-11-25

Responsible WP Leader: Stéphane Erard

Table of Contents

Introduction

The VESPA infrastructure is based on the Table Access Protocol (TAP), developed and maintained by the International Virtual Observatory Alliance (IVOA). A dedicated metadata dictionary has been developed (EPNcore), which defines a set of keywords useful for solar and planetary data discovery. We promote the use of an Virtual Observatory (VO) server framework, namely Data Center Helper Suite (DaCHS), developed by the German Astronomical Virtual Observatory (GAVO) in Heidelberg. 

The document presents how to prepare and store the individual VESPA service resource descriptors for DaCHS. 

Architecture using git

In the DaCHS framework architecture, each service is described in a dedicated directory on the server, located in the /var/gavo/inputs directory. We propose to set up a git repository in each VESPA service directory. 

Each VESPA service directory contains at least a resource descriptor (RD) file (with a .rd extension). See Implementing a VESPA service for more details on building a RD file.  The VESPA service directory may contains other files and directories, depending on the RD capabilities and the selected metadata ingestion process. All the files required to setup the service should be included into the git repository. 

The VESPA service directory git repository is mirrored on an online git facility (such as Gitlab or Github). The VESPA team is proposing to host the repository mirrors on there own gitlab servers, allowing sharing access (and restricting access when necessary). The corresponding gitlab repository is used as a working copy, where several branches can be maintained (for developing new capabilities), and gitlab issues can be raised to discuss problems, fixes, proposition, improvements... This architecture allows to easily deploy prototype services on sandbox servers for testing purposes and deploy finalized versions (eventually using automated testing, and deployment capabilities of code releases). 

Figure 1. Architecture diagram

Since we propose to manage the VESPA service directory git repository through gitlab, each repository should contain a README.md file, briefly presenting the service and the deployment. Extra deployment scripts may be included (e.g., scripts to download data, if stored in an external repository).  In this case, git capabilities such as .gitignore (to exclude files added with the deployment scripts) or .gitkeep (to keep empty directories in preparation for the deployment) should be used. 

A licence should also be selected and advertised.

Example

This example shows the process for the planets service, a very simple VESPA service, hosted on voparis-tap-planeto.obspm.fr. The gitlab repository is located at: https://voparis-gitlab.obspm.fr/vespa/dachs/services/padc/voparis-tap-planeto/planets. 

On the voparis-tap-planeto.obspm.fr server,  the installation goes this way:

dachsroot@voparis-tap-planeto:~$ cd /var/gavo/inputs/
dachsroot@voparis-tap-planeto:/var/gavo/inputs$ git clone https://voparis-gitlab.obspm.fr/vespa/dachs/services/padc/voparis-tap-planeto/planets.git
dachsroot@voparis-tap-planeto:/var/gavo/inputs$ cd planets
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ ls
data	q.rd	README.md	LICENSE
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ ls data
Masses2019.csv

After some modifications on the gitlab repository, the new version is fetched (git pull), and deployed (dachs imp q): 

dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ git pull
remote: Enumerating objects: 33, done.
remote: Counting objects: 100% (33/33), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 30 (delta 7), reused 22 (delta 5)
Unpacking objects: 100% (30/30), done.
From https://gitlab.obspm.fr/vespa/voparis-tap-planeto/planets
   2f3ba6d..03a8ffd  master     -> origin/master
Updating 2f3ba6d..03a8ffd
Fast-forward
 LICENSE            | 621 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 README.md          |  14 +++
 data/Masses2.csv   |   9 --
 q.rd               | 339 +++++++++++++++++++++++++++++++-------------------------------
 4 files changed, 807 insertions(+), 343 deletions(-)
 create mode 100644 LICENSE
 create mode 100644 README.md
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ dachs imp q
Making data planets/q#import
Starting /var/gavo/inputs/planets/data/Masses2019.csv
Done /var/gavo/inputs/planets/data/Masses2019.csv, read 8
Shipped 8/8
Create index Primary key on planets.epn_core
Rows affected: 8

On the gitlab server, the content shows: