EPN2024-RI


eEUROPLANET 2024 Research Infrastructure 

H2020-INFRAIA-2019-1 

Grant agreement no: 871149


Document: VESPA-WP6-2-040-TN-v1.2(46)


DOI: 10.25935/7JJ8-V683






Individual Repository for VESPA Service Resource Descriptor in DaCHS




Date: $action.dateFormatter.formatGivenString("yyyy-MM-dd",$content.getLastModificationDate())


Start date of project: 01 February 2020

 Duration: 48 Months

Responsible WP Leader: Stéphane Erard


Project co-funded by the European Union's Horizon 2020 research and innovation programme

Dissemination level

PU

Public

  •  

PP

Restricted to other programme participants (including the Commission Service)

  •  

RE

Restricted to a group specified by the consortium (including the Commission Services)

  •  

CO

Confidential, only for members of the consortium (excluding the Commission Services)

  •  

Project Number

871149

Project Title

EPN2024 - RI

Project Duration

48 months: 01 February 2020 – 31 January 2024

Document Number

WP6-task2-040-v1.2

Delivery date

 

Title of Document

Individual Repository for VESPA Service Resource Descriptor in DaCHS

Contributing Work package (s)

WP6

Dissemination level

PU

Author (s)

Baptiste Cecconi

Abstract: This document presents the process to prepare the individual VESPA resource descriptor repository.



Document history (to be deleted before submission to Commission)

Date

Version

Editor

Change

Status

 

0.1

Initial version

DRAFT

 

1.0

updated example section

RELEASED

 

1.1

Baptiste Cecconi

updated gitlab repo URL

RELEASED

 

1.2

Baptiste Cecconi 

added git clone step

RELEASED


Table of Contents

Introduction

The VESPA infrastructure is based on the Table Access Protocol (TAP), developed and maintained by the International Virtual Observatory Alliance (IVOA). A dedicated metadata dictionary has been developed (EPNcore), which defines a set of keywords useful for solar and planetary data discovery. We promote the use of an Virtual Observatory (VO) server framework, namely Data Center Helper Suite (DaCHS), developed by the German Astronomical Virtual Observatory (GAVO) in Heidelberg. 

The document presents how to prepare and store the individual VESPA service resource descriptors for DaCHS. 

Architecture using git

In the DaCHS framework architecture, each service is described in a dedicated directory on the server, located in the /var/gavo/inputs directory. We propose to set up a git repository in each VESPA service directory. 

Each VESPA service directory contains at least a resource descriptor (RD) file (with a .rd extension). See Implementing a VESPA service for more details on building a RD file.  The VESPA service directory may contains other files and directories, depending on the RD capabilities and the selected metadata ingestion process. All the files required to setup the service should be included into the git repository. 

The VESPA service directory git repository is mirrored on an online git facility (such as Gitlab or Github). The VESPA team is proposing to host the repository mirrors on there own gitlab servers, allowing sharing access (and restricting access when necessary). The corresponding gitlab repository is used as a working copy, where several branches can be maintained (for developing new capabilities), and gitlab issues can be raised to discuss problems, fixes, proposition, improvements... This architecture allows to easily deploy prototype services on sandbox servers for testing purposes and deploy finalized versions (eventually using automated testing, and deployment capabilities of code releases). 


Figure 1. Architecture diagram

Since we propose to manage the VESPA service directory git repository through gitlab, each repository should contain a README.md file, briefly presenting the service and the deployment. Extra deployment scripts may be included (e.g., scripts to download data, if stored in an external repository).  In this case, git capabilities such as .gitignore (to exclude files added with the deployment scripts) or .gitkeep (to keep empty directories in preparation for the deployment) should be used. 

A licence should also be selected and advertised.

Example

This example shows the process for the planets service, a very simple VESPA service, hosted on voparis-tap-planeto.obspm.frThe gitlab repository is located at: https://voparis-gitlab.obspm.fr/vespa/dachs/services/padc/voparis-tap-planeto/planets

On the voparis-tap-planeto.obspm.fr server,  the installation goes this way:

dachsroot@voparis-tap-planeto:~$ cd /var/gavo/inputs/
dachsroot@voparis-tap-planeto:/var/gavo/inputs$ git clone https://voparis-gitlab.obspm.fr/vespa/dachs/services/padc/voparis-tap-planeto/planets.git
dachsroot@voparis-tap-planeto:/var/gavo/inputs$ cd planets
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ ls
data	q.rd	README.md	LICENSE
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ ls data
Masses2019.csv

After some modifications on the gitlab repository, the new version is fetched (git pull), and deployed (dachs imp q): 

dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ git pull
remote: Enumerating objects: 33, done.
remote: Counting objects: 100% (33/33), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 30 (delta 7), reused 22 (delta 5)
Unpacking objects: 100% (30/30), done.
From https://gitlab.obspm.fr/vespa/voparis-tap-planeto/planets
   2f3ba6d..03a8ffd  master     -> origin/master
Updating 2f3ba6d..03a8ffd
Fast-forward
 LICENSE            | 621 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 README.md          |  14 +++
 data/Masses2.csv   |   9 --
 q.rd               | 339 +++++++++++++++++++++++++++++++-------------------------------
 4 files changed, 807 insertions(+), 343 deletions(-)
 create mode 100644 LICENSE
 create mode 100644 README.md
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ dachs imp q
Making data planets/q#import
Starting /var/gavo/inputs/planets/data/Masses2019.csv
Done /var/gavo/inputs/planets/data/Masses2019.csv, read 8
Shipped 8/8
Create index Primary key on planets.epn_core
Rows affected: 8

On the gitlab server, the content shows:

  • No labels