EPN2024-RI
eEUROPLANET 2024 Research Infrastructure
H2020-INFRAIA-2019-1
Grant agreement no: 871149
Document: VESPA-WP6-2-040-TN-v1.2(46)
DOI: 10.25935/7JJ8-V683
Individual Repository for VESPA Service Resource Descriptor in DaCHS
Date: $action.dateFormatter.formatGivenString("yyyy-MM-dd",$content.getLastModificationDate())
Start date of project: 01 February 2020
Duration: 48 Months
Responsible WP Leader: Stéphane Erard
Project co-funded by the European Union's Horizon 2020 research and innovation programme | ||
Dissemination level | ||
PU | Public | |
PP | Restricted to other programme participants (including the Commission Service) | |
RE | Restricted to a group specified by the consortium (including the Commission Services) | |
CO | Confidential, only for members of the consortium (excluding the Commission Services) |
Project Number | 871149 |
Project Title | EPN2024 - RI |
Project Duration | 48 months: 01 February 2020 – 31 January 2024 |
Document Number | WP6-task2-040-v1.2 |
Delivery date |
|
Title of Document | Individual Repository for VESPA Service Resource Descriptor in DaCHS |
Contributing Work package (s) | WP6 |
Dissemination level | PU |
Author (s) | Baptiste Cecconi |
Abstract: This document presents the process to prepare the individual VESPA resource descriptor repository. |
Document history (to be deleted before submission to Commission) | ||||
Date | Version | Editor | Change | Status |
| 0.1 | Initial version | DRAFT | |
| 1.0 | updated example section | RELEASED | |
| 1.1 | updated gitlab repo URL | RELEASED | |
| 1.2 | added git clone step | RELEASED |
Table of Contents
Introduction
The VESPA infrastructure is based on the Table Access Protocol (TAP), developed and maintained by the International Virtual Observatory Alliance (IVOA). A dedicated metadata dictionary has been developed (EPNcore), which defines a set of keywords useful for solar and planetary data discovery. We promote the use of an Virtual Observatory (VO) server framework, namely Data Center Helper Suite (DaCHS), developed by the German Astronomical Virtual Observatory (GAVO) in Heidelberg.
The document presents how to prepare and store the individual VESPA service resource descriptors for DaCHS.
Architecture using git
In the DaCHS framework architecture, each service is described in a dedicated directory on the server, located in the
/var/gavo/inputs
directory. We propose to set up a git repository in each VESPA service directory.
Each VESPA service directory contains at least a resource descriptor (RD) file (with a .rd
extension). See Implementing a VESPA service for more details on building a RD file. The VESPA service directory may contains other files and directories, depending on the RD capabilities and the selected metadata ingestion process. All the files required to setup the service should be included into the git repository.
The VESPA service directory git repository is mirrored on an online git facility (such as Gitlab or Github). The VESPA team is proposing to host the repository mirrors on there own gitlab servers, allowing sharing access (and restricting access when necessary). The corresponding gitlab repository is used as a working copy, where several branches can be maintained (for developing new capabilities), and gitlab issues can be raised to discuss problems, fixes, proposition, improvements... This architecture allows to easily deploy prototype services on sandbox servers for testing purposes and deploy finalized versions (eventually using automated testing, and deployment capabilities of code releases).
Figure 1. Architecture diagram
Since we propose to manage the VESPA service directory git repository through gitlab, each repository should contain a README.md
file, briefly presenting the service and the deployment. Extra deployment scripts may be included (e.g., scripts to download data, if stored in an external repository). In this case, git capabilities such as .gitignore
(to exclude files added with the deployment scripts) or .gitkeep
(to keep empty directories in preparation for the deployment) should be used.
A licence should also be selected and advertised.
Example
This example shows the process for the planets
service, a very simple VESPA service, hosted on voparis-tap-planeto.obspm.fr
. The gitlab repository is located at: https://voparis-gitlab.obspm.fr/vespa/dachs/services/padc/voparis-tap-planeto/planets.
On the voparis-tap-planeto.obspm.fr
server, the installation goes this way:
dachsroot@voparis-tap-planeto:~$ cd /var/gavo/inputs/ dachsroot@voparis-tap-planeto:/var/gavo/inputs$ git clone https://voparis-gitlab.obspm.fr/vespa/dachs/services/padc/voparis-tap-planeto/planets.git dachsroot@voparis-tap-planeto:/var/gavo/inputs$ cd planets dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ ls data q.rd README.md LICENSE dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ ls data Masses2019.csv
After some modifications on the gitlab repository, the new version is fetched (git pull
), and deployed (dachs imp q
):
dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ git pull remote: Enumerating objects: 33, done. remote: Counting objects: 100% (33/33), done. remote: Compressing objects: 100% (24/24), done. remote: Total 30 (delta 7), reused 22 (delta 5) Unpacking objects: 100% (30/30), done. From https://gitlab.obspm.fr/vespa/voparis-tap-planeto/planets 2f3ba6d..03a8ffd master -> origin/master Updating 2f3ba6d..03a8ffd Fast-forward LICENSE | 621 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 14 +++ data/Masses2.csv | 9 -- q.rd | 339 +++++++++++++++++++++++++++++++------------------------------- 4 files changed, 807 insertions(+), 343 deletions(-) create mode 100644 LICENSE create mode 100644 README.md dachsroot@voparis-tap-planeto:/var/gavo/inputs/planets$ dachs imp q Making data planets/q#import Starting /var/gavo/inputs/planets/data/Masses2019.csv Done /var/gavo/inputs/planets/data/Masses2019.csv, read 8 Shipped 8/8 Create index Primary key on planets.epn_core Rows affected: 8
On the gitlab server, the content shows: