SSHADE infrastructure development

SSDM datamodel upgrade

• internal meetings on the datamodel changes implied by the change of structure from single database (GhoSST) to SSHADE database infrastructure
  • discussion on how to separate the common databases (common data: fundamental data + general data) and the lab databases (partner data)
  • discussed: experimentalists, instruments, samples & matters
  • addition of a 'database' table => Done with all necessary DOI keywords
  • ...

SSHADE infrastructure implementation

Software structure

• The development of SSHADE software is based on:
  • Python (https://www.python.org/)
  • Pyramid framework (http://www.pylonsproject.org/)
  • SQLAlchemy (http://www.sqlalchemy.org/)
  • LXML (http://lxml.de/)
  • Astropy (http://www.astropy.org/)
  • ...
• The development of SSHADE database is based on:
  • PostgreSQL (http://www.postgresql.org/)

Database structure

• Software implementation
  • Use of PostgreSQL schemas (OK - completed)
  • Datamodel definition with SQLAlchemy (OK - completed)

• Refactoring of database structure inherited fom GhoSST (OK - completed)
  
  • Split into multiple schemas (OK - completed)
    
    • one schema dedicated to all common data (fundamental species, ...)
    • one schema per "database" for all "private" data (experiments, spectra, ...)
  • Implementation of the datamodel upgrades
    • Experimentalist (OK - completed)
    • Instrument (OK - completed)
    • Sample (OK - completed)
    • Mineral species (OK - completed)
    • Matter (OK - completed)
    • Database (OK - completed)

Data ingestion software

• Import XML files
  • Source
    • Command line (OK - completed)
    • Web interface (OK - completed)
  • Data types
    
    • Species (OK - completed)
    • Natural objects (OK - completed)
    • Laboratory (OK - completed)
    • Experimentalist (OK - completed)
    • Instrument (OK - completed)
    • Database (OK - completed)
    • Sample (OK - completed)
    • Matter (OK - completed)
    • Experiment & spectrum (OK - completed)

    • Publication (OK - completed)

    • Bandlist/Bands/States (TODO)
  • Features
    • XML validation : OK
    • Import mode: correction, use existing, ... (OK - completed)
    • Optional mandatory (OK - completed)
    • Unit conversion (OK - completed)
    • Resource files : image, PDF,... (OK - completed)
    • Conversion of binary file : Nicolet (OK)
    • History (OK - completed)

SSHADE web interface (search / visualization / export)

• • • Google-like Search (OK - completed)
    • Specialized spectrum filters search (OK - completed)
    • Specialized publication filters search (OK - completed)
    • Provider searchs (OK - completed)
    • Spectrum dynamic visu (OK - completed)
    • experiment details visu (OK - completed)
    • spectrum details visu (OK - completed)
    • sample details visu (OK - completed)
    • species details visu (OK - completed)
    • phase details (OK - completed)
    • Publication details (OK - completed)
    • export spectrum / experiment (OK - completed)
    • user dashboard (OK - completed)
    • profiles and settings (OK - completed)

SSHADE web interface (Providers & Managers data management)

• • • import (OK - completed)
    • data validation + access (OK - completed)
    • database management (OK - completed)
    • users management (OK - completed)
    • groups management (TODO)
    • backup (OK - completed)

SSHADE wiki (Users and Providers)

• • • SSHADE documentation (OK - completed)
    • user help (OK - completed)
    • provider documentation (OK - completed)

Hardware / Software management

• "Hardware"
  
  • Deployment of 2 virtual servers (development/production) within the OSUG datacenter infrastructure (OK - completed)
  • Operating system : Debian GNU/Linux 8 (Jessie - Stable)
• Software
  • Deployment of a forge software: Redmine+Git (OK) with plugins for agile project management (Kanban/SCRUM) and continuous integration (Jenkins)