Tech/Framework | Version | Programming Language | GOBii Module | Owner | Notes |
---|---|---|---|---|---|
Postgresql | 9.5 | SQL, Pl/pgsql | Data warehouse | plans of upgrading to 12 within the next few months | |
Liquibase | 3.5.1 | SQL with annotations | Data warehouse | used for database versioning and migration | |
HDF5 | 1.8.18 | C/C++ | Data warehouse | stores genotype data | |
Psycopg2 | 2.6.2 | Python | Data warehouse | used by the ETL, might get updated with Postgres12 | |
Docker (with swarm) | 18.06.1-ce | n/a | DevOps | (almost) everything GOBii is now containerized | |
Ubuntu | 18.04 LTS | n/a | DevOps | As of GDM 2.2, all our containers run this version of Ubuntu | |
Tomcat | 9.0.27 | n/a | DevOps | As of GDM 2.2, all our containers run this version of Tomcat | |
Atlassian Bamboo | 6.6.0 | Shell scripting (bash) | DevOps | main CI/CD pipeline | |
Selenium | 3.0.2 | Python | DevOps | used for testing the extractorUI | |
ZK CE | 8.6.0.1 | Java | Timescope | UI development for TImescope | |
Angular | 4.2.4 | Typescript | ExtractorUI, Web LoaderUI, PortalUI, Job StatusUI | Vishnu Govindaraj, PFR | UI development for the ExtractorUI |
Vue | 2.6.10 | Javascript | PedVer, | UI development for Pedver | |
Leiningen | 2.9.1 | Clojure / Bash | Bert & Ernie | Bash script for building Clojure Projects | |
JOOQ | 3.10.4 | Java | Timescope | used for fetching data from the database | |
Java SE | OpenJDK 13 | Java | Middleware (Digesters, Extractors) | ||
Linux Cron | Ubuntu 18.04's cron | Bash | Middleware (job queue) | currently just simple configurable cronjobs, but we plan on migrating to a job scheduler in the future | |
SMTP | any email provider that supports SMTP | Java | Notification system | GDM needs an email account to send out notifications on job status. We typically use GMail but it can be any email provider. We plan on moving to a purely web-based notification system in the future (will come with the Job StatusUI). | |
Spring MVC | 4.3.0 | Java | GDM Web services | GDM Web services and Brapi Web Services | |
Hibernate | 5.4.10 | Java | GDM Web Services | Object Relational Mapping for Web service Data Access Layer. | |
Celery | 4.4.0 | Python3 | F1 Pedver | Framework to run analytics pipeline jobs like Filtering, Consensus calling and Pedigree Verification. | |
Flask | 1.1.0 | Python3 | F1 Pedver | F1 Pedver tool API to submit jobs with required data. | |
Redis | 5.0 | Redis | F1 Pedver | Acts as a Message broker between Analytics pipeline and backend web services. | |
Junit | 4 | Java | F1 Pedver | Testing Framework for GDM Webservices. Along with that we use Mockito and Spring Test which accessory Frameworks. | |
GUnicorn | Python | F1 PedVer | Python WSGI HTTP Server for UNIX |
A general overview of the architecture:
Disregard MonetDB as we've dropped support for that.
Legend
- IFL = Intermediate File Loader
- H5L = HDF5 Loader
- MDE = Metadata Extractor
- H5E = HDF5 Extractor
- JDBC = Java Database Connectivity
- API = Application Programming Interface
- JSON = JavaScript Object Notation
- Arrows represent data flow
To update:
- MonetDB is no longer used and should be removed from this diagram
- There is work being done on making the LoaderUI a web app, once that is done, this diagram should be updated accordingly.