GOBii Applications and Modules
This a comprehensive list of the GOBii project's current modules and applications, along with their current version and status. @KevinPalis will keep the version and status as up-to-date as possible.
Module / Application | Owner / Point Person | Description | Latest Version | Status [Deployed To] | Packaging | Notes | |
---|---|---|---|---|---|---|---|
1 | Data Warehouse (PostgreSQL) | Kevin Palis | The main data warehouse as modeled here: Crop Schema This also includes the Data Access Layer. For more info, see the main documentation space here: GOBii Data Warehouse | 2.2 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | database container | This module is actively being developed and improved. It ships with the entire GDM stack and so versioning is the same. |
2 | HDF5 Storage for Genotype | Dave Matthews | The main storage of the genotype matrix. This includes some C libraries written for the creation and extraction of HDF5 files for GOBii. We have yet to update its documentation in our Confluence cloud account. | 2.2 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | database container | This module is actively being developed and improved. It ships with the entire GDM stack and so versioning is the same. |
3 | Digesters | Josh Lamos-Sweeney | This is the middleware. It handles file processing and interfaces with the Data Access Layer directly for both loading and extraction of data. More information is available here: Middleware | 2.2 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | processor container (aka compute container) | This module is actively being developed and improved. It ships with the entire GDM stack and so versioning is the same. |
4 | LoaderUI | Yaw Nti-Addae / Angel Raquel | Desktop application for mapping different input files to GOBii data entities and for creating loading jobs. More information is available here: Data Loader | 2.2 | Done and will be uploaded once 2.2 GDM is deployed [CIMMYT, IRRI, ICRISAT] | dowloadable, executable JAR file | This is built on an SWT version that came with Eclipse back in 2016. It has its challenges. It also doesn’t ship with our Docker containers and each user has to download it. The Web Loader UI that PFR is working on, aims to replace this. |
5 | ExtratorUI / Web Services | Vishnu Govindaraj | Web application for creating extraction jobs. More information is available here: Data Extractor | 2.2 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | web container | This module ships with the web container. One ExtractorUI app = one crop, so in cases like ICRISAT having 6 crops, you will see 6 instances of this app. Also, the web services are packaged in the same WAR as the ExtractorUI. |
6 | Timescope | Angel Raquel / Kevin Palis | Web application that can connect to any GDM backend to manage data. Currently, it covers datasets and markers searching and cleanup. You can find the main documentation space here: Timescope Documentation | 2.2 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | web container (with plans of creating an independent container) | Currently, this application covers complex queries and cascading deletions of both dataset and marker entities (i.e. R and D of CRUD). @KevinPalis is proposing this to be a CRUD-complete software for the GDM backend and be packaged independent of the web container. This way we can spin it up anywhere as long as there’s a network route to GDM’s database port. |
7 | BeRT | Luke Cook | BeRT = Backend Regression Test This is a framework specially built to do automated tests of the GOBii backend - from loading to extraction of different scenarios (i.e. different file formats of different vendor files) | 1.0 | Done and integrated to GOBii’s CI/CD pipeline | n/a | This currently doesn’t ship with GDM, but there are plans to package the binaries with one of the containers so we can do an initial automated test upon deployment to verify that the deployment is successful. |
8 | Marker Portal | JHI (Iain Milne) | A simple web page with links to all the applications deployed to a particular server. | 2.2 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | web container | The PortalUI that PFR is working on aims to replace this and add functionalities. |
9 | KDCompute | DArT (Andrew Kowalczyk) | Web application for data analysis that is currently integrated with our loading process (a checkbox in the LoaderUI toggles it on or off for a particular job). More information on KDCompute is available here: http://www.kddart.org/kdcompute.html | 1.6.1 (server) 0.14.3 (plugin) | Deployed [CIMMYT, IRRI, ICRISAT] | KDCompute container | This ships in its own container and the last release version was May of 2019. |
10 | Portainer | n/a (Kevin Palis / Roy Petrie for deployment) | This is a third-party open-source Docker management toolset. More information is available here: https://www.portainer.io/overview/ | 1.23.1 | Done and awaiting deployment [CIMMYT, IRRI, ICRISAT] | Portainer container | This ships in its own container and is completely optional to clients. It merely provides a UI to manage containers - so everything this does can be done in the command line as well. |
11 | OwnCloud | n/a (Roy Petrie for deployment) | This is a third-party client-server software for file-hosting services. We simply deploy the container they provide and link it to GDM’s filesystem. More information on OwnCloud is available here: https://owncloud.org/ | 10.1 | Deployed [CIMMYT, IRRI, ICRISAT] | ownCloud container | This ships in its own container which we apparently re-imaged back in February 2019. We don’t make any modifications to this software and we just run our image of the owncloud server directly. |
12 | FlapJack | JHI (Iain Milne) | Flapjack is a multi-platform application providing interactive visualizations of high-throughput genotype data, allowing for rapid navigation and comparisons between lines, markers and chromosomes. See https://ics.hutton.ac.uk/flapjack/ | 1.19.09.04 | Available for download from JHI’s website | downloadable executables based on OS |
|
13 | Flapjack Bytes | JHI (Iain Milne) | A web application subset of the Flapjack desktop app. This consumes BrAPI web services for data visualization. | n/a
| In development | TBD | This is still under development by JHI. I will update this row when I have more info (such as the release date, tech stack, etc.) |
14 | Galaxy | Alexis Dereeper / Venice Juanilas | Pipeline for genomic selection and GWAS. See http://galaxy-demo.excellenceinbreeding.org/ | TBD | EiB-hosted | n/a | This software is detached from the rest of GOBii’s products. We merely add a link to the Marker Portal to link to the EiB-hosted instance. It is currently hosted in IRRI’s cloud server and funded by EiB. |
15 | DArtView | DArT (Andrew Kowalczyk) | Desktop application that provide basic data quality stats and allow filtering of datasets (e.g. as part of a curation process to remove a bad plate, etc.). More info TBD
| n/a | In development | TBD | This is still under development by DArT and I will update this row when I have more info (such as the release date, tech stack, etc.) |
16 | PedVer | Vishnu Govindaraj | A web application that extracts data from GDM (via BrAPI calls) and generates a Flapjack project file that can be used in a Flapjack application for pedigree verification. | n/a | In development | 3 containers:
| First version will possibly be released with GDM 2.2. It uses GDM BrAPI web services and as such, requires a connection to the web container. |
17 | KeyCloak | Yaw Nti-Addae | Open-source IAM from RedHat. See https://www.keycloak.org/ | n/a | In development | keyCloak container | This is still under development; the design on how it will integrate with GDM is not defined yet. |
18 | Web Configurator | Kevin Palis | This is part of Timescope and our attempt to make GDM configuration manageable (via a UI). This allows modification to LDAP and database connections, GDM email account, logging level; and allows addition/deletion of crops in a GDM instance. | n/a | In development | web container (part of Timescope) | This is dependent on the current architecture of GDM and may require significant rewrite if we are changing the architecture. It is, however, sitting on its own branch, so if we decide not to use this (in light with architectural changes), we simply do not merge it.
|
19 | MonetDB | n/a | This has been long deprecated (around 2017). | n/a | Deprecated | n/a | Despite being deprecated, some of the scripts still ship with GDM. We need to completely remove it from our stack. |
20 | Web Loader UI | PFR / Yaw Nti-Addae | A web application version of the desktop LoaderUI. | n/a | In development | loaderUI container | This is currently being developed by PFR and will ship as a separate container once done. It uses GDM BrAPI web services and as such, requires a connection to the web container. |
21 | Portal UI | PFR / Yaw Nti-Addae | A web application that will replace JHI’s Marker Portal. | n/a | In development | portalUI container | This is currently being developed by PFR and will ship as a separate container once done. |
22 | JobStatus UI | PFR / Yaw Nti-Addae | A web application for tracking jobs in a GDM instance. A rewrite of the current job status module (including DB schema changes) of GDM is necessary for this application to work. | n/a | In development | jobStatusUI container | This is currently being developed by PFR and will ship as a separate container once done. |
Legend:
Developed by the gobii team
developed by collaborators
third party (Off-the-shelf)