GDM Features

Configuration

Self-contained containers with no need for the gobii_bundle, gobii-web.xml, and crons

For maximum portability and minimum failure points

Docker swarm and docker compose support

Each module can be deployed independently on servers (swarm) or local machines (compose)

DB

Updated PostgreSQL

High - PG 9.5 is EOL since months ago.

Things can break during the upgrade and will need thorough testing

simplified storage

polyploid support

indel support

self-contained swarm-compatible docker container

Meta Schema (job, crop, and template modules)

status:medium

https://ebsproject.atlassian.net/wiki/spaces/DB/pages/130547813/Meta+Schema

Genotype on Postgres data-length flexible (enables any ploidy and indels support)

status:large

This removes the hybrid-DB by effectively removing the need for HDF5.

Compute / Process (in GOBii-EBS, the process node = compute+service)

Serverless

In GOBii-EBS, scheduling is handled by CS' RabbitMQ and goes to GOBii as a job to be executed immediately - removing the need for crons.

Aspects

Validation

@Joshua Lamos-Sweeney, please update - I think we’ve made quite the progress with collab work with Faustino and Cesar. I believe there are multiple levels of validation (outside and within GOBii).

Josh: Honestly, all I’ve done is add the standalone validations. There’s a lot to do for Faustino’s pre-validation steps, and what is possible, and what is feasible, is something to discuss. I made a wiki document somewhere, but basically all of it is manual steps and ‘does this make sense’ checks, that I suggested we create as ‘warnings’, which I’ve got some framework for in EBS-Gobii, but that’s going to be an absolute pain to do in standalone.

QC

@Joshua Lamos-Sweeney here too

Josh- De-nada’s been implemented to do the QC pipeline in EBS standalone, so… yeah. Getting Standalone’s QC pipeline into the workflows and systems EBS wants… could be fun.

job status

Implementation is different. Job tracking in GOBii-EBS is aligned to the jobs from RabbitMQ, hence we store job_ids in the meta schema.

automated data loading

status:medium

aka, place a curated file in a directory, EBS picks it up and loads it to GOBii automagically

self-contained swarm-compatible docker container

Service

crop agnostic service

With automated data loading in GOBii-EBS, there is no crop dependency - each load or extract request is stateless and self-contained.

BrAPI v2.0

status:high

unified authentication

load service

load templates

More templates being created via aspects

load file formats

@Joshua Lamos-Sweeney - I believe we support all the other formats in GOBIi-EBS too, we just need to write the aspect files. Correct me if I’m wrong.

Josh - Yep. Same as templates. VCF loading would need a slight bit of programmatic love, though, to work in the modern standalone way. (We can jimmy the old broken version, but to add the HTSJDK jars will require some retrofitting)

extract service

job tracking

Different implementation

Integrations / WEB UIs

Extract UI

Loader UI

CAST

Flapjack Bytes

Timescope

Timescope, as a DB tool, links directly to the DB so it works on both.

KDCompute