GDM Features

 

Feature

GDM Standalone

GDM-EBS

Effort

Priority

Notes

 

Feature

GDM Standalone

GDM-EBS

Effort

Priority

Notes

1

Configuration

Self-contained containers with no need for the gobii_bundle, gobii-web.xml, and crons

?

yes

large

 

For maximum portability and minimum failure points

2

 

Docker swarm and docker compose support

?

yes

 

 

Each module can be deployed independently on servers (swarm) or local machines (compose)

3

DB

Updated PostgreSQL

PS 9

ps 13

medium

High - PG 9.5 is EOL since months ago.

Things can break during the upgrade and will need thorough testing

4

 

simplified storage

planned

planned

large

 

 

5

 

polyploid support

2 & 4

2 & 4

medium

 

 

6

 

indel support

yes

planned

low

 

 

7

 

self-contained swarm-compatible docker container

no

yes

medium

 

 

8

 

Meta Schema (job, crop, and template modules)

no

yes

medium

 

https://ebsproject.atlassian.net/wiki/spaces/DB/pages/130547813/Meta+Schema

9

 

Genotype on Postgres data-length flexible (enables any ploidy and indels support)

no

ONGOING

large

 

This removes the hybrid-DB by effectively removing the need for HDF5.

10

Compute / Process (in GOBii-EBS, the process node = compute+service)

Serverless

cron

Rabbitmq

medium

 

In GOBii-EBS, scheduling is handled by CS' RabbitMQ and goes to GOBii as a job to be executed immediately - removing the need for crons.

11

 

Aspects

yes

yes

 

 

 

12

 

Validation

yes

yes

large

 

@Joshua Lamos-Sweeney, please update - I think we’ve made quite the progress with collab work with Faustino and Cesar. I believe there are multiple levels of validation (outside and within GOBii).

Josh: Honestly, all I’ve done is add the standalone validations. There’s a lot to do for Faustino’s pre-validation steps, and what is possible, and what is feasible, is something to discuss. I made a wiki document somewhere, but basically all of it is manual steps and ‘does this make sense’ checks, that I suggested we create as ‘warnings’, which I’ve got some framework for in EBS-Gobii, but that’s going to be an absolute pain to do in standalone.

13

 

QC

yes

no

low

 

@Joshua Lamos-Sweeney here too

Josh- De-nada’s been implemented to do the QC pipeline in EBS standalone, so… yeah. Getting Standalone’s QC pipeline into the workflows and systems EBS wants… could be fun.

14

 

job status

yes

yes

low

 

Implementation is different. Job tracking in GOBii-EBS is aligned to the jobs from RabbitMQ, hence we store job_ids in the meta schema.

15

 

automated data loading

?

ONGOING

medium

 

aka, place a curated file in a directory, EBS picks it up and loads it to GOBii automagically

16

 

self-contained swarm-compatible docker container

no

ONGOING

 

 

 

17

Service

crop agnostic service

yes

yes

large

 

With automated data loading in GOBii-EBS, there is no crop dependency - each load or extract request is stateless and self-contained.

18

 

BrAPI v2.0

yes

no

large

high

 

19

 

unified authentication

keycloak

ws02

 

 

 

20

 

load service

API

Direct process call

medium

 

 

21

 

load templates

yes

YES

low

 

More templates being created via aspects

22

 

load file formats

intertek, VCF, hmp, generic

intertek, VCF, hmp, generic

low

 

@Joshua Lamos-Sweeney - I believe we support all the other formats in GOBIi-EBS too, we just need to write the aspect files. Correct me if I’m wrong.

Josh - Yep. Same as templates. VCF loading would need a slight bit of programmatic love, though, to work in the modern standalone way. (We can jimmy the old broken version, but to add the HTSJDK jars will require some retrofitting)

23

 

extract service

BrAPI

DIRECT PRocess call

large

 

 

24

 

job tracking

yes

YES

medium

 

Different implementation

25

Integrations / WEB UIs

Extract UI

yes

no

large

 

 

26

 

Loader UI

yes

no

large

 

 

27

 

CAST

yes

no

low

 

 

28

 

Flapjack Bytes

yes

no

low

 

 

29

 

Timescope

yes

YES

low

 

Timescope, as a DB tool, links directly to the DB so it works on both.

30

 

KDCompute

Yes

no

low