Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Current »


Tech/Framework

Version

Programming Language

GOBii Module

Owner

Notes

Postgresql

9.5

SQL, Pl/pgsql

Data warehouse

Kevin Palis

plans of upgrading to 12 within the next few months

Liquibase

3.5.1

SQL with annotations

Data warehouse

Kevin Palis

used for database versioning and migration

HDF5

1.8.18

C/C++

Data warehouse

Dave Matthews

stores genotype data

Psycopg2

2.6.2 or above

Python

Data warehouse

Kevin Palis

used by the ETL, might get updated with Postgres12

Docker (with swarm)

18.06.1-ce

n/a

DevOps

Kevin Palis Roy Petrie

(almost) everything GOBii is now containerized

Ubuntu (Containers)

18.04 LTS

n/a

DevOps

Kevin Palis Roy Petrie

As of GDM 2.2, all our containers run this version of Ubuntu

Ubuntu (Hosts)

16.04.6 LTS

n/a

DevOps

Roy Petrie , Kevin Palis

version of OS used for hosts

Bash

4.3.48(1)

bash

DevOps

Roy Petrie

Version of bash for scripts/deployment

git

2.7.4

n/a

DevOps

Roy Petrie

git version

Tomcat

9.0.27

n/a

DevOps

Kevin Palis

As of GDM 2.2, all our containers run this version of Tomcat

Atlassian Bamboo

6.6.0

Shell scripting (bash)

DevOps

Kevin Palis

main CI/CD pipeline

Selenium

3.0.2

Python

DevOps

Kevin Palis

used for testing the extractorUI

ZK CE

8.6.0.1

Java

Timescope

Angel Raquel

UI development for TImescope

Angular

4.2.4

Typescript

ExtractorUI, Web LoaderUI, PortalUI, Job StatusUI

Vishnu Govindaraj, PFR

UI development for the ExtractorUI

Vue

2.6.10

Javascript

PedVer,

Angel Raquel

UI development for Pedver

Leiningen

2.9.1

Clojure / Bash

Bert & Ernie

Luke Cook

Bash script for building Clojure Projects

JOOQ

3.10.4

Java

Timescope

Angel Raquel

used for fetching data from the database

Java SE

OpenJDK 13

Java

Middleware (Digesters, Extractors)

Josh Lamos-Sweeney


Linux Cron

Ubuntu 18.04's cron

Bash

Middleware (job queue)

Josh Lamos-Sweeney

currently just simple configurable cronjobs, but we plan on migrating to a job scheduler in the future

SMTP

any email provider that supports SMTP

Java

Notification system

Josh Lamos-Sweeney

GDM needs an email account to send out notifications on job status. We typically use GMail but it can be any email provider.

We plan on moving to a purely web-based notification system in the future (will come with the Job StatusUI).

Spring MVC

4.3.0

Java

GDM Web services

Vishnu Govindaraj

GDM Web services and Brapi Web Services

Hibernate

5.4.10

Java

GDM Web Services

Vishnu Govindaraj

Object Relational Mapping for Web service Data Access Layer.

Celery

4.4.0

Python3

F1 Pedver

Vishnu Govindaraj

Framework to run analytics pipeline jobs like Filtering, Consensus calling and Pedigree Verification.

Flask

1.1.0

Python3

F1 Pedver

Vishnu Govindaraj

F1 Pedver tool API to submit jobs with required data.

Redis

5.0

Redis

F1 Pedver

Vishnu Govindaraj

Acts as a Message broker between Analytics pipeline and backend web services.

Junit

4

Java

F1 Pedver

Vishnu Govindaraj

Testing Framework for GDM Webservices. Along with that we use Mockito and Spring Test which accessory Frameworks.

GUnicorn

Python

F1 PedVer

Vishnu Govindaraj

Python WSGI HTTP Server for UNIX

amqp==2.5.2
appnope==0.1.0
backcall==0.1.0
billiard==3.6.1.0
celery==4.3.0
certifi==2019.9.11
chardet==3.0.4
Click==7.0
config==0.4.2
decorator==4.4.0
Flask==1.1.1
idna==2.8
importlib-metadata==0.23
ipython-genutils==0.2.0
itsdangerous==1.1.0
jedi==0.15.1
Jinja2==2.10.3
kombu==4.6.6
MarkupSafe==1.1.1
more-itertools==7.2.0
numpy==1.17.3
pandas==0.24.2
parso==0.5.1
pexpect==4.7.0
pickleshare==0.7.5
prompt-toolkit==2.0.10
ptyprocess==0.6.0
Pygments==2.4.2
python-dateutil==2.8.0
python-dotenv==0.10.3
pytz==2019.3
redis==3.3.11
requests==2.22.0
six==1.12.0
traitlets==4.3.3
urllib3==1.25.6
vine==1.3.0
wcwidth==0.1.7
Werkzeug==0.16.0
zipp==0.6.0

see first column

Python

F1 PedVer

Vishnu Govindaraj

This list is pulled from PedVer’s requirements.txt directly, please review and separate to individual rows, Vishnu Govindaraj , Angel Manica Raquel .

 npm Dependencies

Framework

Version

Notes

Angular:

@angular/animations
@angular/common
@angular/compiler
@angular/core
@angular/forms
@angular/http
@angular/language-service
@angular/platform-browser
@angular/platform-browser-dynamic
@angular/router
primeng

6+

NGRX

@ngrx/effects
@ngrx/router-store
@ngrx/store
@ngrx/store-devtools

6+

Types

@types/core-js
@types/jasmine
@types/node

2.5.2+

3.4.5+

12.12.5+

bootswatch

3.3.7

core-js

2.4.1+

deep-freeze

0.0.1+

es6-promise
es6-shim

3.0.2+

0.35.0+

font-awesome

4.7.0

ng2-bootstrap
ng2-file-upload

1.6.3

1.2.1

rxjs
rxjs-compat

6+

6.5.4

systemjs

0.19.8+

zone.js

0.8.4+

reselect

3.0.1+

reflect-metadata

0.1.8+


 Maven Dependencies

Main Project Pom:

Framework

Version

Notes

junit

4.11

mokito

1.10.19

hamcrest

1.3

powermock

1.6.4

logback

1.1.3

SLF4J

1.7.5

janino

3.0.6

commons-lang

2.6

commons-io

2.4

commons-cli

1.3.1

commons-fileupload

1.3.1

javax.mail

1.4.3

Used in email generation by Digester and Extractor

spring-context

spring-orm

spring-web

spring-webmvc

spring-test

4.3.0.RELEASE

Both same version

spring-security-core
spring-security-config
spring-security-taglibs

spring-security-ldap

spring-security-web
spring-security-crypto

4.2.2.RELEASE

spring-ldap-core

2.3.1.RELEASE

gson

2.6.2

simple-xml

2.7.1

Used creating and reading gobii-web.xml and other xml config files

jackson-databind

2.9.8

jackson-mapper-asl

1.9.13

postgresql.

42.1.4

guava

19.0

Note in pom confused about why this had to be added

liquibase-core

3.3.2

hibernate-core

5.4.10.FINAL

hibernate-jpa-2.1-api

1.0.0.FINAL

javax.ws.rs-api

2.1-m05

jetty-server
jetty-servlet

9.4.5.v20170502

jersey-server
jersey-container-servlet-core
jersey-media-json-jackson

2.26-b03

tomcat-servlet-api

7.0.30

commons-dbcp

1.4

net.sf.opencsv

2.3

org.jboss.logging

3.3.0.FINAL

org.apache.httpcomponents
httpclient
httpclientcache
httpmime
fluent-hc

4.4

httpcore

4.4

docx4j

3.3.6

hibernate-c3p0

5.3.6.FINAL

maven-compiler-plugin

3.5.1

Gobii Process pom

Framework

Version

Notes

spring-context
spring-tx
spring-security-core
spring-security-config
spring-security-taglibs

<inherited>

junit

<inherited>

commons-cli

<inherited>

mockito-all
hamcrest-library
powermock-api-mockito
powermock-module-junit4

<inherited>

guava

<inherited>

lombok

1.18.8

maven-assembly-plugin

2.4.1

Gobii Web pom

Framework

Version

Notes

swagger-maven-plugin

3.1.7

spring-web
spring-webvmc
spring-context
spring-orm
spring-ldap-core
spring-security-core
spring-security-taglibs
spring-security-ldap
spring-security-web
spring-security-crypto
spring-tx

<inherited>

spring-aspects

explicitly declared to be parent’s spring.version

jboss-logging

<inherited>

hibernate-jpa-2.1-api

<inherited>

hibernate-core

5.4.10.Final

spring-test

3.2.3.RELEASE

junit

<inherited>

jttpclient
httpcore
httpclient-cache
httpmine
fluent-hc

<inherited>

liquibase-core

<inherited>

commons-dbcp

<inherited>

tomcat-servlet-api

<inherited>

jackson-databind
jackson-mapper-asl

<inherited>

opencsv

<inherited>

postgresql

<inherited>

commons-fileupload
commons-ui
commons-lang

<inherited>

swagger-jaxrs2
swagger-jaxrs2-servlet-initializer

2.0.0

mockito

<inherited>

json-path-assert

0.8.1

tika-core

1.2.2

jakarta.xml.bind-api

2.3.2

jaxb-runtime

2.3.2

hibernate-c3p0

5.3.6.FINAL

jetty-maven-plugin

9.1.3.v20140225

maven-compiler-plugin

3.2

grunt-maven-plugin

$grunt-version

frontend-maven-plugin

1.8.0

maven-resources-plugin

2.6

swagger-maven-plugin

3.1.8

jaxb.api

2.3.1

REMAINING POM FILES TODO - Similar content

A general overview of the architecture:


Disregard MonetDB as we've dropped support for that.


Legend

  • IFL = Intermediate File Loader
  • H5L = HDF5 Loader
  • MDE = Metadata Extractor
  • H5E = HDF5 Extractor
  • JDBC = Java Database Connectivity
  • API = Application Programming Interface
  • JSON = JavaScript Object Notation
  • Arrows represent data flow


To update:

  1. MonetDB is no longer used and should be removed from this diagram
  2. There is work being done on making the LoaderUI a web app, once that is done, this diagram should be updated accordingly.









  • No labels