Analytics Framework

Owned by Yaw Nti-Addae
Last updated: Mar 30, 2020 by Monica Franciscus (Unlicensed)
2 min read

Drivers

Approvers

Associated Projects

Stakeholders

Drivers

Approvers

Associated Projects

Stakeholders

@Yaw Nti-Addae

Victor Ulat

Pedro MEDEIROS BARBOSA

Kelly Robbins

Tom Hagen

Stephan Einarson

Moira Sheehan

Lukas Muller

EiB

  • Liz Jones

  • Michael Quinn

EBS - Technical Oversight Committee

Breeding Insight

Innovation Lab

Objective

Develop an analytics framework that will allow breeders to analyze and summarize their phenotypic and genotypic data

Due date

Mar 1, 2021

Key outcomes

  • Agreement on project timelines, resources and scope.

  • Open-source application with emphasis on EBS integration?

Status

NOT STARTED

Scope of Phase 1

  • User interface through which an analysis request can be submitted

  • Analysis pipeline for processing data

  • Job tracking

  • Analysis and Models:

    • Phenotypic analysis

      • ANOVA

      • Simple mixed model (without genotypic information)

      • D-Stat model

    • Experiment design simulator

  • Integration with breeding management system

 

  • User input file processing (Data will be acquired through web services)

  • file output of summary reports (Export features will be provided in UI)

  • Genotypic analysis (will be provided in the future)

  • Covariance matrices (GRM) (will be provided in the future)

  • Integration with genotype management system (will be provided in the future)

Resources for Phase 1

Skill set

% FTE

Duration (months)

Components

Skill set

% FTE

Duration (months)

Components

Data modeler

10

6

  • Database

  • Liquibase

General developer

100

12

  • Authentication

  • Data process layer

Full stack developer

100

12

  • Web services

  • User interface

DevOps

5

12

  • Automated deployment scripts

  • Automated testing scripts

  • Integration to EBS

Statistician

100

12

  • modeling

  • requirement specification

  • UAT

QA

20

6

  • Write and build test scenarios

  • Testing

  • UAT

Timeline for Phase 1

Apr2020MayJunJulAugSepOctNovDecJan2021FebMarAprReview 1Review 2Review 3Review 4UATReview 5Release
Authentication
Database
Process
Stat Model
API
UI
DevOps

IAM

Federation

Roles

User config

Jobs DM

Report DM

Analysis DM

Model DM

Pipeline

RabbitMQ

Job submission

ASReml prep

R prep

Calc Engine

Report Engine

UAT

ASReml models

Test models

Test models

R - Exp Design

R D-Stat Model

Analysis statistics

Report summaries

API Framework

User config

Analysis

Models

Request analysis

Job status

Report

Angular setup

Authentication

User config

Phenotypic analysis view

Pheno data

Request form

Job status

Report view

UAT

IAM & DM

API & UI

Dev deploy

Automated test framework

Test deploy

Prod deploy

Milestones and deadlines for Phase 1

Milestone

Deliverables

Deadline

Status

Milestone

Deliverables

Deadline

Status

Review 1

  • Install and configure IAM, including connections to Cornell University OAuth, Google authenticator, and Active Directory

  • Develop and deploy data models for user configuration and job tracking

May 1, 2020

NOT STARTED

Review 2

  • Role management

  • Analysis and Model data models

  • Message queuing

  • Experiment design statistics

  • Anova model statistics

  • Simple model statistics

Jul 31, 2020

not started

Review 3

  • UI - Analysis request form

  • Report data model

Oct 2, 2020

not started

Review 4

  • Report view

  • Job status

Dec 4, 2020

not started

UAT

  • User acceptance testing

Jan 2, 2021

not started

Review 5

  • Project review and lessons learned

Jan 29, 2021

not started

v1 Release

  • Production release and integration with EBS

Feb 26, 2021

not started

Technologies

Layer

Technologies

Notes

Layer

Technologies

Notes

Code repository

  • Github

Makes it easy for all parties to pull code into their repos

Authentication

  • WSO2 IAM and/or Keycloak IAM

WSO2 does not officially support openJDK

Database

  • PostgreSQL

 

Data processing

  • RabbitMQ

  • Java and/or Python

 

Statistics

  • ASReml 4

  • R

R has got strict versioning, which means that packages/libraries are not guaranteed to be forward or backward compatible

API

  • BrAPI standards

  • WSO2 API manager

  • Java and/or Python

We will use Swagger for API management for easy integration with SWO2 API manager

User interface

  • Angular or Vue or React

Browser compatibility for

  • MS Edge

  • Chrome

  • Firefox

  • Safari

DevOps

  • Ansible

 

Testing

  • Selinium

  • Unit test

Test browser compatibility for MS Edge, Chrome, Firefox and Safari, which covers ~85% of browser usage in the industry.

Reference materials