Job Status

Objective

Create a job tracking system that replaces email notification and allows users to see status of their jobs and provides them access to job information and files

Requirements

As a GDM User I want a job tracking system that replaces email notification and allows me to see the status of my jobs (for all crops to which I have access) and provides me with job information and files  

• Job types: 

  • Load  

  • Extract 

  • Analysis 

• Job Events: 

  • Marker 

  • Sample 

  • Dataset 

• Job Progression: 

  • Load 

    • Submitted 

    • Transforming

    • Validating

    • Loading Metadata

    • Building Dataset(s)

    • QCing 

    • Completed 

  • Extract 

    • Submitted 

    • Extracting Metadata 

    • Extracting Dataset(s)

    • Formatting Outputs

    • Completed 

• Job Status: 

  • Submitted

  • In Progress 

  • Failed 

  • Succeeded 

• Job Information: 

  • Job Stats 

    • Job Identifier (uuid)

    • Project

    • Experiment

    • Dataset

  • Data

    • # DNAruns

    • # Markers

  • Validation 

    • a table based on the ValidationResult-<xxxxxx>.json file in the digest folder

  • Paths 

    • the path to the folder created for that job’s identifier

    • any file(s) created for this job should be moved into this job folder, including:

      • input files

      • instruction files

      • digest files

      • extract files

      • QC files

      • logs

  • Messages 

    • Human readable messages that reflect any warnings and/or error messages encountered while processing the job.

• The Job Status table (in the UI) should contain the following columns:

  • Crop (?) 

  • Date/Time (of job submission) 

  • Date/Time (of job completion) 

  • Job Identifier 

  • User 

  • Type 

  • Event Status 

• To be removed from the DB for each of the crops: 

  • contact table 

  • Job table 

  • job_id in dataset table 

• New Table(s) needed to contain

  • jobs

  • configurations - links for the different DBs, etc

  • Users/Access - would like to pull this into KeyCloak 

User interaction and design

Open Questions

Out of Scope