BERT - Back-End Regression Testing

Objective

Develop a framework that will allow QA to contribute test scenarios to a repository that can be run as regression tests for any GDM instance.

User Story

As a member of the development team, I need an automated back end regression testing framework so that test scenarios can be created and added and run to ensure the health of GDM code

• Load:
  • create the entities needed
    • for samples, need to create
      • PI
      • project
      • experiment
      • platform
      • protocol
      • vendor
      • vendor-protocol
    • for markers, need to create
      • platform
      • mapset
      • marker group(s), if needed in subsequent extract
    • for datasets, need to create analyses
      • calling, and any other(s)
  • files to be provided for any load scenario
    • input file
    • json file
• Extract
  • appropriate loads need to happen first 
  • files to be provided for extract scenario(s)
  • for Extract by Sample -
    • list of entities in .txt file format (germplasm names, external codes, or dnasample names)
  • for Extract by Marker - 
    • list of marker names in .txt file format, or
    • name of marker group
      • marker group will need to be created first
• type of test - positive or negative
  • and whether comparisons are needed at the end
• specify location
  • input files
  • output files if comparison needs to be done
• No emails will be sent
• view results to see which tests have passed and which tests have failed
  • for failed tests, need
    • access to logs
    • access to all artifacts
• if a comparison is needed

  • Status: Failed/ Passed

    • Success Report

      • # of records

      • # of columns

      • Execution time

  • Failed Report:

    • Status: Failed/Passed

      • for each failure:

        • failure type

        • Record #

        • column #

  • Failure Types:

    • Data Validation failure

    • Genotype mismatch

• Once a test has completed
  • All entities created by the test need to be removed from the DB

Success metrics

Assumptions

Stories

User interaction and design

Open Questions

Out of Scope