Regressions

Regressions are groups of Jobs, launched within a single context.

Regressions can contain any type of Job Categorized with a pass/fail result:

• Compilations (Builds)
• Test generation
• Simulations (tests)
• And can contain either passing or failing jobs.

Regressions are generally launched automatically with an external Regression tool, and imported into Simscope for analysis.

Regression Context

Within a single regression, all jobs must share the same Context:

Regression Example

Regression compress_random/412 is run with component compress, on the branch testing, with Git SHA f81249a, and contains:

• 1200 jobs
  • 2 pass compiles
  • 901 pass simulations
  • 297 fail simulations

Categorization

Simscope categorizes results by Signature, which allows filtering and statistic tracking based on job types.

• An example categorization grouping is simulations vs test generations.

Regressions are allowed to have multiple categories within a single regression.

The following example includes compile, gen, and sim Jobs and Scores:

Statistics

Simscope collects aggregate statistics on each regression, which can be used to determine the health of the regression:

• Score (pass/fail rate)
• Total Cycles
• Total Runtime
• Average CPS (cycles per second)
• Duration (wallclock start-to-finish)
• Number of signatures

Live Regressions

Simscope can track both completed regressions, and LIVE (incomplete) regressions.

→ See Live Regressions on the Administration Guide

Compare/Diff Regressions (Comparisons)

→ Simscope can Compare two or more Regressions.

Tanking Regressions

Managers and Administrators can Tank bad Regressions.

• This is equivalent to killing or hiding a regression.