Test List Search and Test History

Simscope tracks Test History, which can be used to aggregate statistics over:

• Component
• Branch
• Date Range
• Regressions

Example Use Cases

1. Show me the summary of all tests run in the past 7 days for the component fpu.

2. Show me the CPS for all tests in the security block, and compare the normal build to the cov (coverage) build.

3. Show me all failing simulations in the security block containing a plusarg +random_bit.

Filters

Test stats can be filtered by regular expressions on:

• Build Target
• Error Message
• Test Config
• Test Group

Filtering by Job Metadata

You can also filter results by job metadata (via regular expression matching or numeric comparisons).

This screenshot shows viewing all the results from a single regression, and then filtering by:

• Fails only
• cycles > 25000

Statistics

Test History calculates the following statistics, based on the matching jobs:

• Tests — total number of jobs run
• Signatures — number of unique signatures
• Build Targets — number of unique build targets
• Score — pass/fail rate
• Cycles — total job cycles
• Runtime — total job runtime
• CPS — cycles per second.
  • Note this is calculated for Pass jobs only, to give a more accurate value.

Regression Statistics

Simscope can calculate Regression statistics on the Test History page.

It can also aggregate multiple regressions' statistics.