Skip to content

Diffing isacov log vs tracer log

Created by: silabs-robin

This change is for checking that ISA coverage samples correctly (try to detect false positives/negatives), and includes some corresponding fixes to the sampling/decoding itself. It changes formatting in isacov's logging to be compatible with 40x's tracer log [1]. It also adds to get_instr_type() and convert2string() to handle those cases which were missing. Essentially, since incoming binary words are decoded into uvm objects (which coverage samples) and these objects are printed to a log, they can be compared against a reference log (if the logs match, then the uvm objects should have correct data in them).

This diffing of logs actually did uncover some instructions being sampled wrong. For example, branch instructions had the offset's sign bit wrong (due to Spike internals) and both B-type and U-type needed shifting of the result obtained from dpi_dasm/Spike disassembler, "M" extension weren't logged properly, etc. Coverage for those items were then false before, until now being found and fixed. NB! RVC (compressed) has not yet been tested thoroughly because we wait for RVFI to deliver actual compressed instructions to isacov (diffing the logs will fail on compressed instrs unless filtered [2] away).

By default, the log diffing is not enabled. Since it needs to be done as post-processing, I added the line $(POST_TEST) to uvmt/xrun.mk test: target. It allows e.g. make test TEST=hello-world POST_TEST='make isacov_logdiff' or it can be used with ci_check, and can potentially be used for any other kind of desired post-processing.

[1] The tracer log format is not standardized. I think it should have tab-separated columns, comma-separated within-column data, and allow for leading/trailing spaces per cell. And it should maybe have defined what gets printed as hex/decimal/string etc. I also don't like how non-general the filtering of the logs and file names must be because of this.

[2] One can use 'sed -i "s/c.//" $(ISACOV_TRACELOG) && make isacov_logdiff' to filter compressed instructions before diffing the logs.

Merge request reports

Loading