Verilator Performance Issue

Created by: davidharrishmc

Verilator runs regression tests, but is about 50x slower than Questa right now. We expect comparable speeds, so there is an issue with something in the cvw coding.

From the sim directory, run verilator on rv64i with

verilator -GTEST="\"arch64i\"" --timescale "1ns/1ns" --timing --binary --top-module testbench "-I../config/shared" "-I../config/rv64gc" ../src/cvw.sv ../testbench/testbench.sv ../testbench/common/*.sv ../src/*/*.sv ../src/*/*/*.sv --relative-includes
time ./obj_dir/Vtestbench
TEST is arch64i
rv64i_m/I/src/add-01.S succeeded.  Brilliant!!!
...
rv64i_m/I/src/xori-01.S succeeded.  Brilliant!!!
SUCCESS! All tests ran without failures.
%Error: ../testbench/testbench.sv:360: Verilog $stop
Aborting...
Aborted (core dumped)

real    8m38.408s
user    8m38.230s
sys     0m0.129s

To run verilator with profiling: Change $stop to $finish in Verilog

verilator -GTEST="\"arch64i\"" --prof-cfuncs --timescale "1ns/1ns" --timing --binary --top-module testbench "-I../config/shared" "-I../config/rv64gc" ../src/cvw.sv ../testbench/testbench.sv ../testbench/common/*.sv ../src/*/*.sv ../src/*/*/*.sv --relative-includes
time ./obj_dir/Vtestbench
...
rv64i_m/I/src/xori-01.S succeeded.  Brilliant!!!
SUCCESS! All tests ran without failures.
%Error: ../testbench/testbench.sv:360: Verilog $stop
Aborting...
Aborted (core dumped)

real    22m17.142s
user    22m16.959s
sys     0m0.116s

gprof ./obj_dir/Vtestbench gmon.out > gmon.log
verilator_profcfunc gmon.log > gmon.log2
more gmon.log2

Overall summary by type:
  % time  type
    9.01  C++
   20.76  Common code under Vtestbench
    0.14  VLib
  590.85  Verilog Blocks under Vtestbench
  -520.76  Unaccounted for/rounding error
Overall summary by design:
  % time  design
    9.01  C++
    0.14  VLib
  611.61  Vtestbench
  -520.76  Unaccounted for/rounding error

590% is clearly a profiling error. The most time that could be used is 100%.

Verilator on chips.eng.hmc.edu has been built with clang (./configure CXX=clang++) and gprof has been rebuilt.

May need to change $stop to $finish in the testbench to generate gmon.out. Try removing the #delays or using --no-timing

See also the issue #4858 filed with Verilator: https://github.com/verilator/verilator/issues/4858

There’s some documentation under Code and Execution Profiling at https://verilator.org/guide/latest/simulating.html#execution-profiling

It is also possible but slow to run verilator on all configurations with ./verilate