Debug halt causes illegal instructions

Created by: gmarkall

Summary

We are working on a verilator model of RI5CY and are attempting to understand how to use the debug unit. Normal execution seems to work correctly as far as we can observe. Whenever we attempt to halt the processor by writing DBG_CTRL_HALT to DBG_CTRL, an illegal instruction exception seems to be generated.

Details

Our test bench does the following (the test bench file is at https://github.com/embecosm/ri5cy/blob/debug_halt/verilator-model/testbench.cpp), which is inspired by looking at tb.svh and spi_debug_test.svh in the PULPino repository:

1. Sets irq_i, debug_req_i, fetch_enable_i and rstn_i to 0.
2. Runs for 100 cycles (for the reset, probably this is more than needed)
3. Set rstn_i to 1 and fetch_enable_i to 1.
4. Runs for 20 clock cycles, to allow the CPU to do some "normal" execution. This appears to work without problems.
5. Writes to the debug unit DBG_CTRL = DBG_CTRL | DBG_CTRL_HALT
6. Writes DBG_IE = 0xF
7. Waits for debug stall by reading DBG_CTRL until we observe that DBG_CTRL_HALT is set.
8. Sets single step by writing DBG_CTRL = DBG_CTRL_HALT | DBG_CTRL_SSTE.

At this point we think we should be ready to single-step. For each single step, we:

1. Clear DBG_HIT (I didn't see this in the test bench, but it's what the debug_bridge GDB server appeared to do for single steps).
2. Writes DBG_CTRL = DBG_CTRL_SSTE.
3. Waits for debug stall (as in step 7 above).

The first one or two single steps seem to work OK (depending on how many cycles we run for and how much we try and interact with the debug unit) but it seems that after about 8 cycles, an illegal instruction exception occurs.

Observations

A colleague of mine has suggested that it looks like something goes awry in the prefetch fifo - when executing normally, core/if_stage/prefetch_32/prefetch/fifo valid_Q[0] is never set, so we bypass the input of the fifo to the output without ever caching it in the fifo itself. In the debug case the core halts and the fifo fills up. The fifo then wipes one of the entries to 0, later it passes that to the decoder and it gives the undefined exception.

Reproducing

If it is helpful to try to reproduce the issue, then this can be done by:

1. Cloning our RI5CY repo: https://github.com/embecosm/ri5cy
2. Checking out the debug_halt branch: https://github.com/embecosm/ri5cy/tree/debug_halt
3. In the verilator-model subdirectory, run make. Note that this requires the latest version of verilator installed (3.906), and for pkg-config to be able to find verilator.
4. Execute ./testbench.

The output that it produces is reproduced here:

$ ./testbench 
DBG_CTRL  10001
DBG_HIT   0
DBG_CAUSE 1f
DBG_NPC   cc
DBG_PPC   c8
DBG_CTRL  10001
DBG_HIT   1
DBG_CAUSE 0
DBG_NPC   d0
DBG_PPC   cc
                1430: Entering exception routine.
                1430: Illegal instruction (core 0) at PC 0x000000d0:
DBG_CTRL  10001
DBG_HIT   1
DBG_CAUSE 0
DBG_NPC   84
DBG_PPC   d0
DBG_CTRL  10001
DBG_HIT   1
DBG_CAUSE 2
DBG_NPC   88
DBG_PPC   84
                1630: Entering exception routine.
                1630: Illegal instruction (core 0) at PC 0x00000088:
DBG_CTRL  10001
DBG_HIT   1
DBG_CAUSE 2
DBG_NPC   84
DBG_PPC   88

The output is produced by the stepSingle function - we read the registers each time we try to single step, to try and get some visibility into what is happening.

Questions

Looking at past issues it seems that other people are successfully making use of the debug unit, so I guess there might be something wrong with what we'e doing or how we're setting things up.

• Are we doing something wrong in the setup or halting of the CPU?
• If not, is there anything else we can look at to try and understand what is going on?
• Any other thoughts?

Any assistance is greatly appreciated - many thanks in advance!