Skip to content

Bug report: FSQRT of non-canonical NaN gives valid large floats

Hi there!

fsqrt was known to produce some off-by-one errors, and may return 1 on canonical NaN.

This snippet shows that additionally to these known problems, fsqrt.s can return large valid floating-point numbers instead of NaNs, when the input is a non-canonical NaN. Hence, this is another bug than the reported off-by-one. I do not know at the moment whether the root cause is shared.

  .section ".text.init","ax",@progbits
  .globl _start
  .align 2
_start:

  # Enable the FPU
  li t0, 0x2000
  csrs mstatus, t0
  csrw	fcsr,x0

  la t0, .fdata0
  fld ft0, (t0)

  fsqrt.s ft2, ft0

infinite_loop:
  j infinite_loop

.section ".fdata0","ax",@progbits
  .8byte 0xc5d584907716dd07

Got ft2=0x5b7c271f which corresponds to 7.09747e+16, instead of ft2=0x7fc00000=nan. Tested with Ariane with cvfpu 0.7.0 (109f9e9ed3adff25464db3aa021cb88119b7bf53).

Thanks! Flavien