Suggestion for floating- and fixed-point support

[stareyedmoon]

Given that support for floating-point has been proposed before (see #92 and #180), I have a suggestion for a way that they could be implemented without locking users into a specific format.

My suggestion is that a new type be added, which I will call f for now, that matches numbers with a fractional component (10.9, 10., .9, etc). This type would have four built-in functions to act on it: whole(f), fraction(f), exponent(f), and mantissa(f). whole gives the whole portion of the number, fraction gives the fractional part of the number, exponent gives the exponent of the number as if it were represented in binary scientific notation, and mantissa gives the mantissa of the number as it were represented in scientific representation, with the radix point removed.

Example for how this could be used to define the formats for a 32-bit IEEE 754 float (ignoring subnormals) and a fixed 16.16 type:

#ruledef f32 {
    {float:f}_f32 => {
        ; sign 1b | exponent 8b | mantissa 23b
        s = float >= 0 ? 0 : 1
        e = exponent(float) + 127 ; bias
        m = mantissa(float)

        assert(exponent > 0) ; Filter out subnormals
        assert(exponent < 255) ; Filter out values to large to represent
    
        ; The assumptions here are that negative indicies exist, and that when using them the value would be padded on the right.
        s @ e`8 @ m[-2:-24] ; Top-most mantissa bit is implicit
    }
    inf => 0x7F800000`32
}

#ruledef fx16_16 {
    {fixed:f}_fx16_16 => {
        assert(fixed >= -32768)
        assert(fixed < 32768)

        whole(fixed)`16 @ fraction(fixed)[-1:-16]
    }
}

    10.125_f32      ; s = 0
                    ; e = 130
                    ; m = 0b101000100000000000000000...
                    ; s @ e`8 @ m[-2:-24]
                    ; 0b0'10000010'01000100000000000000000
                    ; 0x41220000

    1.21_fx16_16    ; whole(fixed) = 1
                    ; fraction(fixed) = 00110101110000101001000....
                    ; whole(fixed)`16 @ fraction(fixed)[-1:-16]
                    ; 0b0000000000000001'0011010111000010
                    ; 0x0001'35C2

I hope what I'm trying to suggest is actually possible to understand from what I've written.

[hlorenzi]

I think this is a very clean and flexible solution for letting the user define any imaginable encoding!

How would the assembler encode this f type internally, though? fraction and mantissa apparently have to return an infinite-precision value, that the user can slice as they please, but how would you represent something like 0.1 or 0.3 internally to allow this process to happen? Do we just give it, say, 128 bits of precision at most and call it a day?

Another idea is that we could pass the precision required as an argument to the functions, as in mantissa(float, 23), and the assembler could expand the decimal representation just-in-time. This seems to be the most correct implementation, since there's an annoying problem with the rounding error of the least-significant bit that depends on knowing the total precision beforehand.

Would we also need to support hex floating-point literals, like 0x1.4p3?

[stareyedmoon]

I think your idea of specifying the precision makes the most sense. All the other options I can think of are more complicated to implement for no real benefit.

Does anyone actually use hex float literals? In any case, they don't seem like they would be terribly difficult to implement.