Skip to content

Tokenizer accepts #m exponent markers s/f/d/l that the parser then rejects #817

Description

@aalpar

Filed from the 2026-07-29 docs/ refresh, while verifying docs/reference/r7rs-differences.md.

The disagreement

The tokenizer accepts the extended exponent markers for #m big floats via readBigNum(isExtendedExponentMarker), and the parser then rejects every one of them except e.

Reproduced against dist/darwin/arm64/wile at 0f9b7751:

#m1.5e2  => 150.0
#m1.5s2  => Error: <eval>:1:9: parse error: invalid big float: #m1.5s2
#m1.5f2  => Error: <eval>:1:9: parse error: invalid big float: #m1.5f2
#m1.5d2  => Error: <eval>:1:9: parse error: invalid big float: #m1.5d2
#m1.5l2  => Error: <eval>:1:9: parse error: invalid big float: #m1.5l2

So the two layers disagree about the accepted grammar. The tokenizer's permissiveness buys nothing today: every token it admits beyond e becomes a parse error one layer up.

Stale documentation

pkg/internal/tokenizer/CLAUDE.local.md still advertises e,s,f,d,l for #m, which describes the tokenizer's behavior rather than the language's.

docs/reference/r7rs-differences.md has been corrected to document the observed (decimal-e-only) behavior rather than the advertised grammar.

Resolution needs a decision

Either:

  • Implement the markers — make the parser honor s/f/d/l as precision selectors, which is the direction the TODO.md reader-fix entry points ("Exponent markers on read and output denote precision"), and which needs checking against R7RS and R6RS first; or
  • Tighten the tokenizer — reject them at the token layer so the error arrives at the right level with a better message, and correct the tokenizer's CLAUDE.local.md.

Related, from the same backlog entry and independently confirmed:

  • #z#x1F and #x#z1F both fail. #z is always decimal (the tokenizer hardcodes tokRadix = 10, and the parser only ever calls parseBigIntegerWithBase(10)).
  • #&5 is a parse error: #& is write syntax only, not read syntax.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsyntaxLanguage syntax extensions

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions