Skip to content

fix: flatten Effect/ST do blocks (magic-do) to avoid Lua's nesting limit (#46)#105

Merged
Unisay merged 4 commits into
mainfrom
issue-46/magic-do-effect-st
Jun 21, 2026
Merged

fix: flatten Effect/ST do blocks (magic-do) to avoid Lua's nesting limit (#46)#105
Unisay merged 4 commits into
mainfrom
issue-46/magic-do-effect-st

Conversation

@Unisay

@Unisay Unisay commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Fixes #46.

Problem

A long straight-line do block desugars to a chain of bind/discard whose continuations nest lexically. Past ~200 levels Lua's parser rejects the chunk with chunk has too many syntax levels (LUAI_MAXCCALLS), so the file fails to load — before any code runs. The prelude test suites hit this in practice.

Approach

A new IR pass Language.PureScript.Backend.IR.MagicDo recognises bind/discard/pure specialised to the Effect or ST monad — whose runtime value is a nullary thunk — and rewrites the nested chain into a flat statement sequence:

function() local x = m1(); local _ = m2(); ...; return last() end

This mirrors the magic-do pass of the upstream JS backend and purs-backend-es, but is realised as a rewrite into existing Let/Abs constructs rather than a new IR node (which would ripple through every RawExp traversal, including the De Bruijn machinery behind #37/#56).

Key points:

  • Recognition normalises the application head — resolving module-local aliases, projecting fields out of literal dictionaries, and beta-reducing — until the Effect/ST bind instance is exposed. The optimizer inlines discard to discardUnit.discard = bind, so the chain head is rarely a bare Control.Bind.bind; normalisation sees through that.
  • Chunking. A single flat thunk would overflow Lua 5.1's other limit, LUAI_MAXVARS (200 locals/function). The statements are split into nested thunks of ≤150 locals, so both the nesting limit and the local-variable limit stay satisfied.
  • Placement. Runs as the final step of optimizedUberModule: after renameShadowedNames (locals are uniquely named, so moving binders into a Let needs no De Bruijn shifting) and after dead-code elimination (so the local _ = statements introduced for discard are not dropped as dead). Folding it into the single pipeline definition means both the compiler and the golden-test harness pick it up — no divergence.
  • Effect/ST only. Other monads keep their bind calls (their bind is not "run a thunk"). A long straight-line do in Maybe/Either/State/… can still hit the parser limit; the generic, monad-agnostic fallback is tracked in Generic fallback: flatten deeply-nested do-blocks for non-Effect/ST monads #104.

Testing

  • New Golden.LongDoBlock regression — a 300-statement Effect block that previously failed to load now evaluates and prints 1..300.
  • All existing eval goldens are unchanged (eval/golden.txt preserved, only golden.ir/golden.lua regenerated), confirming semantics are preserved — including Maybe (MaybeChain) and tailRecM (TailRecM2Shadow), which are deliberately not flattened.
  • Full suite green: 237 examples, 0 failures (unit + property + golden + eval + luacheck).

Docs

Adds docs/QUIRKS.md (user-facing Lua-target guide) and notes the residual non-Effect/ST limit there.

Unisay added 2 commits June 16, 2026 12:36
…mit (#46)

A long straight-line do block desugars to a chain of bind/discard whose
continuations nest lexically. Past ~200 levels Lua's parser rejects the chunk
("chunk has too many syntax levels", LUAI_MAXCCALLS), so the file fails to load
before any code runs.

Add an IR pass (IR.MagicDo) that recognises bind/discard/pure specialised to the
Effect or ST monad — whose value is a nullary thunk — and rewrites the chain
into a flat statement sequence (local x = m(); ...; return r()). Recognition
normalises the application head (resolving module-local aliases, projecting
fields out of literal dictionaries, beta-reducing) until the Effect/ST bind
instance is exposed, so it sees through the forms the optimizer leaves behind.

The flat statements are chunked into nested thunks of <=150 locals to stay under
Lua 5.1's other limit, LUAI_MAXVARS (200 locals per function).

The pass runs as the final step of optimizedUberModule: after
renameShadowedNames (so locals are uniquely named and no De Bruijn shifting is
needed) and after dead-code elimination (so the statements introduced for
discard are not dropped as dead). Other monads keep their bind calls; the
generic deeply-nested case is tracked in #104.

Adds the Golden.LongDoBlock regression (300-statement Effect block) and
regenerates the Effect/ST goldens. Eval goldens are unchanged, confirming
semantics are preserved.
Add docs/QUIRKS.md for users compiling PureScript to Lua: the Lua 5.1 target
floor, how PureScript values map onto Lua, how to write FFI (foreign-module
shape), the residual long-do-block limit for non-Effect/ST monads (#46/#104),
and stack-safety via MonadRec. Cross-reference it from CLAUDE.md's Known
Pitfalls.
Documents the second class of Lua 5.1 size limit that magic-do's chunking
keeps the output under: ~200 local variables per function (LUAI_MAXVARS)
and ~60 upvalues (LUAI_MAXUPVALUES), separate from the parser-nesting cap.
Both have bitten the compiler before (#19); flattening Effect/ST do blocks
into a chunked statement sequence avoids them.

Also records the FFI export-paren requirement: each exported value must be
wrapped as `key = (<lua expression>)` or the FFI parser rejects it.
@Unisay Unisay self-assigned this Jun 21, 2026
@Unisay Unisay requested a review from Copilot June 21, 2026 19:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a backend IR lowering pass (“magic-do”) that rewrites deeply-nested Effect/ST bind/discard chains into flat Lua statement sequences (with chunking) to avoid Lua 5.1’s parser nesting limit, plus a regression golden and user-facing documentation of Lua-target quirks.

Changes:

  • Introduce Language.PureScript.Backend.IR.MagicDo and run it at the end of optimizedUberModule.
  • Add a new long Effect do-block golden regression plus regenerated goldens reflecting the new lowering.
  • Add docs/QUIRKS.md and link it from CLAUDE.md.

Reviewed changes

Copilot reviewed 29 out of 31 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pslua.cabal Registers the new IR pass module in the library build.
lib/Language/PureScript/Backend/IR/Optimizer.hs Hooks magicDo into the optimizer pipeline after renameShadowedNames.
lib/Language/PureScript/Backend/IR/MagicDo.hs New IR rewrite pass to flatten Effect/ST do-blocks and chunk locals for Lua limits.
docs/QUIRKS.md Adds user-facing documentation of Lua 5.1 constraints and backend-specific gotchas.
CLAUDE.md References the new docs/QUIRKS.md from the “Known Pitfalls” section.
test/ps/golden/Golden/LongDoBlock/Test.purs New PureScript source golden that triggers deep Effect do nesting.
test/ps/output/Golden.LongDoBlock.Test/golden.lua New generated Lua golden showing flattened/chunked sequencing.
test/ps/output/Golden.LongDoBlock.Test/eval/golden.txt New eval golden output (1..300) for the long do-block test.
test/ps/output/Golden.LongDoBlock.Test/eval/.gitignore Ignores eval harness actual.txt for the new golden directory.
test/ps/output/Golden.TailRecM2Shadow.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.TailRecM2Shadow.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.StringCodePoints.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.StringCodePoints.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.ProfunctorDictLens.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.ProfunctorDictLens.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.MaybeChain.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.MaybeChain.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.GenericEqTwoTypes.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.GenericEqTwoTypes.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.DerivedFunctor.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.DerivedFunctor.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.CharLiterals.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.CharLiterals.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.BugListGenericEq.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.BugListGenericEq.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.ArrayPatternMatch.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.ArrayPatternMatch.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.
test/ps/output/Golden.ArrayOfUnits.Test/golden.ir Regenerated IR golden reflecting magic-do lowering behavior.
test/ps/output/Golden.ArrayOfUnits.Test/golden.lua Regenerated Lua golden reflecting magic-do lowering behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Rewrite extractAnn/modifyAnn/extractBinderAnn as \case over their final
argument and normalise the data-type Haddock to fourmolu's canonical style.
@Unisay Unisay merged commit d18c52c into main Jun 21, 2026
2 checks passed
@Unisay Unisay deleted the issue-46/magic-do-effect-st branch June 21, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Long do blocks generate Lua that exceeds the parser nesting limit

2 participants