Skip to main content

collect, picked apart

collect is filter and map in one pattern match (a Scala match: a switch whose cases can test, destructure and produce a value). The first eager implementation here kept the PartialFunction object and measurably lost. What ships instead picks the pattern match apart at compile time, the same move the fusion engine makes, done in the eager machinery.

The tax, and where it isn't​

A PartialFunction[A, B] (Scala's reification of a pattern match: a lambda paired with an isDefinedAt test) is a generic runtime object, and a collect loop consults it per element — fs2 and zio via one applyOrElse with a sentinel default, the standard library via isDefinedAt + apply. Generic means erased: on Int every question and answer crosses through Integer. And it isn't about arrays — our first cut had unboxed storage on both sides of the PF boundary and still ran at 0.65× of the best rival at 100k Int. The boundary itself was the whole cost.

The call site, though, contains a pattern-match literal, and a literal is compile-time information:

xs.collect { case x if x % 2 == 0 => x * 2 }

collect is an inline op, so a small macro (code that runs in the compiler, rewriting the typed tree) sees that literal and splits it: pattern and guard become a predicate x => x % 2 == 0, the case body a transform x => x * 2, and both feed a one-pass impl exactly as filter's and map's lambdas do — inline parameters spliced into the per-element loop, unboxed end to end, output written filter-style into one slack-wrapped max-size array (no growable buffer, no trim copy). No PartialFunction is ever constructed; for a single simple case there isn't even a match in the output — the guard is the branch, evaluated once per element.

The fine print:

  • Destructuring and multi-case matches split into two mirrored matches (one answering yes/no, one producing the value), with the pattern binders hygienically freshened. Patterns and guards may then evaluate more than once per element (the usual purity assumption).

  • A stored PF (val pf: PartialFunction[…] passed by name) can't be picked apart; it falls back to the runtime path unchanged.

  • Every element kind is carried: the split lambdas ride the same per-kind dispatch as filter/map, so Float/Byte/Char collections get the same treatment as Int and refs.

measuring…

Eight pattern matches in one method​

This is the method-size effect again, in collect's clothing. A lone collect is the friendliest case a rival sees — one call site, one PartialFunction class, fully inlined — but pack eight distinct pattern matches into one method (a parser, a router, any dispatch-heavy code) and every rival funnels eight PF classes through its one shared collect loop. Collect's twist over map: the shared site is a PartialFunction.applyOrElse, which is generic, so once it goes megamorphic it not only loses inlining but re-boxes every primitive — where map's specialized lambda did not. FArray's collect has no shared site to poison; the macro gave each call site its own loop at compile time:

measuring…

The same idea one level up: in a fused chain, collect's cases splice straight into the pipeline loop (stages page); this page's macro is that move, packaged for the eager op so a lone collect gets it too.