Transducers

A transducer is a transformation decoupled from its input and output. The same f‌i‌l‌t‌e‌rclojure.core/filterLazy seq of items where (pred x) is truthy.view on clojuredocs → or m‌a‌pclojure.core/mapApply f to each element, returning a lazy seq.view on clojuredocs → logic that you'd normally apply to a vector can be fused into a single pass over a stream, a channel, or any sink — without allocating intermediate collections. Transducers are how Clojure makes "map then filter then take" fast at scale — the hot-path counterpart to the -‌>‌>clojure.core/->>Thread the value through forms by inserting it as the last arg.view on clojuredocs → pipelines most services start with, and a concrete instance of the fold patterns in recursion schemes.

The shape of the problem

The familiar -‌>‌>clojure.core/->>Thread the value through forms by inserting it as the last arg.view on clojuredocs → pipeline is clear, but each step builds a new lazy seq:

For 1,000 items it doesn't matter. For 1,000,000, or for streaming through a channel, each intermediate sequence costs memory and indirection.

Minimal example: build a transducer

When you call m‌a‌pclojure.core/mapApply f to each element, returning a lazy seq.view on clojuredocs →, f‌i‌l‌t‌e‌rclojure.core/filterLazy seq of items where (pred x) is truthy.view on clojuredocs →, etc. without a collection, they return a transducer — a recipe — instead of doing the work:

c‌o‌m‌pclojure.core/compRight-to-left function composition: (comp f g) = #(f (g %)).view on clojuredocs → here reads top-to-bottom: increment, then filter, then take 5. (It's the opposite of c‌o‌m‌pclojure.core/compRight-to-left function composition: (comp f g) = #(f (g %)).view on clojuredocs → on plain functions because transducers compose into a nested transformation chain.)

Now apply the same xf to different "contexts":

One recipe, three sinks. No intermediate collections were allocated.

Why this matters

• No intermediate seqs. (map f (filter pred xs)) builds a seq for the filter, then another for the map. A transducer fuses them into one pass.
• Sink-agnostic. The same xf runs over collections, lazy seqs, channels, reducible sources, even Java streams via interop. You write the pipeline once and reuse it.
• Early termination is real. t‌a‌k‌eclojure.core/takeLazy seq of the first n items.view on clojuredocs →, t‌a‌k‌e‌-‌w‌h‌i‌l‌eclojure.core/take-whileLazy seq of items while (pred x) is truthy.view on clojuredocs →, etc. signal "done" and the rest of the pipeline stops — important for both performance and infinite sources.

Stateful transducers

Some transducers carry state across items. dedupe only emits values different from the previous one, partition-all buffers groups, etc. They still work fine in a pipeline, but the state is per-application — it's safe to reuse the recipe.

Writing one by hand

A transducer is a function xf that takes a "reducing function" and returns another reducing function. You usually don't write them by hand, but seeing the shape helps demystify the API:

The three arities are init / completion / step — almost all transducers follow that shape.

Check yourself

Exercise

Build a transducer xf that, over (range 100), keeps only multiples of 3, squares them, and stops after the first 4. Apply it with into [] and with transduce + 0 to see both shapes.