RxJS in Node.js: Processing Event Streams Reactively
RxJS in Node.js: how fromEvent turns an EventEmitter into an observable, how operators transform server-side streams declaratively and why RxJS has no real backpressure.
let buffer = [];
let timer = null;
emitter.on('data', line => {
const entry = JSON.parse(line);
if (entry.level !== 'error') return;
buffer.push(entry);
if (!timer) {
timer = setTimeout(() => {
saveErrors(buffer);
buffer = [];
timer = null;
}, 1000);
}
});
There are three requirements in this handler: parse, filter, write out in bundles once per second. The intermediate state lives in two module variables, and the only way left to test it is with fake timers. If a second consumer comes along tomorrow, or you also need to group by service, the nesting keeps growing – and with it the number of places where the shared state can tip over.
This is exactly where RxJS comes in: wherever an event stream should become a composable program. If you’ve already looked at the imperative foundation – for example in the post on EventEmitter and the event model – you’ll recognize the same idea in RxJS, just approached from the other side.
From on/emit to declarative streams#
The EventEmitter is imperative. I say: “When data happens, run this function.” That is a command. Each handler is its own little island, and if I want to chain several steps, I connect them by hand through shared state.
RxJS describes the same event stream declaratively. I no longer say “do this when that happens” but “this stream is the source, transformed by this chain of steps.” The chain is a description, not an execution. It only gets activated on subscribe.
The bridge between the two worlds is fromEvent. The operator takes a Node EventEmitter and an event name and returns an observable:
const { EventEmitter } = require('events');
const { fromEvent } = require('rxjs');
const emitter = new EventEmitter();
const data$ = fromEvent(emitter, 'data');
const subscription = data$.subscribe({
next: value => console.log('received', value),
});
emitter.emit('data', { level: 'info', message: 'server started' });
fromEvent(emitter, 'data') doesn’t change anything about the emitter’s semantics. On every subscribe it registers a listener via addListener and removes it again on unsubscribe. That makes the observable an adapter: the emitter stays the source, and RxJS just puts a composable layer on top. If you want to dig deeper into the difference between the push model of observables and the pull model of iterators, you’ll find it in the post on RxJS: push and pull.
A note on convention: I append a $ to variables that hold an observable (data$). It isn’t required, but it has become common in the RxJS world and makes it immediately visible in the code what is a stream and what is a single value.
The pipeline: operators as pure functions#
The real gain lies in the operators. They are pure functions that take an observable and return a new one. You chain them with .pipe(...). In RxJS 7, I import everything – creation functions like fromEvent and operators like map – directly from 'rxjs'. The old import from 'rxjs/operators' still works but is outdated.
A typical server-side example is log processing. I want to parse incoming raw lines into objects, keep only the errors and write them out in bundles once per second, instead of touching the database for every single line:
const { fromEvent } = require('rxjs');
const { map, filter, bufferTime } = require('rxjs');
const logStream$ = fromEvent(logEmitter, 'line').pipe(
map(line => JSON.parse(line)),
filter(entry => entry.level === 'error'),
bufferTime(1000),
);
const subscription = logStream$.subscribe({
next: batch => saveErrors(batch),
error: err => console.error('stream failed', err),
});
What’s written here reads almost like a description of the requirement. map transforms every value, filter lets only matching ones through, bufferTime(1000) collects over a one-second time window and then emits an array. So the next handler doesn’t receive individual entries but one bundle at a time – or an empty array if nothing came in during the window.
Each step is understandable on its own and testable on its own. And if tomorrow the requirement comes to also group by service, I insert an operator instead of restructuring a nested callback structure.
A second pattern from practice: throttling sensor data while computing a running average. Suppose a device fires much more often than I want to process:
const { fromEvent } = require('rxjs');
const { throttleTime, scan } = require('rxjs');
const average$ = fromEvent(sensor, 'reading').pipe(
map(event => event.value),
throttleTime(500),
scan((sum, value, index) => (sum * index + value) / (index + 1), 0),
);
average$.subscribe({ next: avg => console.log('running average', avg) });
throttleTime(500) lets one value through and then ignores all others for 500 milliseconds. scan behaves like reduce but emits every intermediate result – so with every measurement that gets through, I get the updated average. This combination of running aggregation and time window is sketched in the repository under operators.js, although there it still uses the older RxJS 6 import style.
If I want to trigger asynchronous follow-up processing for each event – such as loading details – mergeMap helps. It maps every value to an inner observable and flattens the results concurrently. The second parameter limits how many inner operations may run at the same time:
const { mergeMap } = require('rxjs');
const details$ = ids$.pipe(
mergeMap(id => fetchDetails(id), 4),
);
With mergeMap(id => fetchDetails(id), 4), at most four requests run in parallel. This limit matters, for a reason that will play its own role in a moment.
The rough flow through such a pipeline always looks the same:
flowchart LR E[EventEmitter] -->|emit data| F["fromEvent<br/>(Observable)"] F --> M[map] M --> FI[filter] FI --> B[bufferTime] B --> S[Subscriber]
A Subject: turning one stream into many consumers#
So far, every stream had exactly one consumer. As soon as several parts of the application need the same event stream, the Subject comes into play. A Subject is both an Observer and an Observable: I can push values into it, and others can subscribe to it. It distributes every value to all subscribers at the same time – multicast.
const { fromEvent, Subject } = require('rxjs');
const bus = new Subject();
// one source feeds the bus
fromEvent(emitter, 'data').subscribe(bus);
// multiple consumers share the same stream
const loggerSub = bus.subscribe({ next: v => logger.write(v) });
const metricsSub = bus.subscribe({ next: v => metrics.record(v) });
// clean up later
loggerSub.unsubscribe();
metricsSub.unsubscribe();
The difference from subscribing directly to the source observable is fundamental. If logger and metrics both subscribed to fromEvent(emitter, 'data'), the emitter would have two listeners and the source would run twice. With the Subject, there is one execution whose values are distributed to all subscribers.
That is the core of the distinction between unicast and multicast:
flowchart LR P[Producer] --> SUB[Subject] SUB --> C1[Logger subscriber] SUB --> C2[Metrics subscriber] SUB --> C3[Audit subscriber]
Besides the plain Subject, there are variants that fit different use cases:
BehaviorSubject(seed)holds the last value and delivers it to every new subscriber immediately. Handy for state that a late consumer needs to know.ReplaySubject(bufferSize)buffers the last n values and replays them to new subscribers.AsyncSubjectemits only the very last value, and only oncomplete.
In the repository under subject.js, you can see the basic principle: the same source, several subscribe calls. It still uses the positional subscribe(next, error, complete) form there. In RxJS 7, I use the object observer subscribe({ next, error, complete }) throughout instead – more on that in a moment under pitfalls.
Hot and cold: the difference that skews results#
One point that reliably causes confusion in teams is the question of when an observable’s source comes into existence.
A cold observable creates its producer per subscribe. Every subscriber gets its own fresh execution – unicast. A from([1, 2, 3]) is cold: everyone who subscribes sees the values 1, 2, 3 from the start.
A hot observable has its producer outside. The values are produced whether or not anyone is listening. If you subscribe late, you miss what happened before – multicast. A Subject is hot, and a fromEvent over an emitter that’s already running behaves hot as well: the events fire anyway, and I just hook in.
This distinction is not theory. If you expect that “every subscriber starts fresh” but are actually looking at a hot stream, late consumers simply get wrong results – missing initial values, an aggregation that starts in the middle of the stream. If I want a late subscriber to also see the latest state or the history, that is exactly the case for BehaviorSubject, ReplaySubject or the shareReplay operator.
The important distinction: RxJS has no real backpressure#
This point should be cleared up early; otherwise expectations arise that the tool doesn’t meet. If you know Node streams, you know backpressure: a readable stream can pause, a highWaterMark limits the buffer, pipe regulates the flow between producer and consumer automatically. The fast producer gets slowed down when the slow consumer can’t keep up.
RxJS doesn’t have this. There is no built-in mechanism for a subscriber to force the source to slow down. A Subject or a fromEvent pushes values as soon as they’re there, whether or not processing keeps up.
What RxJS offers instead are time-based and buffering operators:
throttleTime(ms)lets one value through and then drops all others for a while.debounceTime(ms)waits for a quiet period and then emits the last value.sampleTime(ms)takes the current value at fixed intervals.bufferTime(ms)collects over a window and delivers bundles.- the
concurrentlimit ofmergeMapcaps the number of parallel inner operations.
The key difference: these operators drop or aggregate values, but they don’t slow down the source. The producer keeps firing unchanged; I only reduce what arrives at the end. For a log display or sensor data, that’s often exactly right – dropping a few intermediate values is harmless. For processing where every single value must count and the source really needs to be throttled, Node streams are the right tool, not RxJS.
You can connect both worlds. The repository shows the bridge via rxjs-stream under node.js:
const { spawn } = require('child_process');
const { streamToRx } = require('rxjs-stream');
const { map } = require('rxjs');
streamToRx(spawn('ls', ['-lha']).stdout)
.pipe(map(chunk => chunk.toString()))
.subscribe({ next: text => console.log(text) });
streamToRx turns a Node readable into an observable while keeping the stream’s backpressure in the lower part. That way, I get the composability of RxJS for the transformation without giving up the stream’s flow control. If you want to refresh the basics of the event loop and the stream architecture behind it, you’ll find them in the Node.js training.
Pitfalls I run into again and again#
Two mistakes show up so regularly that I’ll name them explicitly here.
The first is the subscription leak. fromEvent registers a listener on the emitter with every subscribe. Without unsubscribe(), this listener stays attached – even when the processing is long no longer needed. The result is a memory leak and a pipeline that keeps running unnoticed. The rule is simple: I keep the subscription that .subscribe() returns and call unsubscribe() when I’m done. Alternatively, I limit the stream declaratively with take(n) or takeUntil(stop$) so that it ends itself:
const { fromEvent, timer } = require('rxjs');
const { takeUntil } = require('rxjs');
const stop$ = timer(60000);
fromEvent(emitter, 'data')
.pipe(takeUntil(stop$))
.subscribe({ next: v => process(v) });
Here takeUntil cleans up after 60 seconds without me having to manage the subscription myself. On long-running servers, one of these two strategies is mandatory; otherwise listeners pile up over time.
The second pitfall is mixing up hot and cold, which I described above – it almost always shows up as “my second subscriber doesn’t get the same values.” The answer is usually a Subject of the right variant or shareReplay.
And a third, more craft-related point: many examples on the web, and the older snippets in the repository too, still use RxJS 6 patterns. Concretely, for RxJS 7 that means:
- Instead of
subscribe(next, error, complete)with separate arguments, use the object observersubscribe({ next, error, complete })– the positional form has been deprecated since 6.4. - Instead of
tap(fn, errFn)with several callbacks, also use the object observer. - Import operators from
'rxjs'instead of'rxjs/operators'.
These changes are mechanical but important, so that the code doesn’t build on a pattern that is going away.
When RxJS instead of a raw EventEmitter?#
Not every Node application needs RxJS. A single on('data', ...) handler stays leaner with a plain EventEmitter and has no extra dependency. I reach for RxJS when at least one of these points applies:
- I have to chain several processing steps – transform, filter, aggregate – and don’t want to solve that with nested callbacks and shared state.
- I need time-based control: throttling, debouncing, bundling in windows.
- Several consumers should process the same event stream independently (multicast via a
Subject). - The stream should end cleanly and declaratively, for example via
takeUntil. - The processing code should break down into small building blocks that are testable on their own.
Conversely, I stick with the raw emitter or reach for Node streams when it’s just a single simple handler or when I need real backpressure all the way through.
Conclusion#
In Node.js, RxJS shifts the perspective from command to description. The EventEmitter says what to do on every event; RxJS describes the whole event stream as a composable chain of pure functions. fromEvent is the bridge there without changing anything about the emitter’s semantics, and a Subject turns one stream into many consumers.
Two things you should never forget. First, discipline when cleaning up: without unsubscribe, take or takeUntil, listeners and processing stay alive. Second, the honest limit of the tool: RxJS handles overload only in a time-based and buffering way, not through real backpressure. If you keep these two points in mind, RxJS gives you a very pleasant way to process server-side event streams – readable, testable and extensible, without sinking into a thicket of callbacks.
Further reading#
- Repository examples:
/Users/mikebild/Documents/introduction-nodejs/EXAMPLES/RXJS/operators.js,.../subject.js,.../node.js - fromEvent
- Subject guide
- Observable guide
- Operators guide
- mergeMap
- throttleTime
- bufferTime
- scan
- subscribe arguments (deprecation)
- Glossary and semantics: hot/cold, unicast/multicast
- Node.js events
Revised and translated from German with the help of AI.