Reading a heart rate from a webcam is easy to demo and hard to trust.
The demo version works like this: watch the green channel of someone's face, find the periodicity, print a number. It looks convincing in good light on a still subject. Then the person turns toward a window, or leans back, or the laptop drops a few frames, and the number keeps updating with total confidence while meaning nothing at all.
That gap is the whole problem. A number that is right most of the time and silently wrong the rest of the time is worse than no number, because an app built on top of it has no way to tell the difference.
Between June 1 and June 5 we published eleven releases of @elata-biosciences/rppg-web, taking it from 0.4.0 to 0.14.0. Almost all of it was about closing that gap.
HRV needed sub-frame timing
The first thing we fixed was heart rate variability, which was quietly broken by arithmetic.
At 30 frames per second, one frame is 33 milliseconds. If you find heartbeats by picking the peak sample in a filtered signal, every beat-to-beat interval you measure is a multiple of 33ms. But the variation you are trying to measure in HRV lives in roughly that same range. The quantization is as large as the signal.
Two changes fixed it. First, sub-sample peak interpolation: instead of taking the peak frame, we fit around it and estimate where the true peak fell between samples. Second, and more usefully, we moved beat timing onto Hilbert phase. Rather than asking "which frame was the peak," we track the instantaneous phase of the pulse waveform and read beat boundaries off the phase crossings. That gives continuous timing rather than timing snapped to a frame grid.
We also added NN-interval cleaning, because a single missed or doubled beat will distort an HRV window far more than the noise you were worried about.
One region of interest is not enough
Skin perfusion is not uniform across a face, and neither is lighting. The forehead might be washed out while the cheek is clean, or the reverse.
So we now sample several regions and fuse them, weighted by signal-to-noise ratio, so the regions that are actually carrying pulse dominate the result. In 0.12.0 this moved from opt-in to on by default in the live pipeline, along with unified geometry so the fusion regions and the guidance overlay refer to the same thing.
We also shipped the parts underneath. CHROM, gain, and SNR primitives are exported directly, so if you want to build a different estimator on the same front end, you are not forced to accept ours.
Saying nothing is a valid output
The rest of the sprint was about refusing to emit numbers we could not stand behind.
The no-reference BPM guard catches the case where the pipeline is tracking something periodic that is not a pulse. Capture-confidence gating scores the incoming frames themselves, so poor light or heavy motion registers as poor capture rather than as a plausible heart rate. A headless BaselineCalibrator lets an app establish a per-person baseline without any UI attached to it.
None of this is glamorous. It is the difference between a party trick and something you can put in front of a user.
Face landmarks and a cautious affect signal
We replaced FaceMesh with FaceLandmarker, which gave us better tracking and a richer landmark set, and then used it for two things.
The first is practical: live face-framing guidance and a region overlay, so a person can see when they have drifted out of a usable position instead of wondering why their reading went strange.
The second is a dimensional affect estimate, valence and arousal, fused from facial landmarks and the pulse signal. We want to be precise about what this is. It is a two-axis estimate with real uncertainty, not emotion recognition, and we are not going to label anyone's feelings from a webcam. We expose it as coordinates with confidence attached, and we would rather ship it honestly framed than dress it up.
Alongside it we added multi-cue respiration, which pulls breathing rate from more than one signal path rather than betting everything on one.
Also in the window
- White and light theme across every demo and scaffold, since the dark-only default was making the demos hard to read in bright rooms.
- Replay benchmarking, so a change to the estimator can be measured against recorded footage instead of against whoever happened to be sitting in front of the camera.
run.shsplit intorun-lib.shand ajustfile, withlinkandunlinkrecipes for developing against a local build of the SDK.
Try it
npm create @elata-biosciences/elata-demo my-app -- --template rppgNo headset, no hardware, no account. A webcam and a browser.
Everything above is in @elata-biosciences/rppg-web 0.14.0, and the source is at github.com/Elata-Biosciences/elata-bio-sdk. If you find a case where the pipeline gives you a number it should have withheld, that is the bug report we most want to see.