Put two cameras on the same lobby with overlapping fields of view. Ask each one how many people are in it. Add the answers.
You now have a system that reports twelve people in a room containing seven, and it will be confidently wrong forever. Every downstream number inherits the error: occupancy, dwell time, throughput, the ratio you were going to use to decide whether to staff a second desk.
This is the least glamorous problem in video intelligence and, in our experience, the one that most determines whether a deployment is trusted after month two. The fix is a design commitment: the record resolves the same physical object into one entity, no matter how many sensors see it.
What "one entity" has to mean#
Two cameras watching one person yield one entity with two viewpoints. Not two rows that a reporting layer tries to reconcile later. Not a "probably the same person" confidence score that every consumer of the data has to re-interpret.
The mechanical consequences show up immediately, and they're a decent test of whether a system actually does this:
- A people counter reads deduplicated, and says so.
- Occupancy across overlapping cameras merges by max, never sum.
- Dwell time is measured across the whole path, not restarted at every camera handoff.
- "Where did that person go next" is a query against one entity's history rather than a manual stitch across three streams.
Once that identity exists, everything else in the building can attach to it. Audio events, badge swipes, GPS pings, and transaction rows all reference the same entity id. That is the precondition for joins to be meaningful: they land on entities, not on per-camera pixel fragments.
The correction test#
Here's the sharpest way to see whether identity is real in a system or merely claimed. Relabel something once.
Tag object 41 as "the night-shift contractor." If identity is real, that label now holds everywhere object 41 appears: every camera angle, the badge-swipe join, the audio transcript, the episode history, last month's occupancy report. One id, one row of truth, nothing re-labeled per source.
If identity is not real — if it's a reconciliation step that runs at query time or in a reporting layer — then you've labeled one camera's version of a person, and the other four still show a stranger. We've seen this failure mode enough to treat "does a correction propagate" as the acceptance test rather than a nice-to-have.
Why the join needs it#
The questions operators actually ask usually don't live in any single sensor. They live in the join, and a join against fragments is worse than no join at all because it produces plausible garbage.
Tailgating is more people through the door than badges swiped. If the two cameras covering that door each contribute their own person-row, every legitimate entry looks like tailgating. The alert is now noise, the team turns it off, and the feature is dead — not because the detector was bad, but because identity wasn't resolved before the count.
The same structure repeats across every fusion case we care about:
| Source | What it adds | What breaks without one identity |
|---|---|---|
| Badge / access control | Who was authorized, and when | Every multi-camera door reads as a tailgate |
| RFID / NFC | What was scanned, and where | A pallet tracked by two cameras looks like two pallets |
| GPS / telematics | Where equipment actually went | Vehicle paths fragment at every camera boundary |
| WMS / POS | What the paperwork says happened | Counts don't reconcile, so nobody trusts the exception report |
| Alarm panels | What tripped, at what second | "Was a person in frame" returns duplicates, inflating severity |
The worked case we keep coming back to is the impossible badge swipe. A badge opens a server-room corridor at 9:12 p.m. Camera analytics alone see an authorized badge and a person, and nothing alarms. But the badge holder's laptop has an active VPN session 240 miles away, and her calendar shows PTO. Each system independently passes. The join reveals a cloned credential.
That join is only possible if the person in the corridor is one entity that the badge row, the VPN session, and the calendar can all be attached to. The event does not exist in any single modality, and it does not exist in a fragmented one either.
Channels, not cameras#
The useful generalization is that the record doesn't care what a channel is. A channel is anything that can be turned into typed rows and embeddings. Once it is, it inherits the whole machine: encrypted vector storage, the SQL surface over perception views, provenance columns on every relation — and one identity per physical object across every source.
That's why adding a modality isn't an architecture change. Audio is a channel. Badge events are a channel. Depth is a channel: we've been running three time-of-flight cameras hardware-synced into a live point cloud of one of our offices, and the 3D shapes embed the way frames and crops already do, land in the same encrypted store, and attach to the same entity ids the RGB pipeline resolves. The correction mechanic carries over unchanged — relabel an entity once and the label holds in the flat views, in the point cloud, and in every joined table.
(That depth rig is a working experiment, not a shipped product, and we'll write it up properly when it has benchmarks rather than screenshots.)
The privacy boundary is part of the design#
Cross-camera identity is exactly the capability that deserves scrutiny, so it's worth being precise about what it is and isn't.
Re-identification features are scoped to track association only — matching a person to themselves across a handoff — and are never used for general semantic search. That's a contract on the vector space, not a policy document. Face embedding is opt-in per zone and, where enabled, rides the homomorphic encryption path, where the server computes on ciphertext and never sees the query, the scores, or the answer.
Everything else in the record is encrypted in use as well: searched without server-side decryption, with token-level permissions and source attribution so access is scoped and auditable per query. An entity id is a durable handle to a physical object in a permissioned store — deliberately not a name, and deliberately not a biometric identity unless someone with the authority to do so turned that on for that zone.
There's a genuine benefit for the people being recorded here, and it's the one we'd argue first. A system that can answer "who watched what, when" about its own operators is a meaningfully better privacy posture than a DVR plus a monitoring contract, which cannot answer that question about anyone.
Where this is hard#
Cross-camera association is not solved, and we don't claim it is. Handoffs across non-overlapping views, long gaps, changes of clothing or lighting, crowds dense enough that tracks swap — these are open problems, and our approach is to make the failure legible rather than to hide it: association carries evidence, corrections propagate, and a wrong merge is something an operator can undo once instead of chasing through five systems.
The honest framing is that identity resolution is a quality dial, like every other stage of the pipeline. It gets better with a better model, more viewpoints, and richer non-video channels to anchor against — and a badge swipe, an RFID scan, or a GPS ping is often a far stronger identity signal than any amount of pixels. Which is the argument for the multimodal record all over again, arriving from the other direction.
