Beyond the scenario
Seemorgh operator console · Overview · 1 The crossing · 2 Sensors · 3 Fusion · 4 AI layer · 5 Checks · 6 Beyond the scenario
The challenge ended with a fused picture of a synthetic scenario. Three pieces of work went further:
- a real two-camera sensor, tested on a real drone
- a synthetic experiment that fuses a camera-shaped report into a track
- one run of cloud-hosted reasoning over the fused picture
Each answers a different question, and each has its own kind of evidence. They are kept apart here, as they are in the project’s evidence index, because blending them would make every one of them sound stronger than it is.
| Piece | Question it answers | Kind of evidence |
|---|---|---|
| Field sensor | Can two ordinary cameras find a small drone and say where it is? | Measured on real footage, 10 July 2026 |
| Bridge | Can a camera’s report join a fused track, and what does it do there? | Synthetic, in the scenario |
| Reasoning canary | Can a hosted model reason over the fused picture under guardrails? | Ran once, privately; the receipt is not public |
The field sensor
On 10 July 2026, two cameras filmed a DJI drone hovering at four distances from its take-off point, about 60, 45, 30 and 15 m. The drone logged its own position.
Detection. The detector is the YOLO11n model from the small-object experiment, run over each frame in overlapping 480 px tiles (stride 240, confidence threshold 0.10). It was benchmarked on 8 frames, 4 hovers × 2 cameras, against manually marked reference positions:
| Strategy | Frames where the drone was among the candidates |
|---|---|
| Whole frame, shrunk to the model’s input | 1 of 8 |
| Legacy 3 × 3 tiling | 4 of 8 |
| Overlapping square tiles | 8 of 8 |
With the overlapping tiles, the matched candidate sat a median 1.02 px from the reference mark. The cost is in the candidate count: 95 candidates across the 8 frames, about 12 a frame, of which one is the drone. Finding the drone among candidates works. Picking it out without a human reference does not yet: a separate, operator-cued selection check succeeded in 2 of 4 paired observations.
Position. For four frames from the second camera, synchronised to the first (the second camera runs 0.255 s later), the drone was located in each image by reference matching, and the two cameras’ rays were triangulated into a 3D position. The result was compared with the drone’s telemetry. The telemetry’s take-off point was fitted by least squares, with an RMS residual of 0.707 m.
| Hover | Telemetry slant range | Reconstructed slant range | Horizontal error | Vertical error | 3D error (envelope) | Ray miss |
|---|---|---|---|---|---|---|
| ~60 m | 59.9 m | 42.7 m | 30.4 m | −11.1 m | 32.3 m (25.0–40.7) | 3.5 m |
| ~45 m | 45.5 m | 38.9 m | 24.0 m | −5.8 m | 24.7 m (19.2–31.9) | 3.7 m |
| ~30 m | 30.2 m | 33.1 m | 20.2 m | −2.0 m | 20.3 m (15.2–27.2) | 2.3 m |
| ~15 m | 15.0 m | 25.4 m | 19.9 m | 0.2 m | 19.9 m (14.3–26.8) | 0.0 m |
Three things stand out:
- Range is compressed. The reconstruction puts the far hovers too close and the near one too far: 42.7 m for 59.9, and 25.4 m for 15.0. That pattern points to the camera geometry, such as baseline or orientation errors, rather than to random noise, though it wasn’t isolated.
- Error is mostly horizontal, 20–30 m, and doesn’t shrink much as the drone comes closer.
- The rays agree with each other to within 0–3.7 m. That says the two cameras are consistent with each other. It says nothing about whether they are right.
What this is and isn’t. It is four reference-matched frames, compared with telemetry, not with a surveyed position. The drone’s own GNSS error isn’t characterised. The envelope comes from assumed sensitivity spreads, so it is not a calibrated 1σ and not a confidence interval. Fully automatic association, finding the drone in both cameras with no human cue, is unproven.
The bridge: a camera report inside a fused track
The bridge asks a narrower question: if a camera-grade position report existed, would FUSE accept it into the right track, and what would it change?
At t=0, a synthetic optical observation is placed at the drone’s scenario truth plus noise, with an assumed horizontal accuracy of 6 m. It is not copied from the fused answer. It enters as a precision-class report with no threat label, so classification still comes from C2 and RF.
| At t=0, hostile track | Before | After |
|---|---|---|
| Source systems | 4 | 5 |
| Contributing reports | 5 | 6 |
| Modelled σ | ±28.6 m | ±5.9 m |
| Error against scenario truth | 36.4 m | 8.8 m |
| Confidence heuristic | 0.787 | 0.932 |
| Classification | Hostile | Hostile |
Six tests guard it:
- the observation isn’t copied from the fused answer
- it joins without a threat label
- it tightens σ and error
- it adds exactly one source and one report
- it keeps altitude near the truth
- it keeps report timestamps
The track’s altitude moved from 320.0 m to 327.2 m, because the synthetic observation carries altitude noise of its own.
Read the two together. The bridge assumed 6 m. The real field sensor, at 15–60 m range, was off by 20–32 m. By the same inverse-variance arithmetic, a camera report good to 20–30 m would bring the hostile track from ±28.6 m to roughly ±16–21 m, not ±5.9 m; that is an estimate, not a rerun, and it assumes the camera error is unbiased, which the range compression above suggests it is not. So the bridge proves the interface and the fusion behaviour, and the field result says the camera isn’t yet good enough to deliver what the bridge shows. The bridge is also a single instant, not a continuous camera track, and the field drone is never placed in the scenario.
The reasoning canary
The console’s reasoning brief was designed to come from a hosted model over the fused picture.
What ran. Once, in July 2026, Gemini through Vertex AI produced briefs across all six of the console’s timeline bookmarks. It ran in a private canary on a GKE Autopilot cluster:
- authenticated with Workload Identity
- non-root containers with read-only root filesystems
- health checks that never call the model
The guardrails. Briefs were shown with permanent guardrails, “human review required” and “no engagement recommendation”. The console showed a live-model label only when a brief passed schema validation.
The receipt is dated 15 July 2026 and is not part of the public record. The deployment has since been torn down. The brief in the public replay is the service’s deterministic offline fallback, not Gemini.
What this shows is that the deployment path and the guardrails worked once. It doesn’t show anything about the quality of the reasoning: no canary brief is published, and no evaluation of brief quality is part of the record.
What would come next
These are the gaps the evidence above points at. They are planned, not built:
- For the field sensor:
- surveyed camera positions and orientations, to attack the range compression
- blind association, with no operator cue
- live on-device inference
- For the bridge: feed the measured field accuracy instead of 6 m, over many frames, not one instant.
- For the fusion engine:
- real vendor data with calibrated error models
- temporal tracking with persistent identity
- the long-range radar’s range rate in association
- global assignment (JPDA or global nearest neighbour) instead of greedy grouping
- For reasoning: publish and evaluate briefs before calling them useful.
Basis: the field detection benchmark and the four-frame synchronised triangulation reproduction, both accepted and independently reproduced, and served in the public replay data as field-evidence.json; the bridge record bridge-0.json and the phase 7 tests in the FUSE package at the export commit named in the replay README (private source); and the project’s evidence index and deployment notes. No new runs were made for this page.
Further reading: Richard Hartley and Andrew Zisserman, Multiple View Geometry in Computer Vision, 2nd edition (Cambridge University Press, 2004), on two-view triangulation and why small calibration errors become large range errors.
Previous: How it was checked · Back to the overview