orthomind - click one object in a drone orthomosaic, and it finds every other one.
A processing engine that turns a zip of drone photos into an orthomosaic, a terrain model and a searchable catalog. On top of that sits the part that makes it interesting: delineate anything in the map by clicking it, or by typing what you are looking for. All of it on CPU, no GPU anywhere in the stack.
Fig. 01 Two passes over the same orthomosaic. The prompt "cars" returns nine vehicles scattered across a car park and a verge; the prompt "grass" returns the lawns and verges between the paths. Each pass writes one labelled GeoJSON layer, georeferenced and ready to open in QGIS.
The brief
Drone survey work produces beautiful maps that nobody can query. You get a very large GeoTIFF, and then somebody traces objects off it by hand in QGIS.
The client already had the flying and the photogrammetry sorted. What they wanted was the two steps on either side of it: a way to hand over a folder of photos and get back a catalogued, tiled, web-viewable map without babysitting it, and a way to pull structured features out of that map without a human tracing polygons for a day.
The hard constraint was the machine. This had to run on ordinary hardware, on-premise, with no GPU and no per-call inference bill. That ruled out most of the obvious answers and shaped everything that follows.
Point, click, delineate
The fastest way to tell a model what you mean is to point at it. Click a roof and you get that roof, traced to the pixel.
Under the hood this is SAM 2 running on the orthomosaic. Clicks are foreground hints, shift-clicks are background hints, and the mask updates between clicks so you can argue with it until it agrees with you. Commit the object and move on to the next one. When you are done, export the lot as a georeferenced GeoJSON attached to the map's catalog entry.
Getting this to feel responsive on CPU was most of the work. SAM 2 splits into an expensive image encoder and a cheap mask decoder, so the engine tiles the ortho, computes embeddings once, and caches them to disk. Every click after that only pays for the decoder, which is fast enough that the loop stays interactive even on a laptop-class machine.
Fig. 02 A walkthrough of the interactive session: clicking into an object, refining the mask, committing it, and sweeping the rest of the map for matches.
One example, then the rest
Tracing one object is a demo. Tracing four hundred is the job.
So once a mask is good, Find similar takes its embedding and sweeps every tile of the orthomosaic for things that look like it. No text prompt, no training, no labelled examples: one object you have already delineated is the entire specification.
Candidates stream back as scored masks while the sweep is still running, which matters when the sweep takes a few minutes. A score slider prunes the false positives live, and you accept the survivors in one action. It turns an afternoon of tracing into picking a threshold.
Or just describe it
When you know what you want and there are a lot of them, type it.
$ curl -X POST \ localhost:8000/jobs/<id>/segmentations \ -d '{"prompt": "cars"}'
GroundingDINO turns the phrase into boxes, SAM 2 turns the boxes into masks, and the masks are dissolved into WGS84 polygons. One prompt is one class and produces one layer, so you re-run it with "building", "vehicle", "solar panel" to build up as many layers as you need. It runs asynchronously and you poll it, since a large ortho on CPU is a minutes-scale operation rather than a request-response one.
A prompt that matches nothing completes cleanly with a count of zero rather than erroring. That sounds like a small detail, but it is the difference between a tool people trust in a batch script and one they wrap in a try-except.
The pipeline underneath
None of the above works without a boring, reliable pipeline beneath it.
-
01
Ingest
Photos arrive as a zip on a single endpoint. Processing options are validated against the engine before the job is created, so a typo comes back as a 422 in a second rather than as a failure forty minutes in.
-
02
Photogrammetry
OpenDroneMap turns the photos into an orthomosaic and a digital surface model, written as Cloud-Optimized GeoTIFFs. Progress is polled back onto the job record so the caller sees a live percentage rather than a spinner.
-
03
Catalog
Each finished job becomes a STAC item in PostGIS, with the rasters as assets. That makes the output searchable by bounding box and by date range instead of by remembering a filename.
-
04
Serve
A dynamic tile server reads the COGs straight from object storage, so the map renders in a browser at any zoom without anyone pre-generating a tile pyramid or downloading a multi-gigabyte file.
Note Both views come from one catalog entry and one set of Cloud-Optimized GeoTIFFs. Nothing is exported, converted or copied to make the map render.
How it's built
Seven services, one docker compose up, no cloud
account. The API orchestrates everything and is the only service
that talks to more than one of its neighbours; the rest are
off-the-shelf components doing one job each. The AI code is walled
off in its own module so the pipeline still stands up if you delete
it, which was a deliberate condition of the design rather than a
happy accident.
Where it stands
It works end to end and it is honest about what it is not.
The engine runs jobs, catalogs them, serves them and segments them, and it does that on a machine with no GPU in it. Two things follow from that constraint and are worth stating plainly. Segmentation is a minutes-scale operation, not an interactive one, on anything larger than a modest survey. And interactive sessions live in the API process, so they do not survive a restart, although the expensive caches on disk do.
There is no authentication yet, which is fine behind a private network and not fine anywhere else. The next items on the list are post-processing to collapse the thousands of tiny polygons a generous prompt produces, multi-class prompts resolved in a single pass, and optional GPU support for the sites that have one.