Every AI that finds things in a picture does it the slow way. It writes the box out one number at a time. Top-left corner. Then the other corner. Then the label. For one object in one photo, fine. For a shelf holding two hundred products, it crawls.
NVIDIA changed the order of operations. Their new model stopped spelling boxes out corner by corner. It commits to the whole box in a single move.
The method is called parallel box decoding. On one H100 it clocks about 12.7 boxes per second, roughly ten times a comparable Qwen3-VL and about 2.5 times the next-fastest open model. It is 3 billion parameters. It sits on Hugging Face right now, free to download. There is one line in the license that most of the posts about it skipped, and if you run a business it is the only line that matters. We will get there.
What the Model Actually Does
You hand it an image and a plain-English request. It hands back coordinates.
That is called visual grounding. Not "describe this picture" but "show me exactly where the thing I named is." Point at an invoice and ask for every line item. Point at a screenshot and ask where the Submit button sits. Point at a warehouse photo and ask for every pallet. It returns boxes as raw numbers you can act on: <box> x1, y1, x2, y2 </box>.
The same model covers a spread of jobs from one prompt style.
| Job | What you ask it |
|---|---|
| Object detection | Locate every instance of a category |
| Phrase grounding | Locate the thing this phrase describes |
| Text / OCR detection | Detect all text and box it |
| GUI grounding | Locate the button or field matching a description |
| Pointing | Point to a single object |
| Document layout | Map the regions of a page |
One model, one prompt shape, six jobs. That is the part worth saving.
Why It Is Faster: Whole Boxes, Not Corners
Older grounding models treat a box like a sentence. They generate it token by token. First coordinate, second coordinate, third, fourth, label. Five predictions for one box, in sequence, each one waiting on the last.
Parallel box decoding treats the box as a single unit and predicts it in one structured step. The geometry stays consistent because the corners are decided together, not stitched together after the fact.
Speed is the obvious win. The quiet win is accuracy at tight tolerances. On standard detection benchmarks it edges out the previous best open model at the same size, and the gap widens when you demand the box hug the object closely. Faster and slightly sharper, from the same 3 billion parameters.
It defaults to a hybrid mode: predict everything in parallel first, then fall back to the slow careful method only for the cases it finds ambiguous. You get the speed on the easy boxes and the caution on the hard ones without choosing.
What You Need to Run It
Read this before you get excited, because this is where most people stop.
This is not a Mac tool. It is not a website you log into. It is a model you run on your own NVIDIA GPU, on Linux, in Python.
- GPU: an NVIDIA card from the Ampere line or newer. A100, H100, an L40, or an RTX 4090 all qualify. Without one, it will not run.
- Memory: somewhere between roughly 12 and 35 GB of video memory depending on how you configure it.
- OS: Linux only.
If that is a wall for you, it is a real wall, and no prompt gets you over it. If you have the hardware, setup is short.
Install the dependencies:
Copy this.
pip install opencv-python-headless==4.11.0.86 transformers==4.57.1 \ numpy==1.25.0 Pillow==11.1.0 peft torchvision decord==0.6.0 lmdb==1.7.5
PyTorch installs separately, matched to your CUDA version. Then load the model and ask it something:
Copy this.
from transformers import AutoModel, AutoProcessor
from PIL import Image
processor = AutoProcessor.from_pretrained(
"nvidia/LocateAnything-3B", trust_remote_code=True)
model = AutoModel.from_pretrained(
"nvidia/LocateAnything-3B", trust_remote_code=True, device_map="auto")
image = Image.open("shelf.jpg").convert("RGB")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": "Locate every bottle"}
]
}]
text = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=8192, generation_mode="hybrid")
Keep max_new_tokens high and leave the mode on hybrid. Drop them too low and long results get cut off mid-box. The exact, current runnable script lives on the model card, so pull it from there rather than typing this from memory.
Done. If the model returned a block of coordinates, it is working.
Copy This: The Detection Prompt
The whole interface is the text line inside the message. Swap it and the job changes. Keep this pattern nearby.
Copy the grounding prompt
Copy this.
Detect every [WHAT YOU WANT] in this image. Return each one as a box. Examples of [WHAT YOU WANT]: - "invoice line item" - "product price tag" - "the Submit button" - "handwritten note" - "empty parking space"
If it misses things or boxes the wrong stuff, the fix is not to restart. Tighten the noun. "Bottle" is loose. "Wine bottle, upright, on the shelf" is not. Say what you mean the way you would to a new hire on their first shift.
Where an Operator Would Use It
The demos show cars and people. The money is in the boring pictures your business already generates.
Invoices and forms. Point it at a scanned document and pull every field to a box. That is the front half of any document-processing pipeline, the part that usually needs a paid API per page.
GUI agents. If you are building anything that clicks buttons on a screen, it needs to find the button first. This finds buttons by description, which is the hard step in every screen-automation project.
Shelf and inventory counts. One photo of a shelf, one prompt, a count and a position for every item. The kind of task a person does with a clipboard.
Text out of images. Menus, labels, signage, whiteboards. It boxes the text so you know where each piece sits, not just what it says.
Each of these is a job someone on your team does by hand today, and each is a job this model does in seconds. That is the case for caring about a research release with a technical name.
The Catch: The License Says Research Only
Here is the line the viral posts left out.
LocateAnything-3B ships under NVIDIA's non-commercial license. In plain terms: you may use it for academic and non-profit research. Commercial use is not permitted, except by NVIDIA itself.
"Free to download" and "free to build a business on" are two different sentences. This model is the first one. You can pull it, test it, benchmark it against what you pay for today, and learn exactly what parallel box decoding buys you. You cannot drop it inside a product you sell, or a client pipeline you invoice, without a commercial arrangement you do not currently have.
So treat this release as intelligence, not inventory. It tells you where open vision models now sit on speed and cost, which is a real number to carry into any vendor conversation. It is a yardstick, not a supplier. If your use is genuinely internal research or a non-profit, read the full license terms yourself and confirm your case fits before you rely on it.
If you want the same "run a capable model on your own hardware, no per-call bill" idea on the language side where the licensing is friendlier, the free NVIDIA coding-model route is the companion piece.
Honest, the Other Limits
The license is the big one. These are the rest.
- Hardware or nothing. You need an NVIDIA GPU and a Linux machine to run it at all. There is no hosted version you can just log into.
- Not the fastest to deploy at scale yet. The tuned server paths most teams use for production inference are not available for it yet, so scaling it up takes extra engineering.
- Short, literal prompts. It was trained on short task-style requests. Long, flowery instructions are not what it expects. Keep the ask blunt.
- It finds, it does not decide. It hands you coordinates. What those boxes mean for your process is still your logic to write.
If You Do One Thing
Do not build on this. Not this week. Building on it is the mistake the headlines are quietly pushing.
Instead, if you have the hardware, spend ten minutes running it against one image from your actual work. A real invoice. A real shelf. A real screenshot. Watch how fast it boxes what you asked for, and how close the boxes sit.
Then you know two things you did not know this morning. What open vision detection can now do. And exactly what the tools you pay for should have to beat. That is worth ten minutes and a research license.