Explained / Pictures, sound and video
It can look at photos now. How good is it actually?
The short answer
Modern models can take a picture, a recording or a video as input and answer questions about it. This is genuinely useful and often very good. It is also confidently wrong in particular ways: small text, exact numbers, counting things, and anything where the answer depends on a detail rather than the overall scene.
It reads the scene before it reads the detail
A model handles an image by turning it into the same kind of internal representation it uses for text. What survives that process is the gist: what the thing is, roughly what it says, the general layout.
Fine detail survives less well. That is why it will describe your photo of a letter accurately and then misread the reference number, or tell you there are five people in a picture of six.
Ask it about the overall content and it does well. Ask it for an exact figure buried in the corner and you are testing precisely the part that degrades.
PIXEL / THE COACH
I am genuinely good at “what is this letter about”. I am unreliable at “what is the account number on it”. Those feel like the same question and really are not.
Where it is strong and where it is not
The pattern is consistent enough to plan around.
What to trust, and what to check
- Reliable
- Describing a scene, explaining a chart's message, reading clear printed text, summarising a recording.
- Check it
- Numbers of any kind, dates, reference codes, handwriting, anything small or low-contrast.
- Do not rely on it
- Counting objects, precise measurements, identifying a specific person, reading a busy table.
The shape of the strengths rather than a specification. Models differ and improve, but the weak spots have been stubbornly consistent.
Sound and video have their own traps
Transcription is now very good, and bad audio is where it stops being good. Crosstalk, accents it heard little of in training, and technical vocabulary all produce confident, fluent, wrong transcripts.
Names are the worst case. A misheard name looks exactly like a correctly heard one, and it will be spelled plausibly. Always check names and figures against something else.
Video is usually handled as a series of still frames plus the audio. So it may miss something that happens between the frames it sampled, and it is weak on anything requiring a sense of timing.
What to actually do
- Use it for the gist, then read the detail yourself when the detail matters.
- Never accept a number, name or date read from an image without checking the original.
- Ask it what it cannot make out. It will often tell you where the image is unclear.
- Remember that a photo carries far more than you intended, and crop before uploading.
See it for yourself
Reading about it only gets you so far. This one is written to show you the thing described above, rather than describe it again.
Here is a photo of a handwritten note. Transcribe it exactly as written, and mark clearly anything you are guessing at rather than actually reading.
Then paste it into whichever assistant you use — ChatGPT, Claude, Gemini or Copilot. Not sure which? Meet them all.
If you remember nothing else
- It gets the gist of an image reliably, and loses fine detail.
- Numbers, codes, handwriting and counting are where it fails most.
- A misheard name in a transcript is spelled plausibly and looks correct.
- Video is usually sampled as frames, so it can miss what happens between them.
- Crop before you upload. A photo hands over far more than the part you asked about.
Where this came from
- What each model accepts, recorded per modelchecked 26 September 2026
Anything here that turns out to be wrong is worth telling us about. Figures that move — prices, sizes, energy use — are kept in the registry with their dates, rather than written into the prose where they would quietly go stale.
Questions about “Pictures, sound and video”Asked by readers, answered by us. Open to read them.
Open to load them.
Ask about this page and we will answer it here, where the next person can read it too.
Nobody else can answer you, and you cannot answer anybody else. This is not a forum — we would rather say that than run a room we cannot watch at three in the morning.
Your question goes up with the answer and without your name. Nothing identifies you, ever.
If we cannot answer it we will tell you why, and that one stays between us.