TL;DR
- Claude bills an image as 28 by 28 pixel patches:
ceil(width / 28) x ceil(height / 28)visual tokens. A 1920x1080 screenshot is 1,560 tokens after standard-tier downscaling, and 2,691 on the high-resolution tier. - Sampling a 10 minute recording at one frame per second is 600 frames. That is 936,000 visual tokens on the standard tier and 1,614,600 on the high-resolution tier, before the agent reads a single line of your code.
- There is also a hard ceiling that has nothing to do with cost: 100 images per request on models with a 200k context window, 600 on the others. You cannot send 600 frames to a 200k model at any price.
- The transcript of the same 10 minutes is roughly 2,000 tokens. A real AREC document we publish, covering a 44 second recording with six evidence-backed key moments, is 17,714 bytes, about 4,400 tokens.
- The interesting part is not that text is cheaper. It is that frames and transcripts fail in opposite directions, and a structured document is what lets you pay for the few frames that actually carry information.
The question nobody costs out
"Just give the agent the video" is a sentence people say a lot and almost nobody prices. It sounds like a small ask. You already have the recording. The model has vision. Surely you hand it over and the agent sees what you saw.
It does not work that way, and the reason is worth doing the arithmetic on, because the arithmetic decides the architecture.
This calculation covers sending sampled still images to Claude. Video-capable APIs such as Gemini accept video directly and use different processing and token accounting. Do not apply the numbers below to every model or video API.
What a frame actually costs
Claude's vision documentation is unusually specific about this, which makes it a good thing to calculate against. Claude views an image in patches rather than pixels. Each patch is a 28 by 28 pixel block and counts as one visual token. So an image costs:
ceil(width / 28) x ceil(height / 28)Each model also has a maximum native resolution, expressed as a long-edge limit and a visual-token limit. Anything larger is downscaled first, which caps the per-image cost:
- Standard tier: 1,568 px long edge, 1,568 visual tokens maximum.
- High-resolution tier (Claude 4.7 and later): 2,576 px long edge, 4,784 visual tokens maximum.
Run a normal screen recording through that. A 1920x1080 frame downscales to 1456x819 on the standard tier, which is 1,560 tokens. On the high-resolution tier it is not resized at all, and costs 2,691 tokens. A 4K frame costs 1,560 and 4,784 respectively, because both tiers cap it.
Hold that number: one screenshot of an ordinary desktop is between 1,500 and 2,700 tokens.
Now multiply
A 10 minute recording at 30 frames per second contains 18,000 frames. Nobody sends 18,000 frames, so you sample. Here is what sampling costs:
| Sampling | Frames | Standard tier | High-resolution tier |
|---|---|---|---|
| 1 frame per second | 600 | 936,000 tokens | 1,614,600 tokens |
| 1 frame per 10 seconds | 60 | 93,600 tokens | 161,460 tokens |
| 1 frame per 30 seconds | 20 | 31,200 tokens | 53,820 tokens |
The top row is not a pricing problem, it is a physics problem. 1.6 million tokens does not fit in any context window you are likely to be using, and even if it did, you have spent the entire budget on pictures of a mostly static screen before the agent has looked at one line of your code.
The bottom row fits. It also samples once every 30 seconds, which means it will reliably miss the click you are asking about. Most UI events that matter last well under a second. A dropdown opens, a toast appears, a button enters a loading state and never leaves it. Sample every 30 seconds and you get 20 pictures of a screen at rest.
This is the trap. The sampling rate that is affordable is the sampling rate that misses the evidence, and the sampling rate that catches the evidence is unaffordable. There is no setting in the middle that fixes it, because the problem is not the rate. It is that uniform sampling has no idea which moments matter.
The ceiling that price cannot solve
Cost is the soft limit. There is a hard one underneath it.
The Claude API accepts a maximum of 100 images per request on models with a 200k-token context window, and 600 on other models. claude.ai allows 20 per turn. Above 20 images in one request, a stricter per-image dimension limit kicks in for every image in that request, so you also start losing resolution exactly when you are sending the most pictures.
For a 200k-context model, the 600-frame plan exceeds the 100-image limit. Models allowing 600 images still need enough context and request capacity; the image-count limit alone does not reject that plan.
And one more detail that surprises people: GIF animations are unsupported, and only the first frame is used. A tool that "sends the recording as a GIF" is sending a single still image and a filename.
So use the transcript. Except.
The transcript is the obvious escape hatch, and it is genuinely cheap. Speech runs around 150 words per minute, so 10 minutes is roughly 1,500 words, call it 2,000 tokens. That is three orders of magnitude below the one-frame-per-second plan.
The problem is that a transcript of somebody demonstrating software is full of words that point at things:
"So I click this, and then that should turn green, but instead it just sits there."
Every pronoun in that sentence is a pointer into a frame that the transcript does not contain. Linguists call these deictic references. A transcript preserves the words and throws away everything they refer to. The agent receives a grammatically complete sentence carrying almost no information.
We measured this in a controlled benchmark on a visually grounded task. Transcript-only context scored 5.8 out of 12 with zero strict completions across five trials. It is not that the transcript is wrong. It is that "make that button do the thing I pointed at" is unactionable no matter how accurately it is transcribed.
So frames blow the budget and the transcript loses the referents. Both failures are real, and they are opposite.
What actually fits
The way out is not a better compression ratio. It is spending the frame budget deliberately instead of uniformly.
Almost all of the information in a screen recording is concentrated in a few instants: the moment of a click, the moment an error appears, the moment somebody points at something and says "there". Between those instants the screen is mostly unchanged, and uniform sampling spends the same tokens on the still parts as on the decisive ones.
If you can identify those instants, you can afford full-resolution frames for them, because there are so few. And you can afford to keep the entire transcript, because text is nearly free. Then you attach the two together, so that "that button" resolves to a frame and a coordinate rather than to nothing.
That is the whole idea behind AREC, the open format we publish for agent-readable recordings: a Markdown document carrying the transcript, timestamped key moments with precomputed frame URLs and pointer coordinates, redacted diagnostics, and an explicit statement of what was not captured.
Here are real measurements rather than estimates. The AREC document for the 44 second recording in our published benchmark is 17,714 bytes, roughly 4,400 tokens, and carries six key moments each with a frame and a crop. A shorter hosted recording produces a 6,619 byte document, roughly 1,650 tokens.
Compare like for like. That 44 second recording, sampled at one frame per second, would be 44 frames costing between 68,640 and 118,404 visual tokens. The structured document is about 4,400 tokens of text, plus whichever frames the agent decides to actually look at, which might be one or two for a focused question.
The saving is not the point. The point is what you get for the money: the six moments are the ones where something happened, chosen because of a click, a deliberate highlight, or a spoken reference, rather than because a timer fired.
Frames are still worth paying for
It would be convenient to conclude that text wins and you should never send an image. That is the wrong lesson, and the benchmark above is the reason: the transcript-only condition was the one that failed.
Vision is expensive per unit and irreplaceable for certain questions. Is the spacing wrong. Is the contrast too low. Which of these four buttons is the one being described. No amount of transcript answers those, and a coordinate on its own does not either. You need the pixels.
So the sane architecture is not "text instead of frames". It is text for the timeline, frames for the evidence: keep the cheap continuous thing continuous, and spend the expensive discrete thing only where it carries information. Precompute those frames so the agent fetches one when it needs one, rather than receiving 600 speculatively.
This is also why the AREC specification requires evidence references to point at precomputed artifacts and forbids a document that depends on a server extracting frames on demand. An agent should be able to decide to look at moment four without your server decoding a video to answer.
How to check this yourself
Do not take the numbers on trust. The formula is public and the arithmetic is small:
- Take a screenshot at your normal display resolution.
- Apply the model's documented resizing rule first, then compute
ceil(width / 28) x ceil(height / 28)at the resized dimensions. - Multiply by the number of frames your sampling plan produces for a recording of typical length.
- Compare that to your context window and the image-per-request limit for your model: 100 for 200k-context models, 600 for the others.
Most people find that the plan they had in mind fails on the image count before it fails on the budget, which is usually the moment the architecture question becomes concrete.
Give your agent a normal Clipy recording link and ask it to inspect the available context. The benchmark page also publishes the source video, recording context, and trial results so you can examine the comparison.
The summary that matters
A screen recording is one of the densest artifacts a person can produce in 30 seconds and one of the most expensive things to hand to a model. Sending it as uniformly sampled frames can exceed image-count or context limits. Sending it as a transcript throws away every word that pointed at something.
What works is deciding, before the agent ever sees it, which handful of instants carry the information, and paying full price for exactly those.
Clipy is the agent-readable screen recorder. Record once: people get a normal video link, and coding agents get structured context they can act on. The format is open and documented, with a schema, a validator and a conformance suite.