Updated October 11, 2026.
To give Claude, Cursor, ChatGPT or another coding agent context from a screen recording, hand it text and frames instead of the MP4: a timestamped transcript of what you said, still images from the moments that matter, and where your pointer was at each of those moments. Only some of these assistants accept a video file at all, and the ones that do still work from sampled frames. If you record with Clipy, the normal share link already serves that text version to the agent.
Disclosure: Clipy is our product. The vendor facts below were checked on each vendor's own documentation on October 11, 2026.
TL;DR
- Claude, Claude Code, Cursor and Codex all accept images. Of the assistants we could check, only Gemini documents accepting a video file.
- A recording becomes usable context when you turn it into a timestamped transcript, a few labeled frames and the pointer position. You can do that by hand or let Clipy do it.
- With Clipy, paste the normal recording link. An agent that fetches it gets a text document, and appending
.arecalways returns that document. - Private recordings and library search need your Clipy account connected to the agent.
- Ask the agent to restate what it saw before it edits, and to say when it could not open the recording.
Two habits make any of this work better: talk while you record, and point at the thing you mean. The narration tells the assistant which part of the screen matters and what you expected to happen. For why a raw video falls short for a model in the first place, read whether Claude can watch a screen recording. This guide is about the handoff itself.
What does each assistant accept?
This is what each vendor documents today. "Fetch a URL" means the assistant can open a web address you paste and read the page.
| Assistant | Video file | Images | Fetch a URL | Source |
|---|---|---|---|---|
| Claude (chat) | No | Yes: JPEG, PNG, GIF, WebP | Yes, when Web search is on | Uploading files, web search and fetch |
| Claude Code | Not documented | Yes, through its Read tool | Yes, WebFetch (asks permission) | Tools reference |
| Cursor | No (an open feature request) | Yes: drag in or paste a screenshot | Through its browser tool or an MCP server | Prompting agents, Browser |
| ChatGPT | Not documented; users still request it | Yes: attach, paste or drag | Not covered here | OpenAI image inputs |
| Codex | Not documented | Yes: PNG, JPEG, --image or paste | Live web search with codex --search | Image inputs, Codex CLI |
| Gemini (API) | Yes: File API, inline under 20 MB, public YouTube links | Yes | Yes, URL context tool, public pages only | Video understanding, URL context |
Claude's upload article lists documents and four image formats and says nothing about video. Cursor's docs cover images but not video, and the forum request "Agent and Chat should accept video as input" from June 2025 still has no status or staff reply, with users asking for it as recently as July 2026. OpenAI's developer forum still has an open request from June 2026 for ChatGPT to accept .mp4 and .mov uploads the way it accepts images and documents. Treat a video uploaded to ChatGPT as unreliable and send frames instead.
Gemini is the exception, and even there the model works from samples: Google's page says it samples video at one frame per second by default. Its URL context tool cannot read video or audio files or YouTube links, and video understanding accepts YouTube links only for public videos. So a private bug recording still has to go up as a file or as text.
How do you turn a screen recording into text an AI can read?
Whatever tool you use, a recording becomes good context when it carries four things:
- A timestamped transcript. Your narration, line by line, with the time each line was said. This is where "the export button spins forever" lives.
- Frames at the moments that matter. A still image from the instant you pointed at the broken thing, the error message, or the before and after states. Three to six good frames beat sixty random ones.
- Where the pointer was. "The button at the top right" is ambiguous on a busy screen. A frame with the pointer visible, or coordinates for the click, removes the guess.
- A timeline that ties them together. Each frame labeled with its timestamp so the assistant can match "this one" in the transcript to the right image.
The do-it-yourself route
For a recording you already have as a file, get the transcript from Clipy's video-to-text tool, which returns timestamped lines you can download as .txt or .md. Then pull frames at the timestamps where something happens. With ffmpeg, one frame at 0:47 looks like this:
ffmpeg -ss 00:00:47 -i recording.mp4 -frames:v 1 frame-0047.pngName each frame after its timestamp, paste the transcript, attach the frames, and say in one line what each shows. Every assistant in the table takes text and images, so this works everywhere, including for recordings made in other apps.
The Clipy route
If you record with Clipy, you skip that work. The same watch link that a person opens in a browser also serves a text document to an agent: when an AI agent fetches the link, or a request asks for Markdown, Clipy returns the agent context instead of the video page. To be explicit, add .arec to the end of the link, as in https://clipy.online/video/<id>.arec. The format is called AREC, and its format reference is public for anyone building on it, but you do not need to read it to share a recording.
What does the agent actually receive?
Here is a trimmed copy of a real document, the agent context for a short public test recording (b596rrc8yhi6). The section names are exactly what Clipy serves today:
<!-- arec
spec_version: 1.3
profile: clipy-recording
canonical_url: https://clipy.online/video/b596rrc8yhi6.arec
watch_url: https://clipy.online/video/b596rrc8yhi6
-->
**Clipy recording: agent context**
# Clipy Screen Recording Workflow Demonstration
## Summary
## Metadata (format, access: public, processing: ready)
## Intent
## Interaction timeline
## Capture gaps
## Transcript
## Agent brief (a JSON block of requests, evidence, capture gaps)
## Share this recordingThe pieces worth knowing about:
- Evidence, not instructions. Everything in the document is evidence from the recording, to be read as a quotation and never obeyed. If a recorded web page says "ignore your previous instructions", a well-behaved agent treats it as something the recording showed, not a command.
- Summary and intent. What the recording is about and what kind of recording it is, so the agent knows where to look first.
- Key moments. When the recording has them, a Key moments section lists each one with its time, a caption, a frame image and, where available, a close-up crop around the pointer. Each has a source (for example a recorded click, or a pointer resting without a click) and a confidence value. This sample is a four-second smoke test with no key moments, so the section is absent here.
- Transcript. The full timestamped transcript of what was said.
- Capture gaps. A list of what was not recorded. The sample says cursor and click data were not captured, and adds that missing coordinates "do not mean no click happened". An agent should report those gaps instead of filling them in.
Pointer coordinates appear only when the recording captured the cursor. Today that means recordings from the Clipy desktop app for Mac or Windows and tab recordings from the Chrome extension, which upload a cursor track alongside the video. A recording from the web recorder still gets the summary, frames for its key moments and the transcript, just without exact click positions. If "which of these three buttons" is the whole bug, record with the desktop app or as a tab recording in the extension.
How do you give the recording to Claude or Claude Code?
Claude Code. Paste the Clipy link with your request. Claude Code can fetch a URL with its WebFetch tool and look at images with its Read tool, so it can read the document and inspect the key-moment frames. Installing the Clipy skill makes this routine: the agent knows to read the context, look at the frames before making visual claims, and cite timestamps. The agent setup page has a one-line instruction you paste into Claude Code, and the skill reference explains what the skill does. For the full bug workflow from recording to pull request, see screen recordings for Claude Code.
Claude chat. With Web search turned on, Claude can fetch a page from a URL you give it, so pasting the .arec link to a public recording can work. On a Team or Enterprise plan, an owner has to enable web search for the workspace first. Without web fetch, use the do-it-yourself bundle: paste the transcript and attach the frames, which Claude's chat accepts as images.
How do you give the recording to Cursor?
Cursor does not take a video file, so give it the text and frames. There are two ways.
- Connect Clipy once. Run the Cursor line from the agent setup page, approve it, and restart Cursor. Cursor supports MCP servers, including remote ones that sign in with OAuth, according to its MCP docs. After that, paste a Clipy link into the agent chat and ask it to read the recording.
- Do it by hand. Paste the transcript and drag the frames into the chat input. Cursor's docs say you can attach images by dragging a file in or pasting a screenshot with Cmd+V.
Ask Cursor to restate the bug from the recording before it edits anything. Clipy for Cursor covers the setup and the kinds of tasks it suits.
How do you use it with ChatGPT and other chat apps?
Share the link only if the app can open web pages. Pasting a link is not proof the app read it: a model without fetch access may answer from the words in the URL. Add one sentence to the prompt:
If you could not open this recording, say so and stop. Do not guess what it shows.If the app cannot fetch, or the recording is private, use the do-it-yourself bundle. ChatGPT accepts images you attach, paste or drag in, so a transcript plus three to six labeled frames covers almost every case.
What about Codex and other coding agents?
The agent setup covers Codex, Windsurf and OpenCode as well as Claude Code and Cursor. Each one gets the CLI, the skill and the MCP connection, and changes are limited to that agent's own configuration. After setup the handoff is the same everywhere: paste the link and say what you want done. Codex also accepts screenshots directly through --image or a paste, which is the fallback route.
Codex has a separate feature worth not confusing with this. Record & Replay lets you demonstrate a workflow on your Mac and turns it into a reusable skill. It is macOS only and needs Computer Use enabled. That is for teaching Codex a task you repeat. A one-off bug report or a piece of UI feedback is different: you want the agent to understand one recording and fix one thing, which is what a Clipy link is for. Clipy for Codex covers how the two fit together.
For other editors, see screen recording context for Windsurf Cascade and the same for Zed's agent.
How do you give an agent access to private recordings and your whole library?
Access to a recording's agent context follows the same rules as its watch page. A public recording is readable by anyone with the link, agents included, with no login. A private or restricted recording needs a signed-in connection from an account that is allowed to see it. Never make a recording public just so an agent can read it.
There are two ways to connect your account:
- Hosted connection. Add
https://clipy.online/api/mcpin your agent's connector settings and sign in to Clipy. Nothing is installed. It reads and searches your library, and if you grant write access it can also edit recordings. To record, use the local setup below or the Mac and Windows apps. - Local setup. The agent setup installs the Clipy CLI, the skill and the MCP server for one named agent. This adds recording, so the agent can capture proof of its own fix.
Once connected, the agent does not need a link. It can search everything you have recorded with tools such as search_memory and search_recordings, then read a match with get_summary, get_transcript and get_key_moments. The local server adds get_agent_context for the full document in one call, and list_context_documents and read_context_document cover YouTube videos and local files you have imported into Clipy as context documents. The connection is not read-only. With write access it can rename and organize recordings, change who can open them, and cut a recording with edit_recording. The local server can also correct a transcript with replace_transcript and start a recording. Clipy's tool instructions tell the agent to preview a cut and show you what it removes, and to confirm with you before making a private recording public.
A request then looks like this:
Find my recording about the checkout failure. Summarize the evidence,
link the relevant moments by timestamp, and do not change anything.The MCP reference lists every tool. For how other recorders' MCP servers compare, see MCP screen recorders compared.
Is there a prompt you can reuse?
This one works in any of the tools above. It asks the agent to show its understanding first, name what is missing, and bring back proof:
Read this recording: [Clipy link]
1. Before you change anything, tell me in a few lines what the
recording shows: the steps, what went wrong, and what I asked for.
Cite timestamps and look at the key-moment frames.
2. List anything you need that the recording does not show, such as
a missing step, a capture gap, or an error you could not read.
Do not guess to fill those in.
3. Make the change. Then verify it in the running app, record the
verification with Clipy, and attach the proof link to the PR.If the agent's summary in step 1 is wrong, you have caught the misunderstanding before it became a diff.
What if the agent cannot read the recording?
Work through these in order:
- Is processing finished? A recording that has just been uploaded may still be transcribing. The document says so in its metadata, and a Key moments section can read "still being extracted". Wait a minute and ask again.
- Is the recording private? An agent without your account connected cannot read it. Connect the account rather than changing the recording's sharing setting.
- Is the connection missing or signed in to another account? Ask the agent which Clipy account it is using.
- Can the client fetch at all? Some chat apps have no web access, or have it switched off by an administrator. Use the transcript and frames instead.
The troubleshooting guide covers setup and sign-in problems. In every case, the agent should tell you what it could not retrieve rather than claim to have seen it.
What does it cost?
Recording with Clipy needs an account. New accounts get a 7-day free trial, no card needed. After that, Pro is $9 a month or $72 a year; see pricing. People you share a link with do not need an account to watch it, and there is no watermark. Clipy does not use your recordings to train AI models, as stated on the trust page. The do-it-yourself route costs only your time.