How to Build a Personal Knowledge Base From the YouTube Videos You Watch

This is not about building a customer support knowledge base with videos embedded in it. It is about the other thing that phrase means: turning the YouTube videos you personally watch — conference talks, tutorials, walkthroughs, the twelve-minute clip that finally explained the thing — into a private, searchable library that both you and your AI agent can read later. Search results for the phrase are split roughly down the middle between those two meanings, so let's be explicit about which one this is.

TL;DR

• Bookmarking a video is not saving it. You are saving a pointer to two hours you will never re-watch.
• Transcript-only capture — which is what most tools and NotebookLM do — keeps the commentary and throws away the picture. For tutorials and walkthroughs that is the wrong half.
• A useful personal video knowledge base needs four things: capture that survives the visual content, automatic classification, full-text search across everything, and machine access so an agent can read it mid-task.
• Clipy does this from a single command: npx -y @clipy/cli@latest context import "<url>" --sync.
• Each import is classified into one of seven kinds (tutorial, product demo, bug report, discussion, and so on) and the classifier decides whether the words stand alone or specific timestamps need a picture.
• Everything you import is indexed by title and full transcript, so cross-video search is one query rather than an afternoon of scrubbing.
• Agents read the library over MCP — orient cheaply first, then pull only the span they need.
• None of this replaces NotebookLM for documents. It replaces the bookmark folder.

Two things called "knowledge base"

Search for "build knowledge base from youtube videos" and you get two completely different articles wearing the same title.

The first is B2B: you run a support desk, you have help articles, and you want to embed videos in them or generate articles from your product's YouTube channel. That is a documentation product problem, and tools like Docsie and CustomGPT are built for it.

The second is personal, and it is the one nobody has solved well: you watch a lot of video, almost none of it sticks, and six weeks later you remember that somebody explained the thing but not who or where. The video is in your watch history, or a bookmark folder, or a "Watch Later" playlist with four hundred items in it. Functionally, it is gone.

This article is about the second one. If you are here for the support-desk version, none of the following will help you.

Why the bookmark folder fails, precisely

It is worth being specific about the failure, because the specifics dictate the fix.

A bookmark stores a location, not content. You cannot search it. "Which video mentioned the retry backoff" is unanswerable against a list of titles, because titles are marketing and the substance is forty minutes in.

Re-watching does not scale. Even if you correctly guess which of five candidate videos holds the answer, finding the moment inside a fifty-minute talk means scrubbing. Chapter markers help when creators bother, which is sometimes.

It rots. Videos get unlisted, deleted, or region-locked. A pointer to someone else's content is a pointer that can be revoked, and you find out at the worst moment.

Nothing else can read it. This is the one that changed recently. If you work with a coding agent, everything else in your working life is machine-readable — your repo, your docs, your issues, your notes. Your video is not. The knowledge you specifically acquired by watching something is the one category your agent cannot reach, which means you become the manual bridge, re-explaining from memory what you half-remember.

The part most tools get wrong: the visual half

Nearly every "save YouTube to notes" tool works the same way: pull the caption track, summarize it, write it somewhere. Snipo, Tube2Note, Glasp, Eightify, the Obsidian plugins, and NotebookLM all land in that family. Google's help centre states the behaviour for its own product in one flat sentence — "Only the text transcript of the video is imported as a source" (Gemini Notebook Help, the product formerly and still widely called NotebookLM).

For a panel discussion or an interview, that loses almost nothing. The substance genuinely is the words.

For a tutorial, it loses the point. Watch any competent screen-recorded walkthrough with the audio muted and you will see how much information is carried by the picture: the file path in the sidebar, the flag in the command, the exact error text, the version in the corner, the shape of the config. Now do the inverse — read the transcript with the video closed — and you get sentences like "and then we just add the middleware here." Here is doing enormous work in that sentence, and the transcript does not contain here.

We ran a lot of developer-oriented video through both paths while building this, and the pattern was consistent enough to design around: the more useful a video is to somebody trying to do the thing, the more of its information is dictated by the screen rather than by the speaker. That is our testing on the videos we chose, not a published benchmark, and it is easy enough to check against your own sources — mute one and see what survives.

So the first requirement for a video knowledge base worth building is that capture has to keep the visual evidence, at least where the visual evidence is load-bearing.

The workflow, end to end

Here is the actual shape of it. The commands are real and current as of @clipy/cli 0.9.3.

1. Import a video

One command, no signup needed to see what it produces:

npx -y @clipy/cli@latest context import "https://www.youtube.com/watch?v=VIDEO_ID" --json

This compiles the video into a local context bundle — a directory holding recording.md (the agent-facing document, opening with a self-describing header and then timestamped transcript sections), manifest.json (provenance, tool versions, and a sufficiency report saying how well the transcript stands on its own), and transcript.json.

Captions resolve on your machine through a Clipy-managed yt-dlp that installs into ~/.clipy/bin on first use, with a disclosure when it does. Creator-supplied captions are preferred over auto-generated ones. On this path, no media is downloaded at all — it is a caption fetch, not a video download.

Re-running the same import over the same source is idempotent, which is what makes it safe to retry when something goes sideways halfway through.

2. Sync it into the library

A bundle on your disk is useful for one task. A library is useful for a year. Add --sync:

npx -y @clipy/cli@latest context import "https://www.youtube.com/watch?v=VIDEO_ID" --sync

Two things happen that do not happen locally. First, the derived bundle — the document, never the source media — goes into your private library. Second, the server classifies it, and that classification is what makes the library navigable later.

3. Automatic classification

Every imported video is sorted into exactly one of seven kinds:

• walkthrough_tutorial — somebody teaching you to do a thing, step by step
• product_demo — showing off a product or feature to an audience
• bug_report — a reproduction of something broken
• feature_request — describing something that should exist
• feedback_review — reacting to and critiquing something on screen
• discussion_talk — people talking, nothing visual being referenced
• other

The kind is not decoration. It changes how the video is indexed: a walkthrough_tutorial gets a moment marked per step or chapter boundary, evenly covering the flow, while a discussion_talk usually gets none — because there is nothing on screen worth a picture, and inventing moments there would just add noise.

Alongside the kind, the classifier answers a separate question: does this video actually need its visuals? A talking-head podcast, or a conference talk whose slides are decorative, is a no — the transcript is complete and nothing more is needed. A screen recording where the answer is in a config file is a yes, and the classifier names the specific timestamps.

4. Frames, only where they earn it

When the verdict is that the words are not enough, the CLI cuts exactly those timestamps into images locally with ffmpeg and attaches them to the document with captions describing what is on screen. It is a targeted fetch of the moments that were named, not a bulk screenshot pass.

The trade to be honest about: cutting frames requires the media at that point, so this step does download it. If you would rather never fetch media, --no-frames takes the classification verdict and keeps the document transcript-only. You still learn where the transcript is blind; you just do not get the pictures.

If frame extraction fails — no ffmpeg installed, a download that gets refused — the import is not lost. The document is already in your library with its transcript and classification, and re-running the same command later completes it.

5. File it

Tags and folders are set at import time, so the organizing happens once, at the moment you still remember why you saved the thing:

npx -y @clipy/cli@latest context import "https://www.youtube.com/watch?v=VIDEO_ID" \
  --sync --folder "Rust" --tag async --tag tokio --title "Pinning, properly explained"

--tag repeats. --title overrides whatever SEO-shaped title the creator gave it, which in practice is most of them. --language sets caption language preference, and --output controls where the local bundle lands.

6. Local files, not just YouTube

The recording sitting on your disk that was never uploaded anywhere counts too. Local files need a caption file alongside them:

npx -y @clipy/cli@latest context import ./architecture-review.mp4 \
  --transcript ./architecture-review.vtt --sync --folder "Internal"

It accepts .vtt, .srt, or a Clipy transcript JSON, and probes the file with ffprobe for duration and dimensions. This is the path for material that could never go into a hosted tool that only accepts public YouTube URLs — the internal walkthrough, the recorded design review, the customer call.

7. Search across everything

Every synced document is indexed on its title and its entire transcript, so the question you actually have — "which of these mentioned the webhook retry policy" — is a query across the whole library rather than a guess followed by scrubbing.

This is search over what you chose to keep. It is deliberately not search over the public YouTube corpus; tools like VideoSherlock do that, and it is a different job. A personal knowledge base is valuable precisely because it is small and curated — everything in it is there because you decided it mattered.

8. Let the agent read it

This is the step that makes the whole thing worth the effort rather than a tidier bookmark folder. Install the skill once:

npx -y @clipy/cli@latest agents install claude

codex and cursor work the same way, and clipy agents uninstall <target> reverses it.

After that, your agent can reach the library mid-task without you pasting anything. Over MCP it orients cheaply before it commits context: list_context_documents to see what is there, get_context_document for one document's metadata and classification — what kind of video it is, whether it needs visuals, which moments were planned — without pulling the transcript, and then read_context_document with a start and end timestamp for just the span it needs.

That last part is not a nicety. Dumping a two-hour talk's full transcript into a context window is the difference between an answer and a truncation, and an agent that can ask for minutes 43 through 51 does not have to make that trade.

The practical shape of this is a conversation like: "we're setting up the webhook handler — check my library, I watched a talk about the retry semantics." The agent searches, finds the document, reads the relevant span, and writes code informed by it. You did not re-watch anything and you did not half-remember anything.

Reading the sufficiency report, and why it exists

Every bundle carries a report in manifest.json that answers one question: how much of this video does the document actually contain? It names the timestamps where the transcript goes blind — the moments where the speaker referred to something on screen that the words do not describe.

This sounds like a small feature and it is the one worth understanding, because it is the difference between a document that lies to you and one that does not. Every transcript-only tool produces output that reads complete. There is no visual signal in a wall of well-formed prose that says "a quarter of the information in this video is missing." The reader — you, or worse, an agent that will act on it — has no way to know it is working from a partial record.

An agent reading a Clipy document sees the gaps declared up front, in the header, before the transcript starts. That changes its behaviour in the way you want: it can say "the transcript references a config file at 12:40 but does not show it" instead of confidently inventing what was probably in the file. Hallucination in this setting is usually not a model failing so much as a document that presented itself as complete when it was not.

The same header carries an untrusted-content warning, which matters more than it sounds. A video transcript is text written by a stranger, and an agent that reads it should treat instructions inside it as data rather than as instructions. Marking that boundary explicitly is cheap and the alternative is unpleasant.

Curation is the part that decides whether this works

A knowledge base that grows without pruning becomes the bookmark folder again, just with better search. Two habits keep it useful.

Import at the end of watching, not later. The tag and folder you would choose right now are the ones your future self will search for; the ones you would choose in three weeks are guesses. This is why the CLI takes --tag and --folder at import time rather than asking you to file things afterwards — the afternoon you set aside for filing never arrives.

Import less than you watch. The instinct is to capture everything, and it is wrong. Signal-to-noise in search is the whole value; a library of forty things you deliberately kept answers questions better than a library of four hundred you passively hoovered up, because in the second one every query returns twelve results and you are back to guessing. The filter to apply is simple: would I plausibly go back to this? Most videos, honestly, no.

The corollary is that this is not a "save everything automatically" system and is not trying to be. It is closer to a reference shelf than a hard drive.

How this compares to the manual alternatives

Bookmarks and Watch Later

Free, instant, and they do not work, for the four reasons at the top of this article. Worth saying plainly because most people's actual system is this one, and the honest comparison is not against some other tool — it is against doing nothing.

Clipping to Notion or Obsidian

The Snipo / Tube2Note / NotionSync family, and on the Obsidian side a set of transcript-fetcher plugins. These genuinely work, and if you already live in one of those apps the friction is close to zero.

Two structural limits. The output is a note — the endpoint is a page in your vault, and nothing indexes across what you saved beyond that app's own text search, which does not know a transcript from a grocery list. And they are transcript-shaped, so the visual half is gone (Snipo's manual screenshots being the partial exception, and manual is the operative word). Obsidian's version has one real advantage worth naming: it is plain files on your disk, so an agent with filesystem access can grep it. That is a low ceiling but it is a real floor.

NotebookLM / Gemini Notebook

The strongest of the alternatives and, for many people, the right answer. It is free, it needs no setup, the chat interface with citations back to sources is genuinely good, and Audio Overviews are a real feature nobody else matches.

Where it stops being the right tool for this job comes down to documented behaviour rather than opinion. Its help centre records that only public videos with existing captions can be imported, that videos without speech are unsupported, that only the text transcript comes in, and that a source is auto-deleted within 30 days if the video is deleted or made private (source). Google's FAQ documents the allowance as 100 notebooks with up to 50 sources each (source).

Put together, that describes a notebook: a bounded, curated project you build for a purpose. It is a deliberate and reasonable design. It is just not an unbounded personal memory of everything you have watched, and the 30-day rule means what is in it is not entirely yours. If you want the full breakdown of those limits we wrote one up in NotebookLM and YouTube: every documented limit.

AI summarizer extensions

Eightify, Glasp and the rest give you a summary in the sidebar while you watch. Useful for triage — should I watch this at all — and close to useless as memory, because a summary is lossy by construction and the thing you will need in six weeks is the detail the summary dropped.

What this does not do

It is a CLI. There is a terminal involved, and that is a real cost compared to a browser extension you click. If you want a passive, zero-effort habit where saving is one button, this is not that yet.

It does not generate Audio Overviews, study guides, or slide decks. It does not chat with your documents in a notebook UI. It does not search the public YouTube corpus. And it is not the first tool to expose a video library to an agent over MCP — Recall ships one too, over its saved summary cards. The distinction worth drawing is CLI-first ingest plus timestamped, span-addressable retrieval with visual evidence attached, aimed at a developer working with an agent mid-task. If that is not your workflow, the distinction will not matter to you.

Starting in about ten minutes

Pick three videos you actually reference — the ones you have gone back to more than once. Import them:

npx -y @clipy/cli@latest context import "<url>" --sync --folder "Reference"

Read one back the way an agent would, so you can see what the document looks like before deciding whether to trust it:

npx -y @clipy/cli@latest context read ./clipy-context-<hash>

Then install the skill and ask your agent something you would previously have re-watched a video to answer:

npx -y @clipy/cli@latest agents install claude

The habit that makes a knowledge base real is importing at the moment you finish watching, while you still know why it mattered. Everything else is retrieval, and retrieval is the part the machine is good at.

See how the YouTube-to-agent-context flow works, or read the longer argument for why agents need video context and not just transcripts.