Even AI experts are copying this homework: Building an efficient personal knowledge base using LLM Wiki

Byline: Biteye Core Contributor Shouyi

*The full text is about 2,300 Chinese characters, with an estimated reading time of 6 minutes

Feed AI materials every day, and it just turns around and forgets; Token gets burned like crazy, and the knowledge base ends up becoming a 「half-finished building」?

Former OpenAI co-founder / Tesla AI Director Andrej Karpathy @karpathy has just provided the ultimate solution. On April 3, he posted a tweet with over 17 million views and open-sourced the hard-core guide llm-wiki.

This guide, which has won 5,000+ Stars, proposes: use large models to build your personal knowledge base, and from then on say goodbye to 「carelessly burning Token」—so knowledge can「automatically generate returns」 just like digital assets.

Today, the editor will directly break down this hands-on tutorial that even big shots are using for you!

01 Why did you always fail when building a knowledge base before?

Before you start building, first understand the two most common failure modes, so you don’t repeat the same mistakes.

1. Traditional RAG (Retrieval-Augmented Generation)

The biggest pain point of this approach is that it burns Token but is also「forgetful」. When you throw it tens of thousands of words of crypto whitepapers or the latest AI papers, it struggles through them, summarizes for you in a compressed format. Then next week you ask it,「What’s the difference between that project from last week and this competitor today?」 It only remembers that dry little summary from back then. Because every call depends on fragmented retrieval, knowledge doesn’t form structured and lasting accumulation, and Token consumption is extremely high.

2. Traditional Wiki (Manual notes)

This approach is characterized by pure manual work: adding tags, building two-way links, making a table of contents… Karpathy hit the nail on the head:「The real root of why organizing knowledge is most annoying isn’t reading and thinking—it’s ‘bookkeeping’ (categorizing and formatting).」 Humans get tired, but AI is always on. Since all that dirty work used to be carried by humans alone, the outcome was naturally giving up.

02 Logic breakdown: LLM Wiki’s「fully automated pipeline」

The core of Karpathy’s solution is role substitution: you only need to act as the「content provider」，while all the messy and heavy lifting is handled by AI. This system consists of three logical layers:

First layer: Raw materials library (in only, never out)

The deep research reports, long tweets, AI tutorials, and podcast recordings you usually see—just drop them in. This is an absolute「single source of truth」. The large model is only allowed to read; it’s absolutely not allowed to change.

Second layer: Wiki core area (AI takes full control)

This area is made entirely of pure Markdown files. You don’t have to worry about formatting at all—AI will automatically extract the raw materials into「concept cards」 and「a roadmap/track and competitor comparison table」. You just read; AI is responsible for writing and updating.

Third layer: SOP rules (your house rules)

Write a CLAUDE.md or GPT.md configuration file to tell the AI our rules. For example:「For all crypto research reports, must extract tokenomics and team background」;「For all AI tutorials, must summarize 3 executable prompt code blocks」.

03 Hands-on tutorial: With the pipeline for turning「burning Token」into「asset appreciation」，how do you run it? The following three core actions directly make your knowledge base start generating returns automatically in one second:

Action 1: Automatic ingestion (Ingest)

Lobster hands-on: You toss in a 20k-word Web3 deep research report, leaving behind the line「Help me remember this」。

AI execution: It quickly reads everything in the background. It not only automatically generates Project A_investment-research notes.md, but also conveniently updates your global directory.md, and it will even proactively add this new project into the track competitor analysis.md that you wrote earlier. Read once, and everything on the network connects!

Action 2: Asking questions and「knowledge compounding」(Query)

Lobster hands-on: You casually ask:「Melt together the 5 articles I recently saved about large model Prompt techniques, and write a 小红书 viral copy.」 The AI instantly retrieves high-density highlights and helps you finish writing.

Knowledge compounding: Karpathy emphasizes that good questions and good answers absolutely must not be left to gather dust in the chat box! If you think the summary of this copy is great, just order the AI:「Save this summary back into the Wiki and create a new page called Prompt万能模板.md.」 It’s like「re-staking」 knowledge— the more you use it, the thicker it gets!

Action 3: Late-night deep clean (Lint)

Lobster hands-on: Give the instruction before bed:「Run a check on the knowledge base」.

AI execution: Like a robotic vacuum cleaner, it scans the whole system globally. Early the next morning, it reports to you:「Boss, the AI tool you saved last month is now charging fees. That conflicts with the logic of the ‘free freebie guide’ you saved yesterday—should I update it?」

04 Advanced setup: Obsidian + large model = the ultimate cheat code

When people used to deal with long-term memory, they always couldn’t avoid complex vector databases, but that’s too high a barrier for ordinary people. If local retrieval isn’t strong, the experience is extremely underwhelming. Karpathy’s recommended ultimate combo is: Obsidian (local note-taking software) + a large model.

Obsidian is like a code editor, and the large model is your outsourced programmer. By ditching complex databases, you only need two core files to slash Token consumption:

index.md (global outline): Records the summaries and links of all pages. Each time the AI answers a question, it first scans the outline, then precisely pulls the relevant notes—so you don’t have to reread hundreds of thousands of words every time. Token consumption reduced by 90%!

log.md (operation log): Records what the AI did each day and which files it modified in chronological order, making it easy for you to「check on it」 anytime.

Paired with Obsidian’s one-click web clipping and global knowledge star map, the knowledge base can also become visual.

05 Summary: Start your「knowledge-generating」era

In the information explosion of 2026, whoever can store knowledge with the lowest friction cost will be able to use the fewest Tokens to leverage the biggest upside.

Just like what Karpathy open-sourced this time isn’t a rigid codebase, but an「ideological file」 written for AI to read. You only need to feed his guide link to your dedicated Agent, and you can enter a win-without-trying mode.

Make your knowledge base move, make sure your Tokens never run out, and make sure your lobster isn’t an untrained “side-eye lobster” that can’t be raised well!