How to Automate Andrej Karpathy's LLM Wiki for Continuous Updates
Most knowledge bases fail because they require constant manual updates. Discover how to transform Karpathy's LLM Wiki into a self-updating system that automatically ingests content from YouTube, Slack, and other sources - saving hours of maintenance while keeping your knowledge perpetually fresh.
The Problem With Static Wikis
Andrej Karpathy's LLM Wiki concept revolutionized how we organize AI knowledge - but it has one critical flaw. Like all manual knowledge bases, it becomes outdated the moment you stop feeding it new information. At 2:15 in the video, we see how quickly valuable context decays when the Wiki isn't continuously updated.
The maintenance burden is real. Business teams report spending 3-5 hours weekly manually updating their knowledge bases, only to have them become obsolete between updates. This creates a paradox where the more comprehensive your Wiki becomes, the more maintenance it requires.
The breaking point: When manual updates stop (which they always do), your carefully curated Wiki becomes a museum of outdated information rather than a living knowledge base. This defeats the entire purpose of context-aware systems.
Context Farming: The Automated Solution
Context farmers solve this maintenance problem by automating the update process. These are specialized AI agents that connect to your content sources (YouTube, Slack, etc.) and continuously harvest new information on a schedule you define.
As shown at 4:30 in the demo, farmers operate like specialized cron jobs - they don't make judgment calls about what's important (that's your initial setup), but they handle all the repetitive fetching and ingestion work that normally falls to humans.
Key insight: Separating source selection (human curation) from content fetching (automated) means you maintain quality control while eliminating 90% of the manual work. You decide once what matters, then let farmers handle the ongoing updates.
YouTube Implementation Example
The video demonstrates a YouTube context farmer that automatically tracks favorite creators. At 6:45, we see it in action - discovering new videos, extracting transcripts and thumbnails, and updating creator pages without manual intervention.
Here's how it works step-by-step:
Step 1: Source Configuration
You provide the YouTube channels you want to track (your "source of truth"). This one-time setup establishes your curation boundaries.
Step 2: Farmer Scheduling
The farmer runs daily (or hourly) checks using YouTube's API through an MCP connector. At 7:20, we see it finding two new videos since its last run.
Step 3: Automatic Ingestion
New content gets added to your Wiki's raw folder. The LLM then processes it into structured pages, updating relationships in the index file (shown at 8:10).
Compounding benefit: Each update makes the Wiki richer. Follow-up videos automatically link to previous discussions on the same topic, creating an interconnected knowledge graph that grows smarter over time.
System Architecture Overview
The automated Wiki system consists of three core components working together:
1. Context Farmers: Specialized agents for each source (YouTube, Slack, etc.) that fetch new content on schedule. These run either locally or in the cloud via Claude Code scheduled agents.
2. Processing Layer: The LLM that transforms raw content into structured Wiki pages, maintaining the index and relationship graphs. This uses Karpathy's original markdown-based approach.
3. Storage & Sync: A GitHub repository that serves as the cloud-hosted version of your Wiki, automatically syncing to local Obsidian instances (shown at 10:15).
Cloud advantage: By hosting the Wiki in GitHub, you enable team-wide access and ensure updates continue even when your local machine is off. The demo at 11:30 shows how cloud-scheduled farmers operate independently of your devices.
Maintaining Quality Control
A common concern is quality - won't automated ingestion lead to information overload? The system addresses this through careful architecture:
Human Gatekeeping: You manually select trusted sources once (specific YouTube channels, Slack workspaces, etc.). Farmers only fetch from these pre-approved sources.
Structured Processing: The LLM doesn't just dump raw content - it creates organized pages with clear relationships, as seen in the Obsidian graph view at 9:45.
Update Logging: Every change gets recorded in the Wiki's log file, creating an audit trail you can review (demonstrated at 12:10).
Quality vs. Quantity: The system gives you both - human-curated quality at machine-scale quantity. You maintain control over what enters the system while eliminating the grunt work of keeping it updated.
Multi-Source Integration
While the demo focuses on YouTube, the pattern works for any source with an MCP connector. At 13:20, we see examples of other powerful integrations:
Business Context: Farmers on Slack channels and Fireflies meeting transcripts automatically update team Wikis with decisions, customer updates, and project status.
Research Tracking: Farmers that scrape academic repositories can alert you when new papers contradict existing Wiki entries.
Competitive Intelligence: Automated tracking of competitor blogs, job postings, and social media creates a living competitive analysis.
Implementation tip: Start with one high-value source (like your team's Slack), then expand to others once you've validated the workflow. The system is modular - each new farmer operates independently.
Local vs Cloud Deployment
You have two deployment options, each with advantages:
Local Execution: Farmers run on your machine via Claude Code desktop. Simple to set up but requires your device to be on (shown at 14:50).
Cloud Scheduling: Farmers run on Anthropic's infrastructure via Claude Code scheduled agents. More reliable for always-on operation (demonstrated at 15:30).
The video shows both approaches, with the cloud method being preferable for business-critical Wikis that need to update 24/7. Local execution works well for personal knowledge bases where occasional gaps are acceptable.
Pro tip: Use cloud scheduling for core business sources (Slack, meetings) and local execution for nice-to-have tracking (YouTube, blogs). This balances reliability with simplicity.
Watch the Full Tutorial
At 7:20 in the video, you'll see the context farmer in action as it discovers and processes two new YouTube videos automatically. This real-time demonstration shows exactly how the system updates Wiki pages without manual intervention.
Key Takeaways
Automating Karpathy's LLM Wiki transforms it from a static documentation tool into a living knowledge base that grows smarter daily without manual effort. The system delivers three transformative benefits:
1. Time Savings: Eliminates 90% of the manual work required to maintain current knowledge bases.
2. Better Insights: Compounding updates create richer context layers that reveal patterns invisible in static systems.
3. Team Alignment: Cloud-hosted Wikis ensure everyone accesses the same current knowledge.
In summary: Context farmers turn knowledge management from a maintenance chore into a strategic advantage. By automating updates, you free human attention for higher-value work while ensuring your team always has current context.
Frequently Asked Questions
Common questions about this topic
The original LLM Wiki requires manual updates - whenever you stop feeding it new context, it becomes outdated. This creates a maintenance burden where valuable knowledge bases become useless without constant human intervention.
Teams typically spend 3-5 hours weekly manually updating their Wikis, yet the information still decays between updates. The automated approach solves this by handling all ongoing updates automatically after initial setup.
- Manual Wikis require continuous human maintenance
- Information decays rapidly between updates
- Creates unsustainable maintenance overhead
Context farmers are AI agents that connect to sources through MCP connectors (like YouTube, Slack, or Notion). They run on a schedule you define (daily, hourly, etc.), fetch new content automatically, and ingest it into the Wiki's raw folder where the LLM processes it into structured knowledge.
Each farmer specializes in one source type and operates independently. For example, your YouTube farmer might run daily at 6am, while your Slack farmer runs hourly. This modular approach ensures updates happen at the optimal frequency for each information source.
- Specialized agents per content source
- Customizable scheduling per farmer
- Automatic ingestion into Wiki structure
Any platform with an MCP connector can be automated - YouTube channels, Slack workspaces, Fireflies meeting transcripts, Notion databases, competitor blogs, or academic paper repositories. The system currently supports hundreds of integrations.
Common implementations include tracking industry-specific YouTube creators, monitoring internal Slack discussions, ingesting meeting notes from Fireflies, and tracking competitor activity across their digital properties. The only limit is the availability of MCP connectors for your desired sources.
- YouTube channels and playlists
- Slack workspaces and channels
- Meeting transcripts from Fireflies
- Competitor blogs and social media
The system separates source selection (human curation) from content fetching (automated). You manually choose trusted sources once (like specific YouTube channels), then the farmer handles the repetitive fetching. This maintains quality while removing manual work.
Quality is further ensured through structured processing - the LLM doesn't just dump raw content but organizes it into consistent Wiki pages with clear relationships. Every update is logged, creating an audit trail you can review to verify information quality.
- Human selects trusted sources initially
- Structured processing maintains organization
- Complete audit trail of all updates
Yes, by hosting the Wiki in a private GitHub repository. The system auto-commits updates, making them available to anyone with repository access. This creates a shared knowledge base that updates automatically for entire teams.
Team members can sync the Wiki to their local Obsidian instances or access it directly through Claude Code. The GitHub repository serves as the single source of truth, ensuring everyone works from the same current knowledge base regardless of how they access it.
- GitHub hosts the centralized Wiki
- Auto-commits ensure continuous updates
- Accessible through multiple interfaces
You set custom schedules per source - YouTube channels might update daily, Slack channels hourly, meeting transcripts weekly. Farmers run independently on their own schedules, continuously enriching the Wiki without manual intervention.
The scheduling flexibility allows you to match update frequency to information velocity. Fast-moving sources like Slack can update frequently, while slower sources like YouTube might only need daily checks. Farmers operate independently, so each runs on its optimal schedule.
- Custom schedules per content source
- Matches update frequency to information velocity
- Independent operation per farmer
Local farmers require your computer to be on, while cloud farmers (via Claude Code scheduled agents) run 24/7 on Anthropic's infrastructure. Cloud execution ensures updates continue even when your devices are off.
Local execution is simpler to set up and works well for personal knowledge bases. Cloud scheduling is better for business-critical Wikis that must update reliably. Many implementations use both - cloud for essential sources and local for nice-to-have tracking.
- Local: Simple but device-dependent
- Cloud: Reliable 24/7 operation
- Hybrid approach often works best
GrowwStacks specializes in deploying automated knowledge management systems. We can design custom context farmers for your specific sources, configure the Wiki structure for your needs, and handle the technical implementation so you get a turnkey self-updating knowledge base.
Our implementation process includes identifying your highest-value content sources, designing the optimal Wiki structure, configuring all automation components, and training your team on maintenance. We handle the technical complexity so you can focus on leveraging the knowledge.
- Custom farmer design for your sources
- Tailored Wiki structure for your needs
- Complete technical implementation
- Ongoing support and optimization
Ready to Transform Your Static Wiki Into a Self-Updating Knowledge Base?
Manual Wiki maintenance steals time from strategic work while leaving your team with outdated information. Our automation specialists will design and deploy a custom self-updating Wiki system tailored to your content sources and business needs.