[{"content":" Designed and built a distributed system that detects unbilled usage across all AWS services — reducing charge discrepancies by 300x and eliminating 230 million monthly false positives. Key Metrics # 300x Reduction in Discrepancies $125,000 → $432 230M False Positives Eliminated ~95% Alert Actionability Architecture # flowchart LR A[\"Usage Records\\n(Billions/day)\"] --\u003e B[\"Smart Sampling\\n\u0026 Aggregation\"] B --\u003e C[\"Multi-Signal\\nValidation\"] C --\u003e D[\"Automated\\nResolution\"] D --\u003e E{Real issue?} E -- Yes --\u003e F[\"Alert with\\nDiagnosis\"] E -- No --\u003e G[\"Auto-resolve\\n\u0026 Log\"] style B fill:#6366f1,color:#fff style C fill:#6366f1,color:#fff style D fill:#6366f1,color:#fff Technical Deep Dive # Aggregation over Brute-Force # Instead of checking every individual usage record (which generated 230M false positives), the system aggregates at the service-account-period level.\nBuilt on DynamoDB for consistent low-latency reads at any scale Each record stores expected charge, actual charge, pricing plan, and discount metadata Reduced comparison space by orders of magnitude while preserving detection capability Beyond Simple Mismatch Detection # A single charge mismatch doesn\u0026rsquo;t indicate a problem. The validation pipeline (built on AWS Lambda) checks multiple signals:\nTemporal correlation — Is this a timing issue that self-corrects? Pricing context — Did a pricing change or discount explain the difference? Historical pattern — Has this account shown similar patterns before? Magnitude thresholds — Is the discrepancy large enough to investigate? Only records failing all validation checks are escalated.\nFrom Symptom to Diagnosis # Common discrepancy patterns trigger automated remediation:\nRe-processing dropped usage records Applying missing discounts retroactively Flagging records for manual review with specific context about what went wrong Engineers receive alerts with a diagnosis, not just a symptom.\nImpact # Tech Stack # Java DynamoDB AWS Lambda Distributed Systems Billing Pipeline Read the Full Story ","externalUrl":null,"permalink":"/projects/aws-billing-auditor/","section":"Projects","summary":" Designed and built a distributed system that detects unbilled usage across all AWS services — reducing charge discrepancies by 300x and eliminating 230 million monthly false positives. Key Metrics # 300x Reduction in Discrepancies $125,000 → $432 230M False Positives Eliminated ~95% Alert Actionability Architecture # flowchart LR A[\"Usage Records\\n(Billions/day)\"] --\u003e B[\"Smart Sampling\\n\u0026 Aggregation\"] B --\u003e C[\"Multi-Signal\\nValidation\"] C --\u003e D[\"Automated\\nResolution\"] D --\u003e E{Real issue?} E -- Yes --\u003e F[\"Alert with\\nDiagnosis\"] E -- No --\u003e G[\"Auto-resolve\\n\u0026 Log\"] style B fill:#6366f1,color:#fff style C fill:#6366f1,color:#fff style D fill:#6366f1,color:#fff Technical Deep Dive # Aggregation over Brute-Force # Instead of checking every individual usage record (which generated 230M false positives), the system aggregates at the service-account-period level.\n","title":"AWS Billing Unbilled Usage Auditor","type":"projects"},{"content":" Built the automation and validation tooling to manage kernel updates across 5,000+ production servers at Twitter — with zero-downtime progressive rollouts and automated canary validation. Key Metrics # 5,000+ Production Hosts Weeks → Days Rollout Time 140+ Tickets in One On-Call Week Zero-Downtime Updates Architecture # flowchart LR A[\"New Kernel\\nVersion\"] --\u003e B[\"Canary\\nValidation\"] B --\u003e C[\"Wave 1\\n1% Fleet\"] C --\u003e D[\"Wave 2\\n5% Fleet\"] D --\u003e E[\"Wave 3\\n25% Fleet\"] E --\u003e F[\"Full Fleet\\nRollout\"] C -- anomaly --\u003e G[\"Pause \u0026\\nAuto-Alert\"] D -- anomaly --\u003e G E -- anomaly --\u003e G style B fill:#6366f1,color:#fff style C fill:#6366f1,color:#fff style D fill:#6366f1,color:#fff style E fill:#6366f1,color:#fff style F fill:#6366f1,color:#fff Technical Deep Dive # Validate Before You Roll # A Python library that validates kernel safety before fleet-wide rollout:\nProvisions canary hosts from each hardware/workload combination Applies kernel update and reboots canary hosts Runs validation suites — system stability, performance benchmarks, application health Compares baselines — CPU, memory, I/O latency, network throughput vs. production Only after passing canary validation on every host type is a kernel approved for rollout.\nWave-Based Deployment # Automated deployment in progressive waves with anomaly detection between each:\nWave Coverage Purpose Wave 1 1% Smoke test in production Wave 2 5% Expand to more host types Wave 3 25% Majority coverage Wave 4 100% Full fleet completion Between each wave: automated monitoring for unexpected reboots, performance regression, and application errors. Any signal crossing a threshold pauses the rollout automatically.\nEliminating Drift # Configuration drift was the root cause of most fleet management pain. Built tooling to:\nAudit every host against its declared state Detect drift from intended configuration Auto-remediate safe divergences, flag risky ones for human review This was a prerequisite for safe automation — you can\u0026rsquo;t automate kernel updates on hosts whose configuration you don\u0026rsquo;t fully understand.\nRuntime Cache Control # Custom commands system for Twitter\u0026rsquo;s Redis-based cache services using Go:\nInspect and modify cache behavior at runtime Debug production issues without service restarts Zero customer impact during investigation Impact # Tech Stack # Python Go Redis Linux Kernel Fleet Management Bare Metal Read the Full Story ","externalUrl":null,"permalink":"/projects/twitter-fleet-automation/","section":"Projects","summary":" Built the automation and validation tooling to manage kernel updates across 5,000+ production servers at Twitter — with zero-downtime progressive rollouts and automated canary validation. Key Metrics # 5,000+ Production Hosts Weeks → Days Rollout Time 140+ Tickets in One On-Call Week Zero-Downtime Updates Architecture # flowchart LR A[\"New Kernel\\nVersion\"] --\u003e B[\"Canary\\nValidation\"] B --\u003e C[\"Wave 1\\n1% Fleet\"] C --\u003e D[\"Wave 2\\n5% Fleet\"] D --\u003e E[\"Wave 3\\n25% Fleet\"] E --\u003e F[\"Full Fleet\\nRollout\"] C -- anomaly --\u003e G[\"Pause \u0026\\nAuto-Alert\"] D -- anomaly --\u003e G E -- anomaly --\u003e G style B fill:#6366f1,color:#fff style C fill:#6366f1,color:#fff style D fill:#6366f1,color:#fff style E fill:#6366f1,color:#fff style F fill:#6366f1,color:#fff Technical Deep Dive # Validate Before You Roll # A Python library that validates kernel safety before fleet-wide rollout:\n","title":"Twitter Fleet-Scale Kernel Automation","type":"projects"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/ai/","section":"Tags","summary":"","title":"Ai","type":"tags"},{"content":" Obsidian + Claude Code is everywhere right now. But pointing an AI at a folder of markdown files and hoping for the best doesn\u0026rsquo;t work. What matters is how you structure the knowledge base. Get that right, and Claude becomes genuinely useful. Get it wrong, and you get confident garbage. There\u0026rsquo;s been a wave of posts about this combo lately: James Bedford\u0026rsquo;s full walkthrough, Greg Isenberg\u0026rsquo;s \u0026ldquo;personal OS\u0026rdquo; approach, kepano (Obsidian\u0026rsquo;s CEO) sharing Claude Skills. They\u0026rsquo;re all worth reading.\nWhat most of these guides cover is the setup. Install Claude Code, point it at your vault, go. This post is about the structure that makes the whole thing actually work.\nWhy Structure Matters More Than Tools # You can run cd ~/my-vault \u0026amp;\u0026amp; claude right now. Claude will read your files, answer questions about them, and even create new notes. The problem is quality.\nIf your vault is a flat folder of 200 files with inconsistent naming, no metadata, and no conventions, Claude will produce notes that match: inconsistent, hard to find, disconnected from everything else.\nThe AI inherits your organizational discipline. Or your lack of it.\nJames Bedford made an interesting observation in his walkthrough. He initially put everything in Obsidian, including content Claude generated, but found it diluted his knowledge graph. He now separates notes he personally wrote from AI-generated content.\nI went the opposite direction. Claude writes directly into my vault, alongside my own notes, using the same templates and conventions. The key difference: I invested upfront in structure that constrains Claude\u0026rsquo;s output to match what I\u0026rsquo;d write myself.\nMy Setup at a Glance # I use Obsidian as my primary knowledge base for study notes and technical reference material. The vault has 170+ markdown notes organized into modules and topics, version-controlled with git, and synced to GitHub. Claude Code has 9 custom skills for creating and maintaining content.\nThe structure follows a simple hierarchy:\nVault/ ├── .claude/skills/ # 9 Claude Code skills ├── CLAUDE.md # AI assistant rulebook ├── Topic-Area/ │ ├── Module-01/ │ │ ├── 00-Overview.md │ │ ├── 01-First-Topic.md │ │ ├── 02-Second-Topic.md │ │ └── 03-Third-Topic.md │ ├── Module-02/ │ │ └── ... │ └── Module-03/ │ └── ... ├── Dashboards/ └── Templates/ Nothing fancy. But every part of it is deliberate, and that deliberateness is what makes Claude effective.\nThe Principles That Make It Work # Consistent Naming and Numbering # Every file follows the same pattern: kebab-case English name, number prefix for ordering.\nModule-03/ ├── 00-Overview.md ├── 01-Introduction.md ├── 02-Core-Concepts.md ├── 03-Capabilities.md ├── 04-Advanced-Topics.md ├── 05-Training-Methods.md ├── 06-Optimization.md ├── 07-Activation-Functions.md ├── 08-Practical-Applications.md └── 09-Common-Pitfalls.md When I tell Claude to create a new note in a module, it can look at existing files and figure out the next number automatically. No ambiguity, no conflicts. The naming pattern is predictable enough that Claude doesn\u0026rsquo;t need to be told where to put things.\nRich Frontmatter as a Contract # Every note includes YAML frontmatter with metadata:\n--- title: \u0026#34;Topic Title\u0026#34; module: Module-03 tags: - core-concept - intermediate content_type: concept difficulty: beginner has_practice_questions: true has_diagrams: true estimated_reading_time: \u0026#34;15 min\u0026#34; status: complete created: 2025-12-15 updated: 2026-01-20 --- This isn\u0026rsquo;t just metadata for Obsidian\u0026rsquo;s Dataview plugin. It\u0026rsquo;s a contract between me and Claude about what each note is. When Claude creates a new note, it fills in all these fields because the pattern is documented in CLAUDE.md. When Claude searches the vault, it can filter by difficulty, content type, or status without reading every file.\nThink of frontmatter as a schema. The more consistent your schema, the more reliably AI can work with your data.\nTemplates That Enforce Structure # I have two templates. An overview template for entry points and a note template for individual topics:\n# Topic Title \u0026gt; One-line summary of this note. --- ## Learning Objectives - Objective 1 - Objective 2 --- ## Main Content Your content here... --- ## Related Links - Previous: [[Previous-Note]] - Next: [[Next-Note]] --- ## Practice Questions \u0026gt; Question here? \u0026gt; [!success]- Click for answer \u0026gt; Answer here. The template does two things. It gives me consistent notes that are easy to navigate. And it gives Claude a pattern to follow without explicit instruction. After seeing 50 notes in this format, Claude will produce new notes that look the same. Templates are free structure.\nCLAUDE.md as the Rulebook # The vault has its own CLAUDE.md file at the root. It covers:\nThe full directory structure File naming conventions (English, kebab-case, number prefixes) Editing guidelines (preserve wiki links, use LaTeX for math, support tags) How to add new modules (step-by-step) A note template for reference Which Obsidian plugins are in use A table of all 9 Claude skills and when to use each If you do one thing from this post, write a CLAUDE.md for your vault. It doesn\u0026rsquo;t need to be long. Document your folder structure, your naming conventions, and your note template. That alone will transform how Claude interacts with your knowledge base. Skills for Repeatable Workflows # Claude Code supports custom skills: markdown files in .claude/skills/ that define reusable workflows. I have 9 of them. A few examples:\nConvert Transcript takes raw lecture or reference material, identifies main topics, splits them into separate notes, adds examples and practice questions, and links everything together with wiki links. One transcript becomes a full module with 8-10 interconnected notes.\nVault Health Check audits the vault for broken internal links, missing tags, inconsistent naming, orphan files, and empty folders. I run it before pushing to GitHub. It catches the kind of drift that makes a vault gradually less useful over time.\nSearch and Organize helps me find notes across modules and reorganize them when the structure evolves. Useful when I realize a concept belongs in a different module than where I originally put it.\nThe idea is the same as Boris Cherny\u0026rsquo;s slash commands for Claude Code (covered in my previous post): anything you do more than twice should be automated.\nWhat Claude Code Can Do in a Vault # Here\u0026rsquo;s the concrete workflow. A typical session looks like:\nI paste raw material (a transcript, an article, or rough notes) into Claude Code The \u0026ldquo;Convert Transcript\u0026rdquo; skill breaks it into structured notes with frontmatter, examples, and practice questions I review and edit. Claude handles the scaffolding, I handle the understanding The \u0026ldquo;Health Check\u0026rdquo; skill verifies all links resolve and all notes have proper metadata The \u0026ldquo;Git Push\u0026rdquo; skill commits and pushes to GitHub Other things Claude handles regularly:\nGenerating study materials from existing notes (summaries, practice sets) Reorganizing notes when I restructure a module Filling in missing metadata across batches of notes Creating overview files that link to all notes in a module What Doesn\u0026rsquo;t Work Yet # Honest assessment of the limitations:\nObsidian\u0026rsquo;s graph view is invisible to Claude. Claude reads files and follows wiki links. But it doesn\u0026rsquo;t understand the emergent relationships that Obsidian\u0026rsquo;s graph view reveals. It can\u0026rsquo;t say \u0026ldquo;these two modules are highly connected\u0026rdquo; unless you tell it.\nLarge vaults can exceed context windows. With 170 notes, Claude can\u0026rsquo;t hold the entire vault in context at once. It works file-by-file or module-by-module. For vault-wide operations, you need to be specific about scope.\nCanvas files are awkward. They\u0026rsquo;re JSON under the hood, and Claude can technically edit them, but reasoning about spatial layout in a canvas isn\u0026rsquo;t something Claude does well.\nPlugin-specific features need hand-holding. Bases queries, Dataview syntax, Templater scripts. Claude knows these exist, but you need to be explicit about what you want. The CLAUDE.md helps, but it\u0026rsquo;s not automatic.\nGetting Started # If you want to try this:\nOpen your terminal in your Obsidian vault folder and run claude Create a CLAUDE.md at the vault root. Describe your folder structure, naming conventions, and note format Pick your most repeated task and make it a skill. For most people, that\u0026rsquo;s creating new notes from raw material Add frontmatter to your notes. Even basic fields (title, tags, date) give Claude something to work with Check out kepano\u0026rsquo;s Claude Skills for Obsidian for inspiration on what\u0026rsquo;s possible You don\u0026rsquo;t need to restructure your entire vault. Start with one folder, get the conventions right, and expand from there.\nThe Real Payoff # Here\u0026rsquo;s what I didn\u0026rsquo;t expect: the biggest benefit isn\u0026rsquo;t the AI. It\u0026rsquo;s that building for AI forced me to build a better knowledge base.\nConsistent naming, rich metadata, clear templates, documented conventions. These make the vault better for me, not just for Claude. I can find notes faster, navigate modules more intuitively, and trust that new content will fit the existing structure.\nClaude is the forcing function. Good structure is the actual reward.\nReferences: James Bedford\u0026rsquo;s Obsidian + Claude walkthrough, kepano\u0026rsquo;s Claude Skills for Obsidian, Greg Isenberg\u0026rsquo;s personal OS approach\n","date":"21 March 2026","externalUrl":null,"permalink":"/posts/obsidian-claude-code-knowledge-base/","section":"Posts","summary":" Obsidian + Claude Code is everywhere right now. But pointing an AI at a folder of markdown files and hoping for the best doesn’t work. What matters is how you structure the knowledge base. Get that right, and Claude becomes genuinely useful. Get it wrong, and you get confident garbage. There’s been a wave of posts about this combo lately: James Bedford’s full walkthrough, Greg Isenberg’s “personal OS” approach, kepano (Obsidian’s CEO) sharing Claude Skills. They’re all worth reading.\n","title":"Building a Knowledge Base That AI Can Actually Use","type":"posts"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/claude-code/","section":"Tags","summary":"","title":"Claude-Code","type":"tags"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/developer-tools/","section":"Tags","summary":"","title":"Developer-Tools","type":"tags"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/categories/development-environment/","section":"Categories","summary":"","title":"Development Environment","type":"categories"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/knowledge-management/","section":"Tags","summary":"","title":"Knowledge-Management","type":"tags"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/obsidian/","section":"Tags","summary":"","title":"Obsidian","type":"tags"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/posts/","section":"Posts","summary":"","title":"Posts","type":"posts"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/productivity/","section":"Tags","summary":"","title":"Productivity","type":"tags"},{"content":"","date":"21 March 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":" Boris Cherny created Claude Code. When he shared how he actually uses it day-to-day, the setup was surprisingly simple. I went through every tip, tried most of them, and have opinions about all of them. The original thread is on Boris\u0026rsquo;s X account. A good companion site is howborisusesclaudecode.com which compiles everything in one place.\nThe \u0026ldquo;Surprisingly Vanilla\u0026rdquo; Setup # Boris opens with this: his setup is surprisingly vanilla. Claude Code works great out of the box, and he doesn\u0026rsquo;t customize it much.\nThis is worth sitting with for a second. The person who built the tool doesn\u0026rsquo;t have some secret 500-line CLAUDE.md or a custom plugin stack. He basically uses the defaults.\nI think there\u0026rsquo;s a lesson here. The developer tooling community has a habit of over-engineering setups before actually using the thing. I\u0026rsquo;ve seen people spend more time configuring their AI coding assistant than actually coding with it. Boris\u0026rsquo;s approach is the opposite: start simple, add customization only when you hit a real friction point.\nThat said, \u0026ldquo;vanilla\u0026rdquo; for the creator of Claude Code and \u0026ldquo;vanilla\u0026rdquo; for the rest of us are probably different things. He knows the tool\u0026rsquo;s internals. He knows what it can do without being told. The rest of us might need a bit more scaffolding.\nOpus for Everything # Boris uses Opus 4.5 with extended thinking for every single task. His reasoning: even though Opus is bigger and slower than Sonnet, you steer it less. It\u0026rsquo;s better at tool use, better at following complex instructions, and that makes it faster end-to-end.\nI\u0026rsquo;ve gone back and forth on this. For a while I was using Sonnet for quick tasks (rename this variable, write a test for this function) and Opus for anything architectural. Boris\u0026rsquo;s argument changed how I think about it.\nThe cost isn\u0026rsquo;t just \u0026ldquo;time to generate output.\u0026rdquo; It\u0026rsquo;s also the correction loops. When a smaller model misunderstands the task and you spend three turns fixing it, that Sonnet speed advantage evaporates.\nMy take: Opus for anything that touches more than one file. For single-file, well-scoped changes where I know exactly what I want, Sonnet is still fine. But I\u0026rsquo;ve moved my default to Opus. 5 Parallel Sessions # This is the one that blew people\u0026rsquo;s minds. Boris runs 5 Claude Code instances in parallel across 5 separate git checkouts of the same repo, numbered tabs 1-5. On top of that, he runs 5-10 more sessions on claude.ai/code. He uses system notifications to know when Claude needs input.\nThe result: 20-30 PRs per day.\nThat number is impressive, but context matters. He\u0026rsquo;s working on the Claude Code codebase itself, which he knows intimately. He can scope tasks precisely because he wrote most of the code. Each parallel session gets a well-defined, independent task.\nFor my own work, I\u0026rsquo;ve tried running 3 sessions in parallel. It works best when:\nThe tasks are genuinely independent (different features, different parts of the codebase) Each task is well-scoped enough that Claude can run for a while without needing input You have a good mental model of what each session is doing Where it falls apart: when tasks overlap, when you need to context-switch constantly to answer Claude\u0026rsquo;s questions, or when you don\u0026rsquo;t have enough independent work to fill the sessions. Running 5 instances on a small project is just waste.\nCLAUDE.md as Team Knowledge # The Claude Code team shares a single CLAUDE.md checked into git. The whole team contributes to it multiple times a week. Whenever Claude does something wrong, they add a note so it doesn\u0026rsquo;t happen again.\nThis is the tip I find most valuable.\nI\u0026rsquo;ve been using CLAUDE.md as a personal config file — project structure, build commands, testing conventions. Boris treats it as a living document that captures institutional knowledge. Every mistake becomes a permanent fix.\nThink about it: in a normal team, someone discovers a gotcha, mentions it in Slack, and it\u0026rsquo;s forgotten in two days. With CLAUDE.md, that gotcha becomes a rule that every team member\u0026rsquo;s AI assistant follows forever.\n# Example CLAUDE.md pattern (inspired by Boris\u0026#39;s approach) ## Things Claude gets wrong in this codebase - Don\u0026#39;t use `console.log` for debugging, use the `logger` module - The `user` table has soft deletes — always filter by `deleted_at IS NULL` - Tests must not hit the network — use the `nock` fixtures in `test/fixtures/` If I could only keep one tip from the entire thread, it would be this one. A team-maintained CLAUDE.md is more valuable than any individual configuration trick.\nPlan Mode First # Most of Boris\u0026rsquo;s sessions start in Plan mode (Shift+Tab twice). If the goal is a PR, he iterates on the plan until he likes it, then switches to auto-accept edits mode. Claude usually one-shots it from there.\nI\u0026rsquo;ve adopted this pattern and it works well. The planning step catches misunderstandings before Claude writes 200 lines of code in the wrong direction. Without Plan mode, I\u0026rsquo;d often get a large diff that was 80% right but required manual cleanup. With Plan mode, the success rate on the first attempt is noticeably higher.\nThe workflow looks like:\nEnter Plan mode (Shift+Tab twice) Describe what you want Read Claude\u0026rsquo;s plan, push back on parts you disagree with Once the plan looks right, switch to Act mode Let Claude execute Step 3 is where most of the value is. It\u0026rsquo;s cheaper to fix a plan than to fix code.\nSlash Commands for Inner Loops # Boris uses custom slash commands for every workflow he repeats multiple times a day. These are markdown files in .claude/commands/ that define reusable prompts.\nI haven\u0026rsquo;t used these as much as I should. Most of my repetitive tasks are things like \u0026ldquo;run the tests and fix whatever fails\u0026rdquo; or \u0026ldquo;lint this file and fix the issues.\u0026rdquo; Those are good candidates for slash commands.\n# .claude/commands/fix-tests.md Run the test suite. For any failing tests, analyze the failure, fix the code (not the test unless the test itself is wrong), and re-run until all tests pass. The idea is simple: anything you type more than twice should be a slash command. Boris has these for his inner-loop workflows. I\u0026rsquo;m starting to build mine.\nPermissions, Not \u0026ndash;dangerously-skip # Boris doesn\u0026rsquo;t use --dangerously-skip-permissions. Instead, he uses /permissions to pre-allow specific bash commands he knows are safe in his environment. Most of these are checked into .claude/settings.json and shared with the team.\nThis is the security-conscious approach. --dangerously-skip-permissions is a sledgehammer. /permissions is a scalpel.\nIn practice, I\u0026rsquo;ve found a handful of permissions covers 90% of the prompts:\nnpm test, npm run lint git commands (status, diff, log) hugo server, hugo build npx playwright test Pre-allowing these means Claude can run tests and lint without asking, while still prompting for anything destructive. Much better than either extreme (constant prompts or no safety at all).\nMCP Integrations # Claude Code uses all of Boris\u0026rsquo;s tools for him. It searches and posts to Slack (via MCP server), runs BigQuery queries, grabs error logs from Sentry. The Slack MCP configuration is checked into .mcp.json and shared with the team.\nMCP (Model Context Protocol) is where Claude Code goes from \u0026ldquo;AI code editor\u0026rdquo; to \u0026ldquo;AI coworker.\u0026rdquo; When Claude can pull context from Slack, query your database, and check your error logs, the quality of its suggestions improves dramatically because it has the same context you do.\nI don\u0026rsquo;t have Slack or Sentry hooked up, but I use MCP for documentation lookups. The principle is the same: the more context Claude has about your environment, the less you have to explain.\nVerification is Everything # Boris saved the most important tip for last: give Claude a way to verify its work. If Claude has a feedback loop, it will 2-3x the quality of the final result.\nHis example: Claude tests every single change he lands to claude.ai/code using the Claude Chrome extension. It opens a browser, tests the UI, and iterates until the code works and the UX feels good.\nFor those of us without a Chrome extension testing setup, the same principle applies at a simpler level:\nWrite tests before asking Claude to implement the feature Include \u0026ldquo;run the tests and verify\u0026rdquo; as the last step of every prompt Use CI as a verification step (Claude can read CI output and fix issues) This is the tip that made the biggest difference in my workflow. Before, I\u0026rsquo;d review Claude\u0026rsquo;s output and manually test. Now, I make sure Claude can test its own work. The quality jump is real. Not 2-3x for every task, but for anything UI-related or integration-heavy, it\u0026rsquo;s significant. What I\u0026rsquo;m Taking Away # If I had to compress Boris\u0026rsquo;s thread into three principles:\nStart simple, customize when friction appears. Don\u0026rsquo;t over-engineer your setup. Make CLAUDE.md a team habit. Every mistake Claude makes should become a permanent rule. Give Claude a feedback loop. Tests, linters, browser automation. The tool that can verify its own work produces dramatically better results. The parallel sessions and Opus-for-everything tips are worth experimenting with, but they\u0026rsquo;re optimizations on top of these fundamentals. Get the basics right first.\nWhat\u0026rsquo;s your Claude Code setup? I\u0026rsquo;m always curious how other developers are using it.\nSources: Boris Cherny\u0026rsquo;s original thread, howborisusesclaudecode.com\n","date":"21 March 2026","externalUrl":null,"permalink":"/posts/boris-claude-code-tips/","section":"Posts","summary":" Boris Cherny created Claude Code. When he shared how he actually uses it day-to-day, the setup was surprisingly simple. I went through every tip, tried most of them, and have opinions about all of them. The original thread is on Boris’s X account. A good companion site is howborisusesclaudecode.com which compiles everything in one place.\n","title":"What I Learned from How Claude Code's Creator Uses Claude Code","type":"posts"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/categories/ai--productivity/","section":"Categories","summary":"","title":"AI \u0026 Productivity","type":"categories"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/automation/","section":"Tags","summary":"","title":"Automation","type":"tags"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/aws/","section":"Tags","summary":"","title":"Aws","type":"tags"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/billing/","section":"Tags","summary":"","title":"Billing","type":"tags"},{"content":" I spent five years on the AWS Billing team. The hardest problem I tackled was detecting when customers used AWS services but weren\u0026rsquo;t charged correctly. This post walks through how I designed a system that reduced charge discrepancies by 300x and eliminated 230 million monthly false positives. The Problem # AWS billing is trickier than it looks. When a customer launches an EC2 instance, writes to S3, or queries DynamoDB, each action generates a usage record. These records flow through a pipeline that calculates charges based on the customer\u0026rsquo;s pricing plan, region, and service tier.\nSometimes, usage records don\u0026rsquo;t make it through the pipeline correctly. A record might get dropped, delayed, or processed with the wrong pricing. The customer gets under-charged (AWS loses revenue) or over-charged (customer trust erodes). At AWS\u0026rsquo;s scale, even a tiny error rate across millions of customers adds up fast.\nThe existing detection system flagged potential discrepancies, but it generated 230 million false positive alerts per month. That volume made the alerts useless in practice. Engineers would triage a few, find nothing, and ignore the rest. Real discrepancies were buried in noise.\nThe Challenge # The core tension was precision vs. recall:\nToo aggressive: Flag everything, drown in false positives (the status quo) Too conservative: Miss real discrepancies, lose money and customer trust Just right: Flag only genuine issues, with enough context to act quickly Additionally, the system had to:\nProcess billions of usage records across every AWS service Handle the different pricing models, discount programs, and billing cycles Run continuously without impacting the billing pipeline\u0026rsquo;s latency Produce actionable alerts, not just \u0026ldquo;something looks wrong\u0026rdquo; The Approach # I designed the Unbilled Usage Auditor as a distributed system with three components:\nflowchart LR A[\"Smart Sampling\\n\u0026 Aggregation\"] --\u003e B[\"Multi-Signal\\nValidation\"] B --\u003e C[\"Automated\\nResolution\"] C --\u003e D{Real issue?} D -- Yes --\u003e E[\"Alert with\\ndiagnosis\"] D -- No --\u003e F[\"Auto-resolve\"] 1. Smart Sampling and Aggregation # Instead of checking every individual usage record (which generated the 230M false positives), I aggregated usage at the service-account-period level. This reduced the comparison space by orders of magnitude while preserving the ability to detect genuine discrepancies.\nThe aggregation was built on DynamoDB for its consistent low-latency reads at any scale. Each aggregated record stored the expected charge (from usage records) and the actual charge (from the billing output), along with metadata about the pricing plan and discount programs applied.\n2. Multi-Signal Validation # A single mismatch between expected and actual charges doesn\u0026rsquo;t necessarily indicate a problem. Pricing changes, retroactive discounts, and billing cycle boundaries all create legitimate temporary discrepancies.\nI built a validation pipeline using AWS Lambda that checked multiple signals before escalating:\nTemporal correlation: Is this a timing issue that will self-correct in the next billing cycle? Pricing context: Did a pricing change or discount activation explain the difference? Historical pattern: Has this account/service combination shown similar patterns before? Magnitude thresholds: Is the discrepancy large enough to warrant investigation? Only records that failed all validation checks were escalated as genuine alerts.\n3. Automated Resolution Pipeline # For common discrepancy patterns, the system could trigger automated remediation: re-processing usage records, applying missing discounts, or flagging records for manual review with specific context about what went wrong.\nSo when an engineer did receive an alert, it came with a diagnosis, not just a symptom.\nThe Results # After rolling out the Unbilled Usage Auditor:\nMetric Before After Change Monthly false positives 230,000,000 \u0026lt; 1,000 -99.99% Charge discrepancies $125,000 $432 -99.65% (300x) Time to resolve alerts Days Hours ~10x faster Alert actionability ~0% (noise) ~95% (genuine) Usable The 300x reduction in charge discrepancies (from $125,000 to $432) came from catching real issues that were previously buried under false positives. The system didn\u0026rsquo;t just reduce noise. It uncovered signal that had always been there.\nWhat I Learned # Correctness Is a Spectrum # In billing systems, \u0026ldquo;correct\u0026rdquo; isn\u0026rsquo;t binary. There\u0026rsquo;s the charge that\u0026rsquo;s mathematically right given current pricing, the charge that\u0026rsquo;s right given the customer\u0026rsquo;s expectation, and the charge that\u0026rsquo;s right given the business rules (discounts, credits, negotiations). The system had to reason about all three.\nFalse Positive Reduction Is Its Own Feature # Reducing false positives wasn\u0026rsquo;t just an optimization. It was a prerequisite for the system being useful at all. A detection system that cries wolf 230 million times a month is a noise generator, not a detection system. The biggest impact came not from finding new problems, but from making existing problems visible.\nScale Changes the Problem # At AWS scale, approaches that work for thousands of records completely break at billions. The shift from per-record checking to aggregated analysis wasn\u0026rsquo;t just an optimization; it required rethinking the problem from scratch. When your scale changes by 1000x, your architecture probably needs to change, not just your hardware.\nThe hardest part of this project wasn\u0026rsquo;t the distributed systems engineering. It was understanding the domain deeply enough to tell a real discrepancy from expected behavior. Technical skill got me the system; domain knowledge got me the 300x improvement. This project shaped how I think about system design: start with correctness guarantees, then optimize for signal-to-noise ratio, and always build systems that explain their decisions.\nFor more about my career journey and other projects, see my experience page or projects page.\n","date":"22 February 2026","externalUrl":null,"permalink":"/posts/case-study-aws-billing-auditor/","section":"Posts","summary":" I spent five years on the AWS Billing team. The hardest problem I tackled was detecting when customers used AWS services but weren’t charged correctly. This post walks through how I designed a system that reduced charge discrepancies by 300x and eliminated 230 million monthly false positives. The Problem # AWS billing is trickier than it looks. When a customer launches an EC2 instance, writes to S3, or queries DynamoDB, each action generates a usage record. These records flow through a pipeline that calculates charges based on the customer’s pricing plan, region, and service tier.\n","title":"Case Study: Building AWS Billing's Unbilled Usage Auditor","type":"posts"},{"content":" At Twitter, I was responsible for kernel updates across 5,000+ production servers. Updating a kernel is risky on one machine. Doing it across a fleet, without downtime, without data loss, and without breaking the services that millions of people depend on, is a different problem entirely. The Problem # Twitter\u0026rsquo;s production infrastructure ran on thousands of bare-metal servers across multiple data centers. Each server ran a Linux kernel that needed regular updates for security patches, performance improvements, and hardware compatibility.\nThe challenge wasn\u0026rsquo;t updating one kernel. It was updating thousands, safely:\nHeterogeneous fleet: Different hardware generations, different workloads, different kernel configurations. A kernel that works perfectly on one host type might crash on another. Zero tolerance for downtime: These servers ran core Twitter services. A bad kernel update could take down a shard of the user timeline, DM delivery, or ad serving. Manual process: Before my work, kernel updates were largely manual. Engineers would update hosts in small batches, watch for issues, and roll back if something went wrong. At 5,000+ hosts, this didn\u0026rsquo;t scale. Validation gap: There was no systematic way to validate that a new kernel version was safe for a given host type before rolling it out. The Approach # I built three interlocking systems to solve this:\nflowchart LR A[\"Canary\\nValidation\"] --\u003e B[\"Wave 1\\n1%\"] B --\u003e C[\"Wave 2\\n5%\"] C --\u003e D[\"Wave 3\\n25%\"] D --\u003e E[\"Full Fleet\"] B -- anomaly --\u003e F[\"Pause \u0026\\nAlert\"] C -- anomaly --\u003e F D -- anomaly --\u003e F 1. Canary Kernel Validation Library # Before any kernel could be rolled out fleet-wide, it had to pass canary validation. I built a Python library that:\nProvisioned canary hosts: Selected representative hosts from each hardware/workload combination in the fleet Applied the kernel update: Installed the new kernel and rebooted the canary hosts Ran validation suites: Checked system stability, performance benchmarks, and application-level health checks Compared baselines: Measured the canary against production baselines — CPU utilization, memory pressure, I/O latency, network throughput Only after a kernel passed canary validation on every host type would it be approved for fleet-wide rollout.\nCanary validation isn\u0026rsquo;t just \u0026ldquo;does the kernel boot?\u0026rdquo; It\u0026rsquo;s \u0026ldquo;does the kernel behave identically to the current one under production-like load?\u0026rdquo; A kernel can boot fine and still introduce a 5% latency regression that cascades into user-visible impact. 2. Automated Rollout System # Once a kernel was validated, the rollout system handled deployment in progressive waves:\nWave 1: 1% of the fleet (a handful of hosts per data center) Wave 2: 5% — expanding to more host types Wave 3: 25% — majority coverage Wave 4: Remaining hosts Between each wave, the system monitored for anomalies: unexpected reboots, performance regression, application errors. If any signal crossed a threshold, the rollout paused automatically and alerted the on-call engineer with context about what went wrong and which hosts were affected.\n3. Fleet Configuration Standardization # I discovered that a big chunk of fleet management pain came from configuration drift. Hosts had been manually tweaked over years and no longer matched their expected state.\nI built tooling to:\nAudit configurations: Scan every host and compare its actual state to the declared state Detect drift: Identify hosts that had diverged from their intended configuration Remediate automatically: For safe divergences, apply corrections. For risky ones, flag for human review. This wasn\u0026rsquo;t strictly a kernel problem, but it was a prerequisite. You can\u0026rsquo;t safely automate kernel updates on hosts whose configuration you don\u0026rsquo;t fully understand.\n4. Cache Service Custom Commands # I also designed a custom commands system for Twitter\u0026rsquo;s Redis -based cache services using Go . This let operators inspect and modify cache behavior at runtime without restarting services, which was critical for debugging production issues without customer impact.\nThe Results # Metric Before After Kernel update method Manual, batch-by-batch Automated, progressive rollout Time to update fleet Weeks Days Hosts with validated kernels Partial 5,000+ (full fleet) Configuration drift detection None Continuous On-call ticket volume (peak) Unmanageable 140+ resolved in one week The \u0026ldquo;140+ tickets in one on-call week\u0026rdquo; stat deserves context. This wasn\u0026rsquo;t a normal week; the fleet had accumulated significant technical debt. The tooling I\u0026rsquo;d built let me systematically triage and resolve issues that would have previously required investigating each host individually.\nWhat I Learned # Automation Without Validation Is Dangerous # The temptation with fleet automation is to focus on speed — how fast can we push updates to every host? But speed without validation means speed at failing. The canary validation system was the most important piece, not the rollout automation.\nConfiguration Drift Is the Silent Killer # The hardest bugs to debug in fleet management aren\u0026rsquo;t kernel bugs. They\u0026rsquo;re \u0026ldquo;why does this host behave differently from every other host of the same type?\u0026rdquo; The answer is almost always configuration drift that accumulated over months or years. Investing in configuration auditing paid for itself many times over.\nOn-Call Is a Design Problem # Resolving 140+ tickets in a week wasn\u0026rsquo;t about working harder. It was about having the right tools. When your tooling gives you enough context to diagnose and resolve issues in minutes instead of hours, you can handle 10x the volume. The best on-call experience is one where the tools do the investigation and the human makes the decision.\nFleet management at scale comes down to trust: trusting that your hosts are in the state you think they are, trusting that an update won\u0026rsquo;t break things, and trusting that if something does go wrong, you\u0026rsquo;ll know immediately and can recover automatically. Every system I built was about establishing and maintaining that trust. This experience shaped my approach to infrastructure: instrument everything, validate before acting, and design systems that explain themselves when they fail.\nFor more about my career journey and other projects, see my experience page or projects page.\n","date":"22 February 2026","externalUrl":null,"permalink":"/posts/case-study-twitter-fleet-automation/","section":"Posts","summary":" At Twitter, I was responsible for kernel updates across 5,000+ production servers. Updating a kernel is risky on one machine. Doing it across a fleet, without downtime, without data loss, and without breaking the services that millions of people depend on, is a different problem entirely. The Problem # Twitter’s production infrastructure ran on thousands of bare-metal servers across multiple data centers. Each server ran a Linux kernel that needed regular updates for security patches, performance improvements, and hardware compatibility.\n","title":"Case Study: Fleet-Scale Kernel Automation at Twitter","type":"posts"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/case-study/","section":"Tags","summary":"","title":"Case-Study","type":"tags"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/catppuccin/","section":"Tags","summary":"","title":"Catppuccin","type":"tags"},{"content":" Your development environment should feel like one cohesive tool, not a collection of unrelated windows with clashing colors. I theme everything with the same palette — Catppuccin Mocha — and the result is a workspace where context-switching between tools is effortless.\nWhy One Palette Everywhere? # Most developers pick a theme for their editor and call it a day. Their terminal is one color, their editor another, their tmux status bar a third, and their Git diffs something else entirely. Every time they switch contexts, their brain spends a fraction of a second recalibrating.\nThat friction adds up. When red means error in every single tool — terminal output, editor diagnostics, Git diffs, monitoring dashboards — you process information faster because your visual vocabulary is consistent.\nThe Palette # Catppuccin is a community-driven pastel theme with four flavor variants. I use Mocha, the darkest flavor, across everything.\nBase Colors # Base, Surface0, Surface1 — the foundation. Deep blues that are easy on the eyes during long sessions.\nAccent Colors # Blue (info/links), Red (errors/deletions), Green (success/additions) — these carry semantic meaning across every tool.\nWarm Accents # Peach (warnings), Yellow (highlights), Mauve (keywords) — used for syntax highlighting and UI accents.\nWhere I Use It # Every tool in my daily workflow runs Catppuccin Mocha:\nConfiguration Snippets # Here\u0026rsquo;s how I set up Catppuccin Mocha in each tool:\n# ~/.config/ghostty/config theme = catppuccin-mocha background-opacity = 0.75 background-blur-radius = 20 Ghostty ships with Catppuccin built-in — just set the theme name. The transparency lets the palette blend with the desktop for a frosted-glass aesthetic.\n-- LazyVim extra or manual plugin { \u0026#34;catppuccin/nvim\u0026#34;, name = \u0026#34;catppuccin\u0026#34;, opts = { flavour = \u0026#34;mocha\u0026#34;, transparent_background = true, integrations = { treesitter = true, telescope = { enabled = true }, which_key = true, }, }, } The Catppuccin Neovim plugin has first-class integrations with virtually every popular plugin — Treesitter, Telescope, nvim-cmp, and more all get coordinated colors.\n# ~/.config/tmux/tmux.conf set -g @plugin \u0026#39;catppuccin/tmux\u0026#39; set -g @catppuccin_flavor \u0026#39;mocha\u0026#39; set -g @catppuccin_window_status_style \u0026#34;rounded\u0026#34; The tmux plugin adds a themed status bar with window indicators, session name, and system info — all matching the terminal and editor.\n# ~/.zshrc export FZF_DEFAULT_OPTS=\u0026#34; \\ --color=bg+:#313244,bg:#1e1e2e,spinner:#f5e0dc \\ --color=hl:#f38ba8,fg:#cdd6f4,header:#f38ba8 \\ --color=info:#cba6f7,pointer:#f5e0dc,marker:#f5e0dc \\ --color=fg+:#cdd6f4,prompt:#cba6f7,hl+:#f38ba8\u0026#34; FZF\u0026rsquo;s color configuration takes raw hex values, so you set each UI element individually. Once configured, fuzzy finding matches the rest of your environment perfectly.\n# ~/.gitconfig [delta] syntax-theme = Catppuccin Mocha minus-style = syntax \u0026#34;#3B1219\u0026#34; plus-style = syntax \u0026#34;#1C3A2D\u0026#34; Delta (the Git pager) uses the Catppuccin bat theme for syntax highlighting in diffs. The custom minus/plus styles tint additions green and deletions red using the Mocha palette.\nThe Cognitive Load Argument # The real benefit isn\u0026rsquo;t aesthetics — it\u0026rsquo;s that context-switching between terminal, editor, and browser no longer requires your brain to re-adapt to different color semantics. Red means error everywhere. Blue means info everywhere. Green means success everywhere. When I jump from reading a Git diff (red = deleted, green = added) to my editor (red = error diagnostic, green = test passing) to my terminal output (red = failed command, green = success), my brain doesn\u0026rsquo;t skip a beat. The color semantics are identical.\nThis is the same principle behind why airports, hospitals, and road signs use standardized color systems. Consistency reduces cognitive overhead.\nThe Full Color Reference # For anyone wanting to replicate this setup, here are the key Catppuccin Mocha values I use most:\nRole Color Hex Usage Base Dark blue #1e1e2e Backgrounds Text Light lavender #cdd6f4 Primary text Blue Soft blue #89b4fa Links, info, variables Red Soft red #f38ba8 Errors, deletions Green Soft green #a6e3a1 Success, additions Peach Warm peach #fab387 Warnings, numbers Yellow Soft yellow #f9e2af Highlights, strings Mauve Purple #cba6f7 Keywords, headings Overlay Muted gray #6c7086 Comments, secondary text Getting Started # If you want to try Catppuccin Mocha yourself, the project has ports for 300+ apps. Start with your terminal emulator and editor, then expand from there.\ncatppuccin/catppuccin 😸 Soothing pastel theme for the high-spirited! TypeScript 18600 339 For my complete configuration files including all the snippets above, check out my dotfiles:\nnickboy/dotfiles Dotfiles Shell 0 0 ","date":"22 February 2026","externalUrl":null,"permalink":"/posts/catppuccin-mocha-theming/","section":"Posts","summary":" Your development environment should feel like one cohesive tool, not a collection of unrelated windows with clashing colors. I theme everything with the same palette — Catppuccin Mocha ","title":"Catppuccin Mocha: Why I Theme Everything the Same Color","type":"posts"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/distributed-systems/","section":"Tags","summary":"","title":"Distributed-Systems","type":"tags"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/infrastructure/","section":"Tags","summary":"","title":"Infrastructure","type":"tags"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/terminal/","section":"Tags","summary":"","title":"Terminal","type":"tags"},{"content":"","date":"22 February 2026","externalUrl":null,"permalink":"/tags/theming/","section":"Tags","summary":"","title":"Theming","type":"tags"},{"content":" Imagine you\u0026rsquo;re blindfolded on a mountain and you need to find the lowest valley. You can\u0026rsquo;t see anything, but you can feel the ground under your feet. What would you do? You\u0026rsquo;d feel which direction slopes downward, take a small step that way, and repeat. Congratulations — you just invented gradient descent, the algorithm behind nearly every modern AI system. Why Should You Care? # Optimization is everywhere. When your GPS finds the fastest route, when Netflix recommends a movie, when your phone recognizes your face — behind all of these is an algorithm trying to find the best possible answer from a sea of possibilities. Gradient descent is the workhorse algorithm that makes this happen.\nThe Blindfolded Hiker # Let\u0026rsquo;s flesh out the analogy with a mapping table:\nHiking (Real World) Gradient Descent (Math) Your position on the mountain Current weights \\(\\mathbf{w}\\) Your altitude Error \\(E\\) The slope of the ground under your feet Gradient \\(\\nabla E\\) Your step size Learning rate \\(\\eta\\) The valley floor Optimal solution The hiker\u0026rsquo;s algorithm is simple:\nFeel the slope under your feet (compute the gradient) Step downhill in the steepest direction (update the weights) Repeat until the ground feels flat (convergence) Why Do We Need This? # Here\u0026rsquo;s the problem machines face. Say you\u0026rsquo;re building a model to predict whether a student will pass an exam based on how many hours they studied. Your model has adjustable knobs called weights — and you need to find the weight values that make the best predictions.\nBut you can\u0026rsquo;t just try every possible value. With even a few weights, the number of combinations is astronomical. Instead, you start somewhere and iteratively improve — just like our blindfolded hiker.\nThe Error Function: How Wrong Are We? # First, we need a way to measure \u0026ldquo;how wrong\u0026rdquo; our model is. The most common choice is squared error:\n$$E(\\mathbf{w}) = \\frac{1}{2}\\sum_{(x,y) \\in D}(y - a)^2$$\nSymbol Meaning \\(y\\) The correct answer (target) \\(a\\) Our model\u0026rsquo;s prediction (activation) \\(\\frac{1}{2}\\) A convenience factor that makes the derivative cleaner This function creates an error surface — imagine a landscape where altitude represents how wrong you are. Every point on this landscape corresponds to a different set of weights. Our goal: find the lowest point.\nThe Gradient: Which Way Is Downhill? # The gradient is a vector that points in the direction where the error increases the fastest:\n$$\\nabla E = \\left[\\frac{\\partial E}{\\partial w_0}, \\frac{\\partial E}{\\partial w_1}, \\ldots, \\frac{\\partial E}{\\partial w_n}\\right]$$\nSince we want to decrease error, we walk in the opposite direction of the gradient:\n$$\\mathbf{w} \\leftarrow \\mathbf{w} - \\eta \\nabla E$$\nThe minus sign is doing the heavy lifting here — it flips \u0026ldquo;uphill\u0026rdquo; into \u0026ldquo;downhill.\u0026rdquo;\nCommon confusion: The gradient tells you the slope, not a destination. It says \u0026ldquo;error increases fastest in this direction\u0026rdquo; — it doesn\u0026rsquo;t say \u0026ldquo;the minimum is over there.\u0026rdquo; That\u0026rsquo;s why we take small steps rather than leaping. The Delta Rule: A Concrete Formula # For a single-neuron model predicting with \\(a = \\sum_i w_i x_i\\), we can derive a clean update rule using the chain rule:\n$$\\frac{\\partial E}{\\partial w_i} = -(y - a) \\cdot x_i$$\nSince we move against the gradient:\n$$\\Delta w_i = \\eta(y - a)x_i$$\nThis is called the Delta Rule, and it\u0026rsquo;s beautifully intuitive:\n\\((y - a)\\): How wrong are we? If the prediction is too low, this is positive → increase the weight \\(x_i\\): How much did this input contribute? Bigger inputs get bigger adjustments \\(\\eta\\): How big a step should we take? (the learning rate) A Step-by-Step Example # Problem: Predict if a student passes an exam based on hours studied.\nInput: \\(x = [1, 2]\\) (1 is the bias term, 2 hours studied) Target: \\(y = 1\\) (passed) Initial weights: \\(w = [0.1, 0.3]\\) Learning rate: \\(\\eta = 0.1\\) Iteration 1 # Predict: \\(a = 0.1 \\times 1 + 0.3 \\times 2 = 0.7\\)\nError: \\(y - a = 1 - 0.7 = 0.3\\) (we predicted too low)\nUpdate: $$\\Delta w_0 = 0.1 \\times 0.3 \\times 1 = 0.03$$ $$\\Delta w_1 = 0.1 \\times 0.3 \\times 2 = 0.06$$\nNew weights: \\(w = [0.13, 0.36]\\)\nWatch It Converge # Iteration \\(w_0\\) \\(w_1\\) Prediction \\(a\\) Error 0 0.100 0.300 0.700 0.300 1 0.130 0.360 0.850 0.150 2 0.145 0.390 0.925 0.075 3 0.153 0.405 0.963 0.037 \u0026hellip; \u0026hellip; \u0026hellip; \u0026hellip; → 0 Notice how the error shrinks each iteration — and the updates get smaller too. When you\u0026rsquo;re close to the answer, you take smaller steps. The algorithm naturally slows down as it approaches the solution.\nBatch vs. Stochastic: Two Flavors of Descent # There are two main ways to apply gradient descent, and a restaurant analogy helps explain the difference:\nBatch Gradient Descent # Batch GD looks at all the training data before making a single update.\nAnalogy: You\u0026rsquo;re a chef. You ask every single customer what they thought of the meal, compile all the feedback, then make one careful adjustment to the recipe.\nPro: Stable, smooth path toward the minimum Con: Slow — you have to process everything before each step Stochastic Gradient Descent (SGD) # SGD updates after each individual training example.\nAnalogy: You ask one customer what they thought and immediately tweak the recipe. Then the next customer, tweak again. It\u0026rsquo;s chaotic but fast.\nPro: Much faster, and the randomness can help escape bad solutions Con: Noisy, zigzag path graph LR subgraph Batch[\"Batch GD\"] B1[\"Smooth, directpath to minimum\"] --\u003e B2[\"●Minimum\"] end subgraph SGD[\"Stochastic GD\"] S1[\"Zigzag, noisypath to minimum\"] --\u003e S2[\"●Minimum\"] end In practice, most systems use mini-batch gradient descent — a middle ground where you look at a small batch (say 32 or 64 examples) at a time. It gets the best of both worlds.\nThe Local Optima Trap # Here\u0026rsquo;s the catch: gradient descent finds a local minimum, not necessarily the global minimum.\nThink about it — our blindfolded hiker can only feel the ground directly underfoot. If they walk into a small ditch, the ground slopes up in every direction, so they stop. But there might be a much deeper valley a mile away that they\u0026rsquo;ll never find.\ngraph LR A[\"Start\"] --\u003e B[\"Walk downhill...\"] B --\u003e C[\"Stuck inlocal minimum!\"] C -.-\u003e|\"Can't see this\"| D[\"Global minimum(the real answer)\"] Solutions to Getting Stuck # Strategy How It Works Random restarts Try hiking from multiple random starting points and keep the best result Momentum Keep some \u0026ldquo;inertia\u0026rdquo; so you can roll through small ditches Simulated annealing Occasionally allow uphill steps early on, then settle down over time The good news: in modern deep learning with millions of parameters, the error landscape is so high-dimensional that true local minima are actually quite rare. Most \u0026ldquo;valleys\u0026rdquo; have an escape route in some dimension.\nWhy This Powers Everything # Gradient descent isn\u0026rsquo;t just a classroom algorithm — it\u0026rsquo;s literally how every modern neural network learns:\nChatGPT learned to write by gradient descent over billions of text examples Self-driving cars use it to tune perception models on driving data Drug discovery uses it to optimize molecular property predictions Your phone\u0026rsquo;s keyboard prediction was trained with it Every time you hear \u0026ldquo;the model was trained on data,\u0026rdquo; gradient descent (or a close variant like Adam) is doing the actual training under the hood.\nKey Takeaways # Gradient descent finds good solutions by repeatedly taking small steps in the direction that reduces error The gradient tells you which direction is \u0026ldquo;uphill\u0026rdquo; — so you go the opposite way The learning rate controls step size: too big and you overshoot, too small and you\u0026rsquo;ll take forever Batch gradient descent is stable but slow; stochastic is fast but noisy You might get stuck in a local minimum, but there are tricks to escape Study Notes: This post is based on my personal notes from studying Machine Learning (CS7641) at Georgia Institute of Technology. The concepts are explained in my own words for learning purposes. ","date":"20 February 2026","externalUrl":null,"permalink":"/posts/ml-gradient-descent/","section":"Posts","summary":" Imagine you’re blindfolded on a mountain and you need to find the lowest valley. You can’t see anything, but you can feel the ground under your feet. What would you do? You’d feel which direction slopes downward, take a small step that way, and repeat. Congratulations — you just invented gradient descent, the algorithm behind nearly every modern AI system. Why Should You Care? # Optimization is everywhere. When your GPS finds the fastest route, when Netflix recommends a movie, when your phone recognizes your face — behind all of these is an algorithm trying to find the best possible answer from a sea of possibilities. Gradient descent is the workhorse algorithm that makes this happen.\n","title":"Finding the Bottom of a Valley Blindfolded: Understanding Gradient Descent","type":"posts"},{"content":"","date":"20 February 2026","externalUrl":null,"permalink":"/tags/georgia-tech/","section":"Tags","summary":"","title":"Georgia-Tech","type":"tags"},{"content":" Imagine you\u0026rsquo;re playing 20 Questions. You\u0026rsquo;re trying to guess what animal your friend is thinking of. Would you start with \u0026ldquo;Is it a golden retriever?\u0026rdquo; or \u0026ldquo;Does it live in water?\u0026rdquo; The second question is obviously smarter — it eliminates roughly half the possibilities in one shot. Decision trees in machine learning work exactly the same way, and they use entropy and information gain to figure out what the smartest question is. What\u0026rsquo;s the Big Idea? # When a machine learning algorithm builds a Decision Tree , it needs to decide which question to ask first. Should it split the data by color? By size? By temperature? The answer comes from a beautifully simple concept: ask the question that reduces uncertainty the most.\nBut to measure \u0026ldquo;uncertainty,\u0026rdquo; we need a number for it. That number is called entropy.\nEntropy: Measuring Chaos # The Messy Room Analogy # Think of entropy as a messiness score:\nSituation Entropy Why All your clothes are folded in the closet Low You know exactly where everything is Clothes are scattered everywhere High You have no idea where anything is In machine learning, \u0026ldquo;messiness\u0026rdquo; means how mixed up the categories are in your data.\nThe Marble Bag # Imagine reaching into a bag of marbles without looking:\nBag A: All red marbles → Entropy = 0 (you know what you\u0026rsquo;ll grab) Bag B: Half red, half blue → Entropy = 1 (maximum uncertainty — it\u0026rsquo;s a coin flip) Bag C: All blue marbles → Entropy = 0 (certain again) The more mixed up the bag is, the higher the entropy. When everything is the same, entropy is zero — there\u0026rsquo;s no surprise.\nThe Math (It\u0026rsquo;s Simpler Than It Looks) # For a dataset \\(S\\) with multiple classes, entropy is:\n$$Entropy(S) = -\\sum_{i=1}^{c} p_i \\log_2(p_i)$$\nWhere \\(p_i\\) is the proportion of class \\(i\\) in the dataset, and \\(c\\) is the number of classes.\nFor the simple case of two classes (yes/no, spam/not-spam):\n$$Entropy(S) = -p_+ \\log_2(p_+) - p_- \\log_2(p_-)$$\nLet\u0026rsquo;s plug in numbers for the marble bags:\nBag A (all red): \\(Entropy = -1 \\times \\log_2(1) - 0 \\times \\log_2(0) = 0\\) Bag B (half and half): \\(Entropy = -0.5 \\times \\log_2(0.5) - 0.5 \\times \\log_2(0.5) = 1\\) The entropy curve looks like a hill — it peaks at 0.5 (maximum confusion) and drops to zero at the extremes (total certainty).\nWhy log base 2? Because entropy is measured in bits — the same bits as in computer science. An entropy of 1 bit means you need exactly one yes/no question to figure out the answer. This connects directly to the \u0026ldquo;20 Questions\u0026rdquo; game! Information Gain: Picking the Best Question # Now for the punchline. Information gain measures how much a particular question reduces entropy:\n$$Gain(S, A) = Entropy(S) - \\sum_{v \\in Values(A)} \\frac{|S_v|}{|S|} \\times Entropy(S_v)$$\nIn plain English: Information Gain = Entropy before asking − Entropy after asking\nA bigger information gain means a better question.\nWorked Example: Should I Play Tennis? # Let\u0026rsquo;s say you have 8 days of data about whether you played tennis:\n5 days you played (✅) 3 days you didn\u0026rsquo;t (❌) Starting entropy:\n$$Entropy = -\\frac{5}{8}\\log_2\\frac{5}{8} - \\frac{3}{8}\\log_2\\frac{3}{8} \\approx 0.954$$\nNow let\u0026rsquo;s compare two possible questions:\nQuestion 1: \u0026ldquo;What\u0026rsquo;s the weather?\u0026rdquo; # graph TD A[\"All Data5✅ 3❌Entropy = 0.954\"] --\u003e|Sunny| B[\"2✅ 1❌Entropy = 0.918\"] A --\u003e|Overcast| C[\"2✅ 0❌Entropy = 0\"] A --\u003e|Rainy| D[\"1✅ 2❌Entropy = 0.918\"] Weighted entropy after splitting by weather:\n$$\\frac{3}{8} \\times 0.918 + \\frac{2}{8} \\times 0 + \\frac{3}{8} \\times 0.918 = 0.689$$\nInformation Gain = 0.954 − 0.689 = 0.265\nQuestion 2: \u0026ldquo;Is it windy?\u0026rdquo; # graph TD A[\"All Data5✅ 3❌Entropy = 0.954\"] --\u003e|Light Wind| B[\"4✅ 1❌Entropy = 0.722\"] A --\u003e|Strong Wind| C[\"1✅ 2❌Entropy = 0.918\"] Weighted entropy after splitting by wind:\n$$\\frac{5}{8} \\times 0.722 + \\frac{3}{8} \\times 0.918 = 0.795$$\nInformation Gain = 0.954 − 0.795 = 0.159\nThe Verdict # Attribute Information Gain Winner? Weather 0.265 ✅ Pick this one! Wind 0.159 Weather gives more information gain, so the decision tree puts it at the top. It\u0026rsquo;s the smarter first question — just like in 20 Questions.\nWhat a Good Split Looks Like # graph LR A[\"Mixed DataHigh Entropy\"] --\u003e|Good Split| B[\"Mostly ✅Low Entropy\"] A --\u003e|Good Split| C[\"Mostly ❌Low Entropy\"] A good split takes a messy group and separates it into purer groups. A bad split leaves you with groups that are still mixed up — you haven\u0026rsquo;t learned much.\nReal-World Applications # This isn\u0026rsquo;t just a classroom exercise. Decision trees powered by entropy and information gain are used everywhere:\nSpam filtering: \u0026ldquo;Does the email contain the word \u0026rsquo;lottery\u0026rsquo;?\u0026rdquo; splits your inbox into much purer groups than \u0026ldquo;Was it sent on a Tuesday?\u0026rdquo; Medical diagnosis: A doctor\u0026rsquo;s diagnostic flowchart is essentially a decision tree — \u0026ldquo;Does the patient have a fever?\u0026rdquo; is a high-information-gain question for many conditions Recommendation systems: Streaming services split users into groups based on features that best predict what they\u0026rsquo;ll watch next Credit scoring: Banks use decision trees to determine which factors best separate reliable borrowers from risky ones Key Takeaways # Entropy measures how mixed up (uncertain) a dataset is — from 0 (pure) to 1 (maximum chaos for two classes) Information gain tells you how much a question reduces that chaos Decision trees are greedy — they always pick the question with the highest information gain first This is exactly the strategy of a smart 20 Questions player: ask the question that eliminates the most possibilities Study Notes: This post is based on my personal notes from studying Machine Learning (CS7641) at Georgia Institute of Technology. The concepts are explained in my own words for learning purposes. ","date":"20 February 2026","externalUrl":null,"permalink":"/posts/ml-entropy-and-information-gain/","section":"Posts","summary":" Imagine you’re playing 20 Questions. You’re trying to guess what animal your friend is thinking of. Would you start with “Is it a golden retriever?” or “Does it live in water?” The second question is obviously smarter — it eliminates roughly half the possibilities in one shot. Decision trees in machine learning work exactly the same way, and they use entropy and information gain to figure out what the smartest question is. What’s the Big Idea? # When a machine learning algorithm builds a Decision Tree , it needs to decide which question to ask first. Should it split the data by color? By size? By temperature? The answer comes from a beautifully simple concept: ask the question that reduces uncertainty the most.\n","title":"How Machines Ask Smart Questions: Entropy \u0026 Information Gain","type":"posts"},{"content":" When a factory produces a defective product, how do you trace the problem back through the assembly line to find which worker made the mistake? Neural networks face the exact same challenge. They have layers of \u0026ldquo;workers\u0026rdquo; (neurons), and when the final output is wrong, they need to figure out who\u0026rsquo;s responsible — and by how much. The algorithm that solves this is called backpropagation, and it\u0026rsquo;s the reason deep learning works at all. Neural Networks Are Everywhere # Before we dive into how neural networks learn, let\u0026rsquo;s appreciate what they do. The phone in your pocket uses neural networks for face recognition, voice transcription, photo enhancement, and text prediction. Self-driving cars, medical image analysis, language translation — all neural networks.\nBut here\u0026rsquo;s the thing: nobody programs these networks to do their jobs. Instead, we show them millions of examples and let them learn from mistakes. Backpropagation is the algorithm that makes this learning possible.\nThe Factory Analogy # Imagine a three-stage factory production line:\nStage Factory Neural Network Stage 1 Worker A processes raw materials Input layer → Hidden layer (weights \\(w\\)) Stage 2 Worker B assembles the product Hidden layer → Output layer (weights \\(v\\)) Stage 3 Quality inspector checks the result Error calculation When the inspector finds a defect:\nFirst, ask Worker B (closest to the output): \u0026ldquo;What went wrong in your assembly?\u0026rdquo; → The error is directly visible Then ask Worker A: \u0026ldquo;How much of Worker B\u0026rsquo;s problem traces back to your material processing?\u0026rdquo; → Blame is distributed proportionally This is exactly how backpropagation works — it starts at the output and traces the error backward through the network, assigning \u0026ldquo;blame\u0026rdquo; to each connection along the way.\nThe Network Structure # Let\u0026rsquo;s work with a simple network: 2 inputs, 2 hidden neurons, 1 output.\ngraph LR x1((x₁)) --\u003e|w₁₁| h1((h₁)) x1 --\u003e|w₁₂| h2((h₂)) x2((x₂)) --\u003e|w₂₁| h1 x2 --\u003e|w₂₂| h2 h1 --\u003e|v₁| y((ŷ)) h2 --\u003e|v₂| y Each arrow has a weight — a number that controls how strongly one neuron influences the next. Learning means finding the right weight values.\nStep 1: The Forward Pass (Making a Prediction) # Before we can learn from mistakes, we need to make a mistake. The forward pass pushes data through the network:\nHidden layer — each hidden neuron computes a weighted sum and passes it through an activation function \\(\\sigma\\) (like the sigmoid function):\n$$a_j = \\sum_i w_{ij} \\cdot x_i$$ $$h_j = \\sigma(a_j) = \\frac{1}{1 + e^{-a_j}}$$\nOutput layer — same process:\n$$b = \\sum_j v_j \\cdot h_j$$ $$\\hat{y} = \\sigma(b)$$\nError — how wrong were we?\n$$E = \\frac{1}{2}(y - \\hat{y})^2$$\nThink of the forward pass as water flowing downstream: input → hidden → output → error. Simple, one-directional.\nStep 2: The Backward Pass (Assigning Blame) # Now the magic. We know the final error, but we need to figure out how to adjust every single weight in the network to reduce that error. We work backward.\nOutput Layer: Direct Blame # The output layer error signal combines two things:\n$$\\delta_{out} = (y - \\hat{y}) \\cdot \\sigma\u0026rsquo;(b)$$\n\\((y - \\hat{y})\\): How wrong is the prediction? \\(\\sigma\u0026rsquo;(b)\\): How sensitive is the output neuron to changes? Think of it as: \u0026ldquo;How wrong × How adjustable = How much to change\u0026rdquo;\nThen we update the output weights:\n$$\\Delta v_j = \\eta \\cdot \\delta_{out} \\cdot h_j$$\nHidden Layer: Proportional Blame # Here\u0026rsquo;s the core insight of backpropagation. Hidden neurons don\u0026rsquo;t have their own \u0026ldquo;correct answer\u0026rdquo; — we only know the final output was wrong. So we distribute blame proportionally to each connection\u0026rsquo;s strength:\n$$\\delta_j = \\delta_{out} \\cdot v_j \\cdot \\sigma\u0026rsquo;(a_j)$$\nBreaking this down:\n\\(\\delta_{out}\\): The error signal from the output \\(v_j\\): The weight connecting hidden neuron \\(j\\) to the output — a stronger connection means more responsibility \\(\\sigma\u0026rsquo;(a_j)\\): How sensitive is hidden neuron \\(j\\) to changes? Then we update the hidden weights:\n$$\\Delta w_{ij} = \\eta \\cdot \\delta_j \\cdot x_i$$\nThe Information Flow # graph RL E[\"Error E\"] --\u003e|\"(y-ŷ) · σ'(b)\"| dout[\"δ_out\"] dout --\u003e|\"× h_j\"| dv[\"Update v weights\"] dout --\u003e|\"× v_j · σ'(a_j)\"| dh[\"δ_hidden\"] dh --\u003e|\"× x_i\"| dw[\"Update w weights\"] Error flows backward through the network. At each layer, it gets split and scaled according to the connection strengths. The chain rule from calculus is what makes this mathematically precise.\nThe Chain Rule: Why It All Works # Backpropagation is really just the chain rule from calculus, applied cleverly. If you want to know how changing a weight \\(w\\) deep inside the network affects the final error \\(E\\), you multiply the local effects at each step:\n$$\\frac{\\partial E}{\\partial w} = \\frac{\\partial E}{\\partial \\hat{y}} \\cdot \\frac{\\partial \\hat{y}}{\\partial b} \\cdot \\frac{\\partial b}{\\partial h} \\cdot \\frac{\\partial h}{\\partial a} \\cdot \\frac{\\partial a}{\\partial w}$$\nEach factor is a local gradient — how much one thing affects the next thing in line. Multiply them all together, and you know how the distant weight affects the final error.\nIt\u0026rsquo;s like a chain of dominoes: knocking over the first one (changing \\(w\\)) causes a cascade that eventually reaches the last one (changing \\(E\\)). The chain rule tells you exactly how hard that last domino falls.\nA Concrete Example with Numbers # Let\u0026rsquo;s run through a complete forward and backward pass.\nSetup:\nInputs: \\(x_1 = 0.5\\), \\(x_2 = 0.3\\) Hidden weights: \\(w_{11} = 0.4\\), \\(w_{21} = 0.2\\), \\(w_{12} = 0.3\\), \\(w_{22} = 0.5\\) Output weights: \\(v_1 = 0.6\\), \\(v_2 = 0.4\\) Target: \\(y = 1\\), Learning rate: \\(\\eta = 0.5\\) Forward Pass # Hidden neuron 1: $$a_1 = 0.4 \\times 0.5 + 0.2 \\times 0.3 = 0.26 \\quad \\Rightarrow \\quad h_1 = \\sigma(0.26) \\approx 0.565$$\nHidden neuron 2: $$a_2 = 0.3 \\times 0.5 + 0.5 \\times 0.3 = 0.30 \\quad \\Rightarrow \\quad h_2 = \\sigma(0.30) \\approx 0.574$$\nOutput: $$b = 0.6 \\times 0.565 + 0.4 \\times 0.574 = 0.569 \\quad \\Rightarrow \\quad \\hat{y} = \\sigma(0.569) \\approx 0.638$$\nError: \\(E = \\frac{1}{2}(1 - 0.638)^2 \\approx 0.065\\)\nWe predicted 0.638 but the target is 1. Time to learn!\nBackward Pass # Output error signal: $$\\delta_{out} = (1 - 0.638) \\times 0.638 \\times (1 - 0.638) \\approx 0.083$$\nOutput weight updates: $$\\Delta v_1 = 0.5 \\times 0.083 \\times 0.565 \\approx 0.024$$ $$\\Delta v_2 = 0.5 \\times 0.083 \\times 0.574 \\approx 0.024$$\nHidden error signals (this is where blame gets distributed): $$\\delta_1 = 0.083 \\times 0.6 \\times 0.565 \\times (1 - 0.565) \\approx 0.012$$ $$\\delta_2 = 0.083 \\times 0.4 \\times 0.574 \\times (1 - 0.574) \\approx 0.008$$\nNotice that \\(\\delta_1 \u0026gt; \\delta_2\\) because \\(v_1 = 0.6 \u0026gt; v_2 = 0.4\\) — the neuron with the stronger connection to the output gets more blame!\nHidden weight updates: $$\\Delta w_{11} = 0.5 \\times 0.012 \\times 0.5 \\approx 0.003$$\nThe Key Observation # Weight Update Size Distance from Output \\(v\\) (output) ~0.024 1 layer away \\(w\\) (hidden) ~0.003 2 layers away The updates are 10x smaller for the hidden layer. Every layer the error travels through shrinks it further. This foreshadows a serious problem\u0026hellip;\nThe Vanishing Gradient Problem: A Game of Telephone # Remember the game of Telephone (Chinese Whispers)? A message gets distorted as it passes through more people. The same thing happens with error signals in deep networks.\nThe sigmoid function\u0026rsquo;s derivative has a maximum value of just 0.25. Every layer the error signal passes through, it gets multiplied by a number less than 0.25. In a deep network with many layers:\n$$\\text{Gradient} \\propto \\underbrace{0.25 \\times 0.25 \\times \\cdots \\times 0.25}_{N \\text{ layers}}$$\nFor a 100-layer network: \\(0.5^{100} \\approx 7.89 \\times 10^{-31}\\)\nThat\u0026rsquo;s essentially zero. The early layers receive virtually no error signal, so they can\u0026rsquo;t learn at all. This is called the vanishing gradient problem, and it\u0026rsquo;s why deep neural networks didn\u0026rsquo;t work well for decades.\nWhy Deep Learning Finally Worked (~2012) # The vanishing gradient problem was a show-stopper until researchers found clever solutions:\n[2010] **ReLU Activation** replaces Sigmoid. Instead of squishing everything through the S-shaped sigmoid curve (max derivative 0.25), ReLU simply passes positive values through unchanged (derivative = 1). No more vanishing gradients. [2015] **Residual Connections (ResNets)** add \"skip connections.\" These create shortcuts that let the gradient flow directly to earlier layers, bypassing the shrinkage problem entirely. This enabled networks with 100+ layers. [2015] **Batch Normalization** stabilizes training. By normalizing the inputs to each layer, it prevents the gradients from getting too large or too small, keeping everything in a healthy range. These breakthroughs (along with powerful GPUs and massive datasets) are why deep learning exploded in the 2010s and now powers everything from language models to image generators.\nKey Takeaways # Backpropagation distributes blame for errors backward through a network, so every weight knows how to adjust It works by applying the chain rule — multiplying local gradients at each layer Weights with stronger connections get more blame (and bigger updates) The vanishing gradient problem caused error signals to shrink to nothing in deep networks Modern fixes like ReLU, residual connections, and batch normalization solved this, enabling the deep learning revolution Study Notes: This post is based on my personal notes from studying Machine Learning (CS7641) at Georgia Institute of Technology. The concepts are explained in my own words for learning purposes. ","date":"20 February 2026","externalUrl":null,"permalink":"/posts/ml-backpropagation/","section":"Posts","summary":" When a factory produces a defective product, how do you trace the problem back through the assembly line to find which worker made the mistake? Neural networks face the exact same challenge. They have layers of “workers” (neurons), and when the final output is wrong, they need to figure out who’s responsible — and by how much. The algorithm that solves this is called backpropagation, and it’s the reason deep learning works at all. Neural Networks Are Everywhere # Before we dive into how neural networks learn, let’s appreciate what they do. The phone in your pocket uses neural networks for face recognition, voice transcription, photo enhancement, and text prediction. Self-driving cars, medical image analysis, language translation — all neural networks.\n","title":"How Neural Networks Learn from Mistakes: Backpropagation Explained","type":"posts"},{"content":"","date":"20 February 2026","externalUrl":null,"permalink":"/tags/machine-learning/","section":"Tags","summary":"","title":"Machine-Learning","type":"tags"},{"content":"","date":"20 February 2026","externalUrl":null,"permalink":"/series/ml-fundamentals/","section":"Series","summary":"","title":"ML Fundamentals","type":"series"},{"content":"","date":"20 February 2026","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","date":"20 February 2026","externalUrl":null,"permalink":"/categories/study-notes/","section":"Categories","summary":"","title":"Study Notes","type":"categories"},{"content":"","date":"20 February 2026","externalUrl":null,"permalink":"/tags/study-notes/","section":"Tags","summary":"","title":"Study-Notes","type":"tags"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/ghostty/","section":"Tags","summary":"","title":"Ghostty","type":"tags"},{"content":" After years of refining my terminal workflow, I\u0026rsquo;ve landed on a stack I genuinely enjoy using every day: Ghostty as the terminal emulator, tmux with sesh for session management, and Neovim with LazyVim for editing. Everything runs on macOS (Apple Silicon) with a consistent Catppuccin Mocha theme across all tools.\nHere\u0026rsquo;s a deep dive into how it all fits together.\nGhostty: The Terminal Emulator # I switched to Ghostty from Kitty a while back and haven\u0026rsquo;t looked back. It\u0026rsquo;s written in Zig , GPU-accelerated, and buttery smooth.\nMy key configuration choices:\n# ~/.config/ghostty/config theme = catppuccin-mocha font-family = \u0026#34;Hack Nerd Font Mono\u0026#34; font-size = 14 background-opacity = 0.75 background-blur-radius = 20 The 75% transparency with 20px blur gives a nice frosted-glass look where my desktop wallpaper bleeds through — purely aesthetic, but it makes long coding sessions more pleasant.\nQuake-style Drop-down Terminal # Pro tip: This is my most-used keybinding. It slides a terminal down from the top of the screen for quick commands without leaving your current context. keybind = global:super+grave_accent=toggle_quick_terminal quick-terminal-position = top quick-terminal-screen = main quick-terminal-animation-duration = 0.1 I use it constantly for git operations, running tests, and quick file checks.\nSplit Panes # keybind = super+shift+d=new_split:right keybind = super+d=new_split:down keybind = super+shift+enter=toggle_split_zoom These mirror my tmux bindings closely, so muscle memory works regardless of whether I\u0026rsquo;m in a tmux session or not.\ntmux + sesh: Session Management # tmux is the backbone of my workflow. I use Ctrl+A as my prefix (classic screen users will relate) and sesh for smart session management.\nCore Configuration # # Prefix set -g prefix C-a unbind C-b bind C-a send-prefix # Start windows and panes at 1, not 0 set -g base-index 1 setw -g pane-base-index 1 # Mouse support set -g mouse on # True color support set -g default-terminal \u0026#34;tmux-256color\u0026#34; set -ag terminal-overrides \u0026#34;,xterm-256color:RGB\u0026#34; The sesh Session Picker # The killer feature: Press Prefix + T to get an FZF-powered session picker with instant access to all tmux sessions, zoxide directories, and config paths — all fuzzy-searchable. bind-key \u0026#34;T\u0026#34; run-shell \u0026#34;sesh connect \\\u0026#34;$( sesh list | fzf-tmux -p 55%,60% \\ --no-sort --border-label \u0026#39; sesh \u0026#39; \\ --prompt \u0026#39;\u0026gt; \u0026#39; \\ --header \u0026#39; ^a all ^t tmux ^g configs ^x zoxide ^d tmux kill ^f find\u0026#39; \\ --bind \u0026#39;tab:down,btab:up\u0026#39; \\ --bind \u0026#39;ctrl-a:change-prompt(\u0026gt; )+reload(sesh list)\u0026#39; \\ --bind \u0026#39;ctrl-t:change-prompt(\u0026gt; )+reload(sesh list -t)\u0026#39; \\ --bind \u0026#39;ctrl-g:change-prompt(\u0026gt; )+reload(sesh list -c)\u0026#39; \\ --bind \u0026#39;ctrl-x:change-prompt(\u0026gt; )+reload(sesh list -z)\u0026#39; \\ --bind \u0026#39;ctrl-f:change-prompt(\u0026gt; )+reload(fd -H -d 2 -t d -E .Trash . ~)\u0026#39; \\ --bind \u0026#39;ctrl-d:execute(tmux kill-session -t {})+change-prompt(\u0026gt; )+reload(sesh list)\u0026#39; )\\\u0026#34;\u0026#34; I can jump between projects in under a second.\nvim-tmux-navigator # For seamless navigation between tmux panes and Neovim splits, I use vim-tmux-navigator. Ctrl+h/j/k/l moves between panes regardless of whether they\u0026rsquo;re tmux panes or Neovim windows.\nSession Persistence # # tmux-resurrect + tmux-continuum set -g @resurrect-capture-pane-contents \u0026#39;on\u0026#39; set -g @continuum-restore \u0026#39;on\u0026#39; set -g @continuum-save-interval \u0026#39;15\u0026#39; Sessions survive restarts. I never lose my workspace layout.\nNeovim with LazyVim # I run Neovim via bob (a Neovim version manager) and use LazyVim as my configuration framework.\nLazyVim Extras # I have 19 extras enabled, including:\nclaudecode copilot Docker Go Java Python Rust TypeScript harpoon2 treesitter-context Key Customizations # -- Transparent background to match Ghostty { \u0026#34;catppuccin/nvim\u0026#34;, opts = { transparent_background = true, }} -- Smooth scrolling { \u0026#34;karb94/neoscroll.nvim\u0026#34;, opts = { duration_multiplier = 0.4, easing = \u0026#34;linear\u0026#34;, }} -- Global statusline vim.opt.laststatus = 3 vim.opt.scrolloff = 10 vim.opt.cmdheight = 0 Claude Code Integration # I recently added keybindings for Claude Code directly in Neovim:\n-- Toggle Claude Code terminal vim.keymap.set(\u0026#34;n\u0026#34;, \u0026#34;\u0026lt;leader\u0026gt;ac\u0026#34;, function() require(\u0026#34;snacks\u0026#34;).terminal.toggle(\u0026#34;claude\u0026#34;, { win = { style = \u0026#34;terminal\u0026#34; }}) end, { desc = \u0026#34;Claude Code Toggle\u0026#34; }) -- Send visual selection to Claude vim.keymap.set(\u0026#34;v\u0026#34;, \u0026#34;\u0026lt;leader\u0026gt;as\u0026#34;, function() -- Yanks selection and sends to Claude terminal end, { desc = \u0026#34;Claude Code Send Selection\u0026#34; }) This lets me highlight code, send it to Claude for analysis, and apply suggestions — all without leaving the editor.\nThe Full Flow # Launch Ghostty Step 1 Transparent, GPU-accelerated terminal appears with frosted-glass effect. sesh Session Picker Step 2 Press Prefix + T — fuzzy-search for a project across tmux sessions, zoxide dirs, and configs. tmux Session Loads Step 3 Saved layout restores: editor on the left, terminal on the right. Neovim Opens Step 4 LazyVim with treesitter highlighting, LSP, and AI assistance ready to go. Navigate Freely Step 5 Ctrl+h/j/k/l moves between Neovim and tmux panes seamlessly. Cmd+backtick for one-off commands. Everything shares the same Catppuccin Mocha color scheme, so the visual experience is cohesive. The transparent backgrounds mean my terminal, editor, and status bars all blend together.\nWrapping Up # This setup has evolved over many iterations. The key principles:\nConsistency — Same theme, same keybindings philosophy across all tools Speed — Every action should take less than a second Integration — Tools should talk to each other (vim-tmux-navigator, sesh + zoxide, etc.) All my configs are managed with yadm and available on GitHub:\nnickboy/dotfiles Dotfiles Shell 0 0 ","date":"15 February 2026","externalUrl":null,"permalink":"/posts/my-terminal-setup-2026/","section":"Posts","summary":" After years of refining my terminal workflow, I’ve landed on a stack I genuinely enjoy using every day: Ghostty as the terminal emulator, tmux with sesh for session management, and Neovim with LazyVim for editing. Everything runs on macOS (Apple Silicon) with a consistent Catppuccin Mocha theme across all tools.\n","title":"My Terminal Setup in 2026: Ghostty, tmux, and Neovim","type":"posts"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/neovim/","section":"Tags","summary":"","title":"Neovim","type":"tags"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/tmux/","section":"Tags","summary":"","title":"Tmux","type":"tags"},{"content":"","date":"1 February 2026","externalUrl":null,"permalink":"/tags/cli/","section":"Tags","summary":"","title":"Cli","type":"tags"},{"content":" I\u0026rsquo;ve been gradually replacing classic Unix tools with modern alternatives, mostly written in Rust . After a year of daily use, these aren\u0026rsquo;t experiments anymore — they\u0026rsquo;re muscle memory.\nThe Replacements # Classic Modern Why cat bat Syntax highlighting, line numbers, git integration ls eza Icons, git status, tree view, color-coded grep ripgrep 10x faster, respects .gitignore, smart case find fd Simpler syntax, respects .gitignore, colored output cd zoxide Learns your habits, fuzzy matching sed sd Intuitive regex syntax, no escaping nightmare du dust Visual directory size with a tree view df duf Colorful, filterable disk usage top btop Beautiful TUI with mouse support, per-core graphs ps procs Colorized, searchable, tree view history atuin Encrypted sync, full-text search, workspace filtering Setting Up Aliases # In my .zshrc, I alias the classics to their replacements so the transition is invisible:\nalias cat=\u0026#34;bat\u0026#34; alias vim=\u0026#34;nvim\u0026#34; alias vi=\u0026#34;nvim\u0026#34; alias top=\u0026#34;btop\u0026#34; alias du=\u0026#34;dust\u0026#34; alias df=\u0026#34;duf\u0026#34; alias ps=\u0026#34;procs\u0026#34; For eza, I use a Zinit plugin that configures it with sensible defaults:\nzinit light z-shell/zsh-eza # Gives me: ls, ll, la, lt all backed by eza with --git, --icons, --group-directories-first Deep Dive: The Tools That Matter Most # bat — cat with Wings # bat isn\u0026rsquo;t just cat with colors. I use it as:\nA pager for other tools (git diff, man pages) A preview engine for FZF A syntax highlighter for Atuin\u0026rsquo;s history preview # Use bat as the FZF preview export FZF_CTRL_T_OPTS=\u0026#34;--preview \u0026#39;bat -n --color=always --line-range :500 {}\u0026#39;\u0026#34; sharkdp/bat A cat(1) clone with wings. Rust 57776 1492 ripgrep — The Search Engine # ripgrep is absurdly fast. My .ripgreprc configures smart defaults:\n--smart-case # Case-insensitive unless you use uppercase --follow # Follow symlinks --hidden # Search hidden files --max-filesize=100M I also define custom file type groups:\n--type-add=web:*.{html,css,js,ts,jsx,tsx,vue,svelte} --type-add=config:*.{json,yaml,yml,toml,ini,conf} --type-add=shell:*.{sh,bash,zsh,fish} Then I can search only web files: rg \u0026quot;useState\u0026quot; --type web.\nPower move: rgf — ripgrep piped into FZF with a preview, opening results in Neovim. rgf() { rg --color=always --line-number --no-heading \u0026#34;$@\u0026#34; | fzf --ansi --delimiter \u0026#39;:\u0026#39; \\ --preview \u0026#39;bat --color=always {1} --highlight-line {2}\u0026#39; \\ --preview-window \u0026#39;up,60%,border-bottom,+{2}+3/3,~3\u0026#39; | awk -F: \u0026#39;{print \u0026#34;+\u0026#34;$2, $1}\u0026#39; | xargs -r nvim } BurntSushi/ripgrep ripgrep recursively searches directories for a regex pattern while respecting your gitignore Rust 61210 2438 zoxide — cd That Learns # After using zoxide for a few months, I can\u0026rsquo;t go back. It learns which directories you visit most:\nz work # jumps to ~/workspace (most visited match) z dot # jumps to ~/dotfiles zi # interactive mode with FZF I integrate it with FZF for even more power:\neval \u0026#34;$(zoxide init zsh --hook pwd)\u0026#34; fd — find for Humans # # find hidden .env files, ignoring node_modules find . -name \u0026#34;.env\u0026#34; -not -path \u0026#34;*/node_modules/*\u0026#34; # same thing fd .env fd respects .gitignore by default and is the engine behind my FZF file finder:\nexport FZF_DEFAULT_COMMAND=\u0026#39;fd --type f --strip-cwd-prefix --hidden --follow --exclude .git\u0026#39; FZF: The Glue That Binds Everything # FZF isn\u0026rsquo;t a replacement — it\u0026rsquo;s an amplifier. It makes every other tool interactive.\nfzf-tab: Tab Completion on Steroids # zinit light Aloxaf/fzf-tab # Preview files and directories during tab completion zstyle \u0026#39;:fzf-tab:complete:cd:*\u0026#39; fzf-preview \u0026#39;eza -1 --color=always $realpath\u0026#39; zstyle \u0026#39;:fzf-tab:complete:cat:*\u0026#39; fzf-preview \u0026#39;bat --color=always $realpath\u0026#39; Now when I type cd \u0026lt;TAB\u0026gt;, I get a fuzzy searchable list with directory previews. Same for cat, vim, and any other command.\nGit Integration # # Fuzzy branch switcher gb() { git branch --all --sort=-committerdate | fzf --preview \u0026#39;git log --oneline --graph --color=always {1}\u0026#39; | sed \u0026#39;s/remotes\\/origin\\///\u0026#39; | xargs git checkout } Catppuccin Theme for FZF # Everything gets the same color treatment:\nexport FZF_DEFAULT_OPTS=\u0026#34; \\ --color=bg+:#313244,bg:#1e1e2e,spinner:#f5e0dc,hl:#f38ba8 \\ --color=fg:#cdd6f4,header:#f38ba8,info:#cba6f7,pointer:#f5e0dc \\ --color=marker:#b4befe,fg+:#cdd6f4,prompt:#cba6f7,hl+:#f38ba8 \\ --color=selected-bg:#45475a\u0026#34; Atuin: Shell History Reimagined # Atuin replaces the basic shell history with a SQLite-backed, encrypted, cross-machine synced history.\nKey features I rely on:\n# ~/.config/atuin/config.toml enter_accept = true # Execute on Enter, not just select keymap_mode = \u0026#34;auto\u0026#34; # Respects my Zsh vi-mode workspaces = true # Filter by git repo when pressing Up filter_mode_shell_up_key_binding = \u0026#34;workspace\u0026#34; style = \u0026#34;compact\u0026#34; show_preview = true Killer feature: Workspace filtering. When I press Up in a git repo, I only see commands I ran in that repo. No more scrolling through unrelated history. The secrets filter automatically strips AWS keys, GitHub tokens, and passwords from history:\nsecrets_filter = true history_filter = [ \u0026#34;^export (AWS|GITHUB|SLACK|TOKEN|SECRET|PASSWORD|KEY)\u0026#34;, ] Starship: The Prompt # Starship gives me a fast, informative prompt with minimal configuration. My setup uses Catppuccin Mocha colors with status-bar-style background segments:\nOS icon + username Current directory (truncated to 3 levels, with custom icons for ~/Documents, ~/Developer, etc.) Git branch + status (ahead/behind/dirty indicators) Active language versions (Python, Go, Rust, Node — only shown when relevant) Command duration (only if \u0026gt; 45 seconds) Installation # All of these tools are in my Brewfile — one brew bundle and you\u0026rsquo;re set. # Modern CLI replacements brew \u0026#34;bat\u0026#34; brew \u0026#34;eza\u0026#34; brew \u0026#34;ripgrep\u0026#34; brew \u0026#34;fd\u0026#34; brew \u0026#34;fzf\u0026#34; brew \u0026#34;zoxide\u0026#34; brew \u0026#34;sd\u0026#34; brew \u0026#34;dust\u0026#34; brew \u0026#34;duf\u0026#34; brew \u0026#34;btop\u0026#34; brew \u0026#34;procs\u0026#34; brew \u0026#34;atuin\u0026#34; brew \u0026#34;starship\u0026#34; Worth the Switch? # Absolutely. The common thread across all these tools:\nBetter defaults — they do what you want without flags Respect .gitignore — no more accidentally grepping node_modules Colored output — easier to scan visually Speed — most are written in Rust and noticeably faster on large codebases The transition cost is low — alias the classics, and you\u0026rsquo;ll barely notice the switch. But you\u0026rsquo;ll definitely notice when you\u0026rsquo;re on a machine without them.\nFull configs in my dotfiles repo:\nnickboy/dotfiles Dotfiles Shell 0 0 ","date":"1 February 2026","externalUrl":null,"permalink":"/posts/modern-cli-tools-replacing-unix-classics/","section":"Posts","summary":" I’ve been gradually replacing classic Unix tools with modern alternatives, mostly written in Rust . After a year of daily use, these aren’t experiments anymore — they’re muscle memory.\n","title":"Modern CLI Tools That Replaced My Unix Classics","type":"posts"},{"content":"","date":"1 February 2026","externalUrl":null,"permalink":"/tags/rust/","section":"Tags","summary":"","title":"Rust","type":"tags"},{"content":"","date":"1 February 2026","externalUrl":null,"permalink":"/tags/tools/","section":"Tags","summary":"","title":"Tools","type":"tags"},{"content":"","date":"15 January 2026","externalUrl":null,"permalink":"/tags/devops/","section":"Tags","summary":"","title":"Devops","type":"tags"},{"content":"","date":"15 January 2026","externalUrl":null,"permalink":"/tags/dotfiles/","section":"Tags","summary":"","title":"Dotfiles","type":"tags"},{"content":" Every developer eventually reaches the point where their configs become too valuable to lose. Here\u0026rsquo;s how I use yadm to manage my macOS dotfiles with automated testing, daily maintenance, and a pre-commit workflow that keeps everything in check. For me, the turning point was spending a weekend setting up a new MacBook and realizing I couldn\u0026rsquo;t reproduce my environment reliably. That\u0026rsquo;s when I started managing my dotfiles properly.\nAfter trying bare git repos, GNU Stow, and chezmoi, I settled on yadm — and it\u0026rsquo;s been my go-to for over a year.\nWhy Yadm? # There are many dotfile managers. Here\u0026rsquo;s why yadm won:\nTool Approach My Take Bare git Raw git with $HOME as work tree Works but fragile, no extras GNU Stow Symlink farm manager Requires specific directory structure chezmoi Template-based with state management Powerful but complex, uses its own DSL yadm Thin wrapper around git Git-native, minimal learning curve, built-in extras The key insight: yadm is just git. Every git command works — yadm add, yadm commit, yadm push, yadm diff. If you know git, you know yadm. What yadm adds on top:\nAlternate files: Different configs per machine using ##hostname or ##os suffixes Encryption: Encrypt sensitive files with GPG before pushing Bootstrap: Run a setup script on first clone Native $HOME tracking: No symlinks, files live where they belong My Directory Structure # Here\u0026rsquo;s what I track:\n~ ├── .zshrc # Shell config (Zsh + Zinit + Oh-My-Zsh) ├── .tmux.conf # tmux configuration ├── .config/ │ ├── ghostty/config # Ghostty terminal │ ├── kitty/kitty.conf # Kitty terminal (backup) │ ├── nvim/ # Neovim/LazyVim config │ ├── starship.toml # Prompt │ ├── atuin/config.toml # Shell history │ ├── mise/config.toml # Version manager │ └── ripgrep/config # Ripgrep defaults ├── .local/bin/ # Custom scripts ├── .Brewfile # Homebrew packages └── .yadm/ └── hooks/pre-commit # Pre-commit validation The .gitignore is crucial — you want to explicitly track only what you need:\n# Ignore everything by default * # Then selectively un-ignore !.zshrc !.tmux.conf !.config/ghostty/ !.config/nvim/ !.Brewfile # ... etc Automated Daily Maintenance # Homebrew Update Auto brew update \u0026\u0026 brew upgrade \u0026\u0026 brew cleanup — keeps all packages fresh. Zinit Plugins Auto zsh -ic 'zinit update --all' — updates all Zsh plugins. Neovim via bob Auto bob update --all — updates Neovim version manager and builds. LazyVim Sync Auto nvim --headless \"+Lazy! sync\" +qa — syncs all LazyVim plugins. Cleanup Auto Removes broken symlinks in ~/.local/bin. Tracks last run date to prevent duplicates. The script:\nTracks last run date to prevent duplicate runs Catches up if the laptop was off (runs on next login) Has quick aliases: mr (run), ms (status), ml (logs) I also have a control script for managing it:\ndaily-maintenance-control.sh start # Enable auto-run daily-maintenance-control.sh stop # Disable daily-maintenance-control.sh status # Check state daily-maintenance-control.sh logs # View recent logs Pre-Commit Testing # Every yadm commit runs through a pre-commit hook:\n#!/bin/bash # .yadm/hooks/pre-commit # Run the test suite bash ~/test-dotfiles.sh if [ $? -ne 0 ]; then echo \u0026#34;Tests failed. Commit aborted.\u0026#34; exit 1 fi The test suite (test-dotfiles.sh) validates:\nShell syntax ShellCheck Markdown lint YAML lint File permissions No secrets This catches mistakes before they reach the repo. I never push broken configs.\nVersion Management with Mise # Mise (formerly rtx) manages language runtimes across my machines:\n# ~/.config/mise/config.toml [tools] node = \u0026#34;lts\u0026#34; python = \u0026#34;latest\u0026#34; go = \u0026#34;latest\u0026#34; ruby = \u0026#34;latest\u0026#34; [settings] idiomatic_version_file_enable = true # Reads .nvmrc, .python-version, etc. not_found_auto_install = true # Auto-install missing versions jobs = 4 # Parallel installations Key setting: idiomatic_version_file_enable means mise respects .nvmrc, .python-version, and .tool-versions files in project directories. When I cd into a project that needs Node 18, mise automatically activates it. Practical Tips # 1. Start Small # Don\u0026rsquo;t try to track everything at once. Start with:\nyadm add ~/.zshrc yadm add ~/.config/ghostty/config yadm commit -m \u0026#34;initial: shell and terminal config\u0026#34; Add more as you modify things.\n2. Use Branches for Experiments # yadm checkout -b experiment/new-shell-config # Try things out... yadm checkout main # Revert if it didn\u0026#39;t work 3. Bootstrap Script for New Machines # Create a bootstrap that gets a fresh machine to your preferred state:\n#!/bin/bash # ~/.config/yadm/bootstrap # Install Homebrew /bin/bash -c \u0026#34;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\u0026#34; # Install packages brew bundle --file=~/.Brewfile # Set default shell chsh -s $(which zsh) echo \u0026#34;Bootstrap complete. Restart your terminal.\u0026#34; Then on a new machine:\nyadm clone https://github.com/youruser/dotfiles.git yadm bootstrap 4. Keep Sensitive Data Out # Use .gitignore aggressively and yadm\u0026rsquo;s encryption for anything sensitive:\n# Encrypt SSH configs yadm encrypt Git credentials should go through Git Credential Manager, never in dotfiles.\n5. Document Your Setup # I keep a CLAUDE.md in my dotfiles repo — it documents the architecture, conventions, and mandatory rules. This serves as both documentation for myself and instructions for AI assistants helping me modify configs.\nThe Payoff # With this setup:\nNew machine setup — Clone + bootstrap, done in under an hour Daily updates — Automated, zero manual intervention Config changes — Tested before commit, never push broken configs Cross-machine sync — yadm pull on any machine Rollback — Full git history, revert any change The initial investment is a few hours. The ongoing cost is near zero. And the peace of mind knowing your entire development environment is versioned, tested, and reproducible? Priceless.\nCheck out my full setup:\nnickboy/dotfiles Dotfiles Shell 0 0 ","date":"15 January 2026","externalUrl":null,"permalink":"/posts/managing-dotfiles-with-yadm/","section":"Posts","summary":" Every developer eventually reaches the point where their configs become too valuable to lose. Here’s how I use yadm to manage my macOS dotfiles with automated testing, daily maintenance, and a pre-commit workflow that keeps everything in check. For me, the turning point was spending a weekend setting up a new MacBook and realizing I couldn’t reproduce my environment reliably. That’s when I started managing my dotfiles properly.\n","title":"Managing Dotfiles Like a Pro with Yadm","type":"posts"},{"content":"","date":"15 January 2026","externalUrl":null,"permalink":"/tags/yadm/","section":"Tags","summary":"","title":"Yadm","type":"tags"},{"content":" Tzu-Hua (Nick) Liu # Professional Summary # I am a Senior Software Engineer at Meta working on infrastructure for Facebook Feed Ranking. With over 8 years of experience building and scaling distributed systems at major tech companies, my expertise spans backend services, data infrastructure, cloud computing, and system optimization. I have a proven track record of delivering high-impact solutions that process billions of transactions and manage thousands of servers.\nThe Journey # My path into infrastructure engineering wasn\u0026rsquo;t a straight line. I started with network security at Trend Micro in Taipei, moved to the US for grad school, and quickly realized that the problems I loved most were about scale. How do you make systems work when \u0026ldquo;a few users\u0026rdquo; becomes \u0026ldquo;a few billion\u0026rdquo;?\nAt eBay, a data science internship showed me the power of building systems that catch what humans can\u0026rsquo;t. In that case, fraud detection models processing millions of transactions. That\u0026rsquo;s when I understood that the real leverage in engineering isn\u0026rsquo;t writing clever code; it\u0026rsquo;s building systems that compound.\nAWS was where I grew up as an engineer. Five years on the Billing team taught me what it means to build something that absolutely cannot be wrong. When your system handles the financial records for every AWS customer, \u0026ldquo;good enough\u0026rdquo; doesn\u0026rsquo;t exist. I designed the unbilled usage auditor that caught $125,000 in discrepancies and reduced them to $432, a 300x improvement. That project taught me to think in terms of correctness guarantees, not just functionality.\nTwitter was a different kind of challenge: operating at scale under pressure. Managing kernel updates across 5,000+ production servers means you can\u0026rsquo;t afford to break things, but you also can\u0026rsquo;t afford to fall behind. I built the automation and validation tooling that made this possible, and resolved 140+ support tickets in a single on-call week. That taught me as much about prioritization as it did about systems.\nAt Walmart, I shifted to data infrastructure, leading the Data Lake integration for recruiting systems. That meant connecting 10+ data sources through Spark, Kafka, and Hudi pipelines. It was my first time leading an initiative at enterprise scale from design through delivery.\nNow at Meta, I\u0026rsquo;m building the infrastructure that powers Facebook Feed Ranking, the system that decides what 2 billion people see when they open Facebook. It\u0026rsquo;s the intersection of everything I\u0026rsquo;ve worked on: high-performance C++ services, ML infrastructure, and systems that have to work at a scale most engineers never encounter.\nWhat Drives Me # I\u0026rsquo;ve worked at enough companies to know what I care about:\nThe invisible layer. I\u0026rsquo;m drawn to infrastructure, the systems nobody sees until they break. There\u0026rsquo;s something satisfying about building something that billions of people rely on without ever knowing your name. Correctness over cleverness. The AWS billing experience permanently changed how I think. I\u0026rsquo;d rather have a boring system that\u0026rsquo;s reliably correct than an elegant one that\u0026rsquo;s probably correct. Making teams faster. The best infrastructure work isn\u0026rsquo;t about building systems. It\u0026rsquo;s about removing friction for the engineers who depend on your systems. Every automation I build, every tool I create, is measured by how much it unblocks others. Areas of Expertise # Backend \u0026amp; Infrastructure # High-performance backend services in C++ and Go Large-scale distributed system design and optimization ML infrastructure and ranking systems Data Engineering # Real-time stream processing with Apache Spark and Kafka Building data lakes and ETL pipelines Managing TB-scale data operations Cloud \u0026amp; DevOps # AWS services architecture and optimization Kubernetes orchestration and containerization Infrastructure as Code with Terraform and Ansible What I\u0026rsquo;m Working On Now # Outside of my day job at Meta, I\u0026rsquo;m currently:\nPursuing my MS at Georgia Tech — taking courses in ML, deep learning, and reinforcement learning while working full-time Building this site with vibe coding — using Claude Code to explore how AI tools change the way senior engineers work Refining my terminal and dotfiles setup — an ongoing obsession documented in my terminal setup post and managed with yadm Beyond Code # When I\u0026rsquo;m not architecting systems or writing code, I enjoy:\nTinkering with my terminal setup and dotfiles — I\u0026rsquo;ve written multiple posts about my Ghostty + tmux + Neovim workflow and dotfile management with yadm Contributing to open-source projects Mentoring junior engineers — my experience as a TA teaching 180+ students at UH shaped how I think about knowledge sharing Sharing knowledge through technical blog posts Let\u0026rsquo;s Connect # I\u0026rsquo;m always interested in discussing technology, innovation, and opportunities to collaborate on challenging projects.\nEmail Me LinkedIn GitHub Latest Posts # Recent AWS Billing Unbilled Usage Auditor 1 January 0001\u0026middot;276 words\u0026middot;2 mins Designed and built a distributed system that detects unbilled usage across all AWS services — reducing charge discrepancies by 300x and eliminating 230 million monthly false positives. Key Metrics # 300x Reduction in Discrepancies $125,000 → $432 230M False Positives Eliminated ~95% Alert Actionability Architecture # flowchart LR A[\"Usage Records\\n(Billions/day)\"] --\u003e B[\"Smart Sampling\\n\u0026 Aggregation\"] B --\u003e C[\"Multi-Signal\\nValidation\"] C --\u003e D[\"Automated\\nResolution\"] D --\u003e E{Real issue?} E -- Yes --\u003e F[\"Alert with\\nDiagnosis\"] E -- No --\u003e G[\"Auto-resolve\\n\u0026 Log\"] style B fill:#6366f1,color:#fff style C fill:#6366f1,color:#fff style D fill:#6366f1,color:#fff Technical Deep Dive # Aggregation over Brute-Force # Instead of checking every individual usage record (which generated 230M false positives), the system aggregates at the service-account-period level. Twitter Fleet-Scale Kernel Automation 1 January 0001\u0026middot;344 words\u0026middot;2 mins Built the automation and validation tooling to manage kernel updates across 5,000+ production servers at Twitter — with zero-downtime progressive rollouts and automated canary validation. Key Metrics # 5,000+ Production Hosts Weeks → Days Rollout Time 140+ Tickets in One On-Call Week Zero-Downtime Updates Architecture # flowchart LR A[\"New Kernel\\nVersion\"] --\u003e B[\"Canary\\nValidation\"] B --\u003e C[\"Wave 1\\n1% Fleet\"] C --\u003e D[\"Wave 2\\n5% Fleet\"] D --\u003e E[\"Wave 3\\n25% Fleet\"] E --\u003e F[\"Full Fleet\\nRollout\"] C -- anomaly --\u003e G[\"Pause \u0026\\nAuto-Alert\"] D -- anomaly --\u003e G E -- anomaly --\u003e G style B fill:#6366f1,color:#fff style C fill:#6366f1,color:#fff style D fill:#6366f1,color:#fff style E fill:#6366f1,color:#fff style F fill:#6366f1,color:#fff Technical Deep Dive # Validate Before You Roll # A Python library that validates kernel safety before fleet-wide rollout: Building a Knowledge Base That AI Can Actually Use 21 March 2026\u0026middot;1455 words\u0026middot;7 mins Development Environment Obsidian Claude-Code Knowledge-Management Productivity Obsidian + Claude Code is everywhere right now. But pointing an AI at a folder of markdown files and hoping for the best doesn\u0026rsquo;t work. What matters is how you structure the knowledge base. Get that right, and Claude becomes genuinely useful. Get it wrong, and you get confident garbage. There\u0026rsquo;s been a wave of posts about this combo lately: James Bedford\u0026rsquo;s full walkthrough, Greg Isenberg\u0026rsquo;s \u0026ldquo;personal OS\u0026rdquo; approach, kepano (Obsidian\u0026rsquo;s CEO) sharing Claude Skills. They\u0026rsquo;re all worth reading. ","externalUrl":null,"permalink":"/about/","section":"About Me","summary":"Tzu-Hua (Nick) Liu # Professional Summary # I am a Senior Software Engineer at Meta working on infrastructure for Facebook Feed Ranking. With over 8 years of experience building and scaling distributed systems at major tech companies, my expertise spans backend services, data infrastructure, cloud computing, and system optimization. I have a proven track record of delivering high-impact solutions that process billions of transactions and manage thousands of servers.\n","title":"About Me","type":"about"},{"content":" Georgia Institute of Technology 2023 - Present Master of Science in Computer Science · Atlanta, GA Focus Areas\nComputing Systems: Operating systems, compilers, and systems architecture Machine Learning: Statistical learning, deep learning, and reinforcement learning University of Houston Aug 2013 - May 2017 PhD Candidate in Computer Science · Houston, TX Research Focus\nDistributed Computing: MapReduce optimization and parallel processing algorithms Mobile Security: Android application security analysis and vulnerability detection Big Data Systems: Scalable data processing frameworks and performance optimization Teaching Experience\nAs a Teaching Assistant, I had the privilege of educating and mentoring over 180 students across multiple core computer science courses:\nData Structures and Algorithms (28 students) Taught fundamental algorithms and complexity analysis Guided students through hands-on programming assignments Computer Architecture (78 students) Explained CPU design, memory hierarchy, and instruction sets Supervised lab sessions on assembly programming Software Design (40 students) Introduced design patterns and software architecture principles Mentored team projects using agile methodologies Introduction to Computer Science (35 students) Taught programming fundamentals in Python and Java Helped students develop problem-solving skills Research Contributions\nPublished papers on MapReduce optimization techniques Developed security analysis tools for Android applications Contributed to open-source distributed computing projects National Taiwan University Sept 2010 - June 2013 Master of Science in Computer Science · Taipei, Taiwan Specialization Areas\nDistributed Systems: Consensus algorithms, fault tolerance, and system design Computer Security: Cryptography, network security, and secure coding practices Cloud Computing: Virtualization technologies and cloud architecture Academic Achievements\nGraduate Research Assistant in the Distributed Systems Lab Thesis: \u0026ldquo;Efficient Resource Allocation in Cloud Computing Environments\u0026rdquo; GPA: 3.8/4.0 Key Coursework\nAdvanced Operating Systems Distributed Database Systems Network Security Machine Learning Parallel Computing Continuous Learning # Beyond formal education, I maintain a commitment to continuous learning through:\nOnline Courses: Coursera, Udacity, and edX certifications Technical Conferences: Regular attendance at AWS re:Invent, KubeCon, and Spark Summit Professional Development: AWS certification programs and specialized training Community Involvement: Local tech meetups and hackathons ","externalUrl":null,"permalink":"/education/","section":"Education","summary":" Georgia Institute of Technology 2023 - Present Master of Science in Computer Science · Atlanta, GA Focus Areas\nComputing Systems: Operating systems, compilers, and systems architecture Machine Learning: Statistical learning, deep learning, and reinforcement learning University of Houston Aug 2013 - May 2017 PhD Candidate in Computer Science · Houston, TX Research Focus\n","title":"Education","type":"education"},{"content":" Distributed Systems ML Infrastructure C++ Python Go Java AWS Kafka Spark Learn More About Me Read My Blog ","externalUrl":null,"permalink":"/","section":"Nick Liu","summary":" Distributed Systems ML Infrastructure C++ Python Go Java AWS Kafka Spark Learn More About Me Read My Blog ","title":"Nick Liu","type":"page"},{"content":" A mix of open source projects and key professional contributions across distributed systems, data infrastructure, and developer tooling. Open Source # nickboy/dotfiles Dotfiles Shell 0 0 nickboy/nickboy.github.io Personal website for Nick TypeScript 0 0 Professional Highlights # FB Feed Ranking Infrastructure # C++ ML Infrastructure Ranking Systems Meta Building the infrastructure that powers Facebook\u0026rsquo;s Feed Ranking — high-performance backend services in C++ that support ML engineers in delivering better ranking models. Working at the intersection of infrastructure and product, optimizing how billions of users see their Facebook Timeline.\nAWS Billing Unbilled Usage Auditor # Java DynamoDB AWS Lambda Distributed Systems Designed and built a system that detects unbilled usage across AWS services. Reduced false positives by 230 million monthly transactions and cut charge discrepancies by 300x — from $125,000 to $432.\nView Project Details Read the Story Twitter Fleet Kernel Automation # Python Go Redis Linux Kernel Standardized hardware and software configurations for 5,000+ production hosts. Built a Python library for Canary Kernel Validation and designed automation for kernel updates at fleet scale. Resolved 140+ tickets in a single on-call week.\nView Project Details Read the Story Walmart Data Lake Integration # Apache Spark Kafka Apache Hudi Big Data Led enterprise-wide Data Lake integration for recruiting data pipelines, processing data from 10+ sources using Apache Spark, Kafka, and Apache Hudi. Architected scalable solutions for real-time data processing at enterprise scale.\nTechnical Stack # C++ Python Go Java TypeScript Rust Apache Spark Kafka Kubernetes Docker AWS Terraform Redis DynamoDB View Full Experience ","externalUrl":null,"permalink":"/projects/","section":"Projects","summary":" A mix of open source projects and key professional contributions across distributed systems, data infrastructure, and developer tooling. Open Source # nickboy/dotfiles Dotfiles ","title":"Projects","type":"projects"},{"content":" Technical Skills # Programming Languages Expert Level # Python - 7+ years of production experience, used for data pipelines, automation, and ML Java - Enterprise applications, Spring framework, microservices architecture JavaScript/TypeScript - Full-stack development, React, Node.js Proficient # Go - High-performance services, gRPC implementations Scala - Apache Spark applications, functional programming SQL - Complex queries, optimization, stored procedures Bash/Shell - Automation scripts, DevOps tooling Familiar # C++ - System programming, performance-critical applications C# - .NET applications, Windows services PHP - Web applications, Laravel framework Ruby - Scripting, Rails applications Big Data \u0026amp; Analytics Data Processing # Apache Spark - Large-scale data processing, ETL pipelines Apache Kafka - Real-time streaming, event-driven architecture Apache Hudi - Data lake management, incremental processing Hadoop - HDFS, MapReduce, ecosystem tools Machine Learning # Scikit-Learn - Classical ML algorithms, model training XGBoost - Gradient boosting, fraud detection models Pandas/NumPy - Data analysis, feature engineering TensorFlow - Deep learning experiments Cloud \u0026amp; Infrastructure AWS Services # Compute: EC2, Lambda, ECS, Fargate Storage: S3, EBS, EFS Database: RDS, DynamoDB, ElastiCache Analytics: EMR, Athena, Kinesis Networking: VPC, CloudFront, Route 53 DevOps: CloudFormation, CodePipeline, CloudWatch Container \u0026amp; Orchestration # Docker - Containerization, multi-stage builds Kubernetes - Container orchestration, Helm charts Mesos/Aurora - Large-scale cluster management Infrastructure as Code # Terraform - Cloud resource provisioning Ansible - Configuration management, automation CloudFormation - AWS-specific IaC Databases SQL Databases # PostgreSQL - Advanced features, performance tuning MySQL - Replication, clustering Oracle - Enterprise features, PL/SQL NoSQL Databases # DynamoDB - Serverless, auto-scaling Redis - Caching, pub/sub, data structures MongoDB - Document store, aggregation framework Cassandra - Wide column store, distributed systems DevOps \u0026amp; Tools CI/CD # GitHub Actions - Workflow automation Jenkins - Pipeline as code GitLab CI - Integrated DevOps platform Monitoring \u0026amp; Observability # Prometheus/Grafana - Metrics and visualization ELK Stack - Log aggregation and analysis New Relic - Application performance monitoring PagerDuty - Incident management Version Control \u0026amp; Collaboration # Git - Advanced workflows, branching strategies GitHub/GitLab - Code review, project management Jira/Confluence - Agile project management Soft Skills Leadership # Technical team leadership Mentoring junior engineers Cross-functional collaboration Strategic planning and execution Communication # Technical documentation Stakeholder management Presentation skills Knowledge sharing Problem Solving # System design and architecture Performance optimization Root cause analysis Data-driven decision making Certifications \u0026amp; Training AWS Certified Solutions Architect (in progress) Continuous learning through online courses and conferences Regular participation in hackathons and tech talks ","externalUrl":null,"permalink":"/skills/","section":"Technical Skills","summary":"Technical Skills # Programming Languages Expert Level # Python - 7+ years of production experience, used for data pipelines, automation, and ML Java - Enterprise applications, Spring framework, microservices architecture JavaScript/TypeScript - Full-stack development, React, Node.js Proficient # Go - High-performance services, gRPC implementations Scala - Apache Spark applications, functional programming SQL - Complex queries, optimization, stored procedures Bash/Shell - Automation scripts, DevOps tooling Familiar # C++ - System programming, performance-critical applications C# - .NET applications, Windows services PHP - Web applications, Laravel framework Ruby - Scripting, Rails applications Big Data \u0026 Analytics Data Processing # Apache Spark - Large-scale data processing, ETL pipelines Apache Kafka - Real-time streaming, event-driven architecture Apache Hudi - Data lake management, incremental processing Hadoop - HDFS, MapReduce, ecosystem tools Machine Learning # Scikit-Learn - Classical ML algorithms, model training XGBoost - Gradient boosting, fraud detection models Pandas/NumPy - Data analysis, feature engineering TensorFlow - Deep learning experiments Cloud \u0026 Infrastructure AWS Services # Compute: EC2, Lambda, ECS, Fargate Storage: S3, EBS, EFS Database: RDS, DynamoDB, ElastiCache Analytics: EMR, Athena, Kinesis Networking: VPC, CloudFront, Route 53 DevOps: CloudFormation, CodePipeline, CloudWatch Container \u0026 Orchestration # Docker - Containerization, multi-stage builds Kubernetes - Container orchestration, Helm charts Mesos/Aurora - Large-scale cluster management Infrastructure as Code # Terraform - Cloud resource provisioning Ansible - Configuration management, automation CloudFormation - AWS-specific IaC Databases SQL Databases # PostgreSQL - Advanced features, performance tuning MySQL - Replication, clustering Oracle - Enterprise features, PL/SQL NoSQL Databases # DynamoDB - Serverless, auto-scaling Redis - Caching, pub/sub, data structures MongoDB - Document store, aggregation framework Cassandra - Wide column store, distributed systems DevOps \u0026 Tools CI/CD # GitHub Actions - Workflow automation Jenkins - Pipeline as code GitLab CI - Integrated DevOps platform Monitoring \u0026 Observability # Prometheus/Grafana - Metrics and visualization ELK Stack - Log aggregation and analysis New Relic - Application performance monitoring PagerDuty - Incident management Version Control \u0026 Collaboration # Git - Advanced workflows, branching strategies GitHub/GitLab - Code review, project management Jira/Confluence - Agile project management Soft Skills Leadership # Technical team leadership Mentoring junior engineers Cross-functional collaboration Strategic planning and execution Communication # Technical documentation Stakeholder management Presentation skills Knowledge sharing Problem Solving # System design and architecture Performance optimization Root cause analysis Data-driven decision making Certifications \u0026 Training AWS Certified Solutions Architect (in progress) Continuous learning through online courses and conferences Regular participation in hackathons and tech talks ","title":"Technical Skills","type":"skills"},{"content":" Meta Dec 2025 - Present Senior Software Engineer · Bellevue, WA Building infrastructure for Facebook Feed Ranking to support ML engineers in delivering better models Developing high-performance backend services in C++ that manage how users see their Facebook Timeline Working at the intersection of infrastructure and product, optimizing ranking systems at massive scale Walmart Global Tech Nov 2024 - Dec 2025 Principal Software Engineer · Remote Led Data Lake integration for recruiting related data across the enterprise Built data pipelines with Apache Spark, Kafka, and Apache Hudi processing data from 10+ sources Architected scalable solutions for enterprise-level real-time data processing Twitter (X) Nov 2022 - Nov 2024 Software Engineer · Remote Standardized hardware and software configurations for 5,000+ production hosts Delivered Python library for Canary Kernel Validation across the fleet Designed automation mechanism for kernel updates at scale Implemented custom commands system for Cache services using Go and Redis Top ticket resolver with 140+ tickets resolved in one on-call week Amazon Web Services July 2017 - Nov 2022 Software Development Engineer II · Seattle, WA Designed and implemented unbilled usage detection mechanism for AWS Billing Built Billing Unbilled Usage Auditor reducing false positives by 230M monthly transactions Reduced charge discrepancies by 300x from $125,000 to $432 Led migration of billing applications to new pricing dependency Started as intern building Billing UI with JavaScript and AngularJS eBay Inc. May 2016 - Aug 2016 Data Science Intern · San Jose, CA Applied XGBoost with Scikit-Learn for fraud detection achieving 71% accuracy Improved fraud detection algorithms using advanced ML techniques on transaction data Trend Micro June 2011 - Aug 2011 Software Engineering Intern · Taipei, Taiwan Implemented routing table maintenance system for parental control products Developed network filtering solutions for consumer security software ","externalUrl":null,"permalink":"/experience/","section":"Work Experience","summary":" Meta Dec 2025 - Present Senior Software Engineer · Bellevue, WA Building infrastructure for Facebook Feed Ranking to support ML engineers in delivering better models Developing high-performance backend services in C++ that manage how users see their Facebook Timeline Working at the intersection of infrastructure and product, optimizing ranking systems at massive scale Walmart Global Tech Nov 2024 - Dec 2025 Principal Software Engineer · Remote Led Data Lake integration for recruiting related data across the enterprise Built data pipelines with Apache Spark, Kafka, and Apache Hudi processing data from 10+ sources Architected scalable solutions for enterprise-level real-time data processing Twitter (X) Nov 2022 - Nov 2024 Software Engineer · Remote Standardized hardware and software configurations for 5,000+ production hosts Delivered Python library for Canary Kernel Validation across the fleet Designed automation mechanism for kernel updates at scale Implemented custom commands system for Cache services using Go and Redis Top ticket resolver with 140+ tickets resolved in one on-call week Amazon Web Services July 2017 - Nov 2022 Software Development Engineer II · Seattle, WA Designed and implemented unbilled usage detection mechanism for AWS Billing Built Billing Unbilled Usage Auditor reducing false positives by 230M monthly transactions Reduced charge discrepancies by 300x from $125,000 to $432 Led migration of billing applications to new pricing dependency Started as intern building Billing UI with JavaScript and AngularJS eBay Inc. May 2016 - Aug 2016 Data Science Intern · San Jose, CA Applied XGBoost with Scikit-Learn for fraud detection achieving 71% accuracy Improved fraud detection algorithms using advanced ML techniques on transaction data Trend Micro June 2011 - Aug 2011 Software Engineering Intern · Taipei, Taiwan Implemented routing table maintenance system for parental control products Developed network filtering solutions for consumer security software ","title":"Work Experience","type":"experience"}]