How I Stopped Overengineering and Learned to Love SSH
TL;DR: I ripped out Kafka and Kubernetes from my distributed system. Replaced them with SSH tunnels, a simple RPC protocol, and text files. Onboarding new nodes takes seconds now, and when something breaks I can actually find the problem. Turns out the tools that can't really break are the ones you end up wanting.
Where This Started
It began as a personal experiment: what would a distributed system for my own infrastructure look like? I wanted my machines to talk to each other without everything falling apart every other week.
The goal was assimilation: the ability to absorb a new node as the environment changes, without requiring a PhD in distributed systems for each addition.
One principle became the compass: start from the most primitive foundations possible.
Consider binary. Zero means no current, one means current. Nothing simpler. Yet from those two states, on and off, everything we've built emerged. That same constraint can produce surprisingly capable systems.
Evolution works the same way. The system with the best fitness function wins, no committee needed, just selection pressure on what actually works. Technology moves fast, and the only way to keep up is staying lean enough to pivot when needed.
So I kept asking: what's the most primitive thing that solves each problem? Machines need to talk, so give them a TCP connection. The connection needs security, so wrap it in an SSH tunnel. Commands need structure, so add an RPC protocol. State needs to persist, so write it to disk.
Universal. Generic. Works on a raspberry pi or a rack server. Works in 1995. Works in 2035.
My Kafka Phase
Kafka wasn't foreign to me, I'd built a proof-of-concept during a technical interview. Consumer groups, partition strategies, the whole apparatus. Impressive machinery. But deploying it here felt like commissioning an oil tanker to cross a river.
I spun up a cluster anyway. Producers, consumers, topics, Zookeeper. It worked, technically speaking.
Then I looked at my VPS. Already running GitLab, my MCP server, a few other services. Kafka's JVM was eyeing that remaining memory like a dog watching you eat dinner. Meanwhile, Vindaloop, my actual compute beast with GPU and storage to spare, sat idle, waiting for work that would never come because the coordinator was choking on its own overhead.
The architecture had become absurd: unlimited compute on one end, bottlenecked by messaging infrastructure at the hub. I'm no Netflix, but I know dysfunction when I see it.
What I Was Actually Trying to Do
There I was at 2 AM, debugging consumer offset nonsense, when the fog cleared. Machine A tells Machine B to do something. Machine B responds with "done" or "failed." That's the whole problem. That's all I needed to solve.
Kafka handles replay, event sourcing, horizontal scaling. But the tasks are idempotent so replay is pointless. Taskwarrior already handles all the history I need. And Vindaloop has headroom for years of growth, so horizontal scaling is a solution to a problem that doesn't exist yet. YAGNI isn't just about features you don't need today, it applies to problems you've already solved through other means.
Once I dropped Kafka, the silence was almost startling. And in that quiet, the answer became obvious.
SSH. Battle-tested since 1995. Encrypted by default. Key-based authentication built in. Port forwarding included. Already installed on every Unix machine I own.
Workers open a reverse tunnel to the hub. The hub connects to localhost to reach each worker's MCP server. SSH already handles encryption and authentication. Autossh keeps tunnels alive when connections drop. And because workers initiate outbound connections, firewalls aren't a problem, no ports to open, no rules to maintain.
Funny how the answer had been sitting in my .ssh folder the entire time, waiting for me to stop overthinking.
MCP: Simple Messages Between Machines
SSH gave me the tunnel. I still needed structure for the messages themselves.
MCP, or Model Context Protocol, originally designed for LLM tool-calling, but really just JSON-RPC with a clean tool abstraction. Request a tool, get a result. Stateless by design.
I was aware of the trap here. Many users who went heavy on MCP servers ended up reverting to vanilla setups, reporting better performance without the overhead. Fits the philosophy: primitive wins.
Each node runs a lightweight MCP server. The hub doesn't care what's behind it, beefy GPU server or a Raspberry Pi running off a phone charger. Same interface regardless.
result = await query_node("hal", "local_exec", 'command: "df -h"')
Adding a new node means: install MCP server, open SSH tunnel, done. Node 4 looks exactly like node 3. The protocol stays constant while the capabilities scale.
State Without the Database Circus
"But where do you store state?"
I almost installed PostgreSQL out of habit. Then I remembered Taskwarrior was already sitting on my machine, doing exactly what I needed.
Taskwarrior is a masterpiece of focused design, a CLI tool that does one thing exceptionally well: manage tasks. No Electron bloat. No subscription fees. No cloud lock-in. Just a binary that reads and writes JSON files. It composes with shell scripts, pipes into other tools, and stores everything as plain text you actually own. Unix philosophy in its purest form.
Plain JSON storage. Syncs to a central server via taskd. Conflict resolution built in. Human-readable, git-friendly.
Here's the trick: tasks aren't just todos anymore. They've become a distributed log.
tw_log("MCP server restarted", "action", "hal")
tw_log("Tunnel reconnected", "info", "vps")
My "database" is text files that sync across nodes. The "job queue" is tasks tagged with +executor, and the audit log is git history. I never had to configure an ORM or run a migration, which is the kind of luxury you don't appreciate until you've lived the alternative.
Taskwarrior TUI: tasks synced across all nodes, filterable, annotatable.
When an LLM agent needs to check the task queue, it runs task +executor and parses stdout. No API client needed. No authentication dance. Just text in, text out.
Making the AI Do the Work
The traditional approach: write a daemon. Python script, loop forever, check queues, dispatch work. Lots of if/else branching. Lots of edge cases to handle.
I chose to make the AI agent the orchestrator.
The agent already knows how to parse requests, break tasks into steps, call tools, handle errors gracefully. Why write a daemon when you can delegate to something that already handles the complexity?
Operator (Telegram): "Check if Hal is online and run the backup"
Agent (hub):
1. query_node("hal", "ping") → online
2. node_exec("hal", "restic backup /data")
3. telegram_alert("Backup complete: 2.3GB")
Human in the loop: the operator stays in control via Telegram while the agent handles execution details.
If Hal is offline, the agent doesn't blindly fail and log an error. It checks why, tries alternatives, alerts with useful context. The agent becomes the distributed system's brain, and the architecture doesn't lock me to any particular model. Today it runs on Claude, tomorrow it could run on Llama or whatever frontier model makes sense. When open-source models catch up, the system updates itself.
Remembering Between Sessions
Agents are stateless by nature. Every conversation starts fresh. That's a problem for systems that need memory.
My solution: text files and git.
Every session ends with update_handoff, documenting what was done, what's pending, what to do next. State changes get state_snapshot, committed with a timestamp.
~/.marklar/
├── HANDOFF.md # Session continuity
├── state.yaml # Current state
└── memories/ # Persistent facts
A cron job prunes state.yaml over time, keeping long-term knowledge while discarding the noise. Structured memory, not accumulated cruft.
What I Learned
Most of it was sitting in front of me the whole time. SSH, git, Taskwarrior were already installed and working. I almost fell into the trap of evaluating tools I didn't need just because tutorials mentioned them.
Text files turned out to scale better than I expected, once machines could read them natively. YAML, Markdown, git history, human-readable, version-controlled for free. Agents parse them directly, execute directly. No translation layer.
And boredom turned out to be a signal. If setting up infrastructure feels tedious, you're probably overcomplicating things. I'm not building for millions of users or investor pitches. I'm building for myself, and there's a specific kind of freedom that comes with that.
What Comes Next
The architecture stays deliberately open-ended. New worker equals new tunnel equals new capabilities.
Current explorations:
- More nodes (Raspberry Pi cluster for edge tasks)
- Smarter state hygiene (auto-pruning, memory consolidation)
- Better failure recovery (autonomous reconnection without intervention)
But honestly? The system already does everything I need. No investors to impress, no SLAs to meet, no uptime dashboards to anxiously refresh. Just my own machines doing what I tell them.
Closing Thoughts
During my internship at Société Générale's investment banking division, I saw what real infrastructure complexity looks like: audit trails, failovers, regulatory compliance, redundancy upon redundancy. That complexity was justified, they're moving billions and regulators are watching every transaction. My home lab is not an investment bank, shockingly.
The gap between "what tutorials teach" and "what you actually need" can be vast. Kubernetes, Kafka, microservices solve coordination at massive scale. I needed assimilation at any scale, the ability to add a new machine in minutes. Different problem, different tools.
Start small, use boring technology, add complexity only when simplicity breaks.
Kafka is still in my toolbox. The moment I need event sourcing or thousands of messages per second, it's ready. That moment hasn't arrived, and might never. The architecture respects SOLID principles and clean domain boundaries, so plugging it in later is just a matter of wiring. The system can ask itself to implement new infrastructure when needs arise. I've been tempted to replicate everything on NixOS for reproducibility, but the needs aren't clear enough yet, it would add more friction than value for assimilation. More interesting experiments to come.
It works. After all the detours, that's really all I wanted.
Built with mass-market hardware, mass-market tools, and the kind of stubbornness that refuses to let simple things become complicated.