Shoebox

Context

The gap between a Go channel and a real broker is enormous. A bare channel is fast but ephemeral — the process dies, the message is gone, and there's no retry or visibility. Kafka gives you durability but drags in a cluster, topic config, and operational overhead that's absurd for "400 leads a night." I'd already shipped Hangfire-style job persistence in a .NET project, so I knew the answer was somewhere in the middle: store the messages somewhere durable, poll from that storage, and drop the broker entirely.

What I built

Shoebox is a Go message queue that runs inside your application process. Import the library, call Enqueue, keep moving.

• Storage as an interface, not a decision. One Storage abstraction with three implementations: Memory for pure speed, SQLite for zero-infrastructure durability, and Postgres for when you need multiple processes. You pick at construction time; the queue API never changes (internal/storage/storage.go).
• The Hangfire model. Persisted queues claim work by polling: SELECT ... FOR UPDATE SKIP LOCKED on Postgres, signed leases on SQLite. No pub/sub fan-out, no consumer group semantics — just exactly-once-budgeted claim, process, ack.
• Survives the process dying. Messages in flight are marked processing. If the app crashes, Reclaim rolls them back to pending on next startup. This is the part raw channels can't do and the reason I didn't reach for Kafka.
• Failure handling. Retry with exponential backoff, and every queue gets a {queue}.dlq shadow queue holding the original payload, last error, retry count, and timestamp.
• Extras that make it feel complete. Priority queues, dedupe, delayed messages, middleware (recovery included), and Prometheus metrics — depth, processed, errors, retries.
• shoeboxd — an optional standalone server binary so you can run the same engine out-of-process with an HTTP API and a small dashboard, without changing the library API.

Outcome

The full broker path — enqueue → dispatcher → handler → ack — sustains roughly 345k messages/sec on the in-memory backend, on a single M2 core-pair. SQLite trades about 300x of that throughput for persistence, like every durable queue has to. The design ended up as a clean three-tier answer to "Kafka is too much, channels die with the app": memory when you need speed, SQLite when you need durability and one process, Postgres when you need to scale horizontally.