Shoebox

Context

There is a large gap between a Go channel and Kafka. A channel loses its messages when the process exits and has no built-in retry or inspection. Kafka solves those problems, but a cluster and topic configuration are excessive for a service processing a few hundred messages a night. Shoebox keeps the queue inside the application while adding the persistence and operations that a background-work system needs.

What I built

Shoebox is an embedded Go queue. Import it, register a handler, call Enqueue, and keep working.

• Storage without an API split. One queue API works with Memory, SQLite, and PostgreSQL. SQLite uses a pure-Go driver with WAL; PostgreSQL uses a connection pool and SELECT ... FOR UPDATE SKIP LOCKED for concurrent consumers.
• At-least-once delivery. A message is acknowledged only after its handler succeeds. Persistent backends recover in-flight messages after a crash, so handlers must be safe to run again.
• Failure handling operators can inspect. Handler options support retry limits, timeouts, and exponential, constant, or custom backoff. Exhausted messages go to {queue}.dlq, where they can be listed, inspected, and replayed.
• Scheduling and control. Queues support priorities, delayed and absolute-time scheduled messages, persistent periodic jobs, and pause, resume, and drain operations. Enqueue and acknowledgement batching reduce durable storage overhead when needed.
• Deduplication with explicit guarantees. The default in-memory UnboundedTTL policy covers a five-minute window; BoundedLRU caps memory; PostgreSQL can use Durable deduplication that survives restarts and concurrent writers. The in-memory policies are best-effort, not exactly-once delivery.
• One engine, two deployment modes. shoeboxd exposes the same queue over HTTP with a dashboard, REST API, Prometheus metrics, authentication, and webhook delivery. The library also has typed JSON task helpers and an optional OpenTelemetry module.

Outcome

The repository now includes automatic forward-only schema upgrades for SQLite and PostgreSQL, reusable migration files, CI coverage, benchmark baselines, and a race-checked acknowledgement batching path. The latest local benchmark snapshot reports 102,298 ns/op for broker memory throughput, 294,987 ns/op for SQLite, and 6,040,244 ns/op for PostgreSQL. Those numbers are a local Ryzen 5 7430U / Go 1.27.1 snapshot, not a hardware-independent promise.