Sole Backend EngineerHapminds2026Confidential · details generalised
LLM layer and capacity
hap_ai: one LLM interface with Gemini failover, a daily token cap, and model calls moved out of database transactions.
✓ VERIFIED200k
Tokens per user per day, enforced in RedisHard daily cap per user.~ ESTIMATE~50%
Fewer prompt tokens on long conversationsEstimated effect of rolling summarisation.✓ VERIFIED150+
Automated deploys, ~4 min median push to liveGitHub Actions: 1,286 pipeline runs.The problem
Model calls of 3 to 30 seconds ran inside database transactions and held connections open. There was no limit on how many tokens one user could spend.
What I built
I built hap_ai, a provider-agnostic LLM layer: OpenAI first, Gemini on failure. I moved model calls outside transactions, capped tokens per user in Redis, and made message retries idempotent.
Architecture
Request path
Model calls run after the database transaction closes.
01
Core APINode.js / Fastify.02
AI servicePython / FastAPI.03
hap_aiOpenAI, Gemini on failure.04
RedisDaily token count per user.Core API → AI service · idempotent retriesAI service → hap_ai · model callhap_ai → Redis · check the 200k cap
Decision log
What I chose, and what I gave up
01Model calls outside database transactionsover keeping the call inside the transactionEach open transaction held a pooled connection for the whole model call.
Releasing the connection first raised estimated concurrent-conversation capacity by about 100× per pool.
+ State is saved in two steps− Connections held for up to 30 seconds
Scaling story
How it grew
Deployment
GitHub Actions deploys to DigitalOcean.
GitHub Actions · Docker · DigitalOcean01
Idempotent retriesA retried message costs no extra tokens and never creates a duplicate booking or crisis alert.
02
Automated deploys1,286 pipeline runs and 150+ deploys, about 4 minutes from push to live.
Next case studyAurNexa marketplace backendBackend design for a pre-launch B2B construction-materials marketplace: 9 bounded contexts, 5 in the MVP.