← Back to work
API BUILD · 2023
DataPulse
An analytics API that doesn't fall over on Black Friday.
From a Postgres table that timed out at 5k events/min to a streaming pipeline handling 80k.
Client
DTC fashion brand
Year
2023
Duration
10 weeks
Our role
Backend architecture, API design
DataPulse
THE CHALLENGE
Their dashboard query took 14 seconds at peak. Marketing couldn't trust the numbers because half the events were dropped under load.
OUR APPROACH
- 01Moved hot writes to Kafka, cold storage to ClickHouse.
- 02Designed a versioned, documented REST API the frontend team could test against locally.
- 03Wrote a load test that replays a real Black Friday hour, ran it nightly in CI.
WHAT HAPPENED
Dashboard responds in under 200ms at 16x the previous load. The first Black Friday on the new pipeline had zero pages.
- Peak throughput
- 80k/min
- Dashboard p95
- 180ms
- Pages on launch
- 0
BUILT WITH
PythonFastAPIClickHouseKafkaDocker
GALLERY