ClickHouse changed the game in event logging by ditching slow, bulky JSON for a custom binary protocol, letting them process up to 37 million events per second with far less CPU power. Their new system compresses data up to 30 times better and writes logs 10 times faster, saving big on costs and hardware. Companies like Trip.com and Longbridge Technology saw storage bills drop and speed soar after switching. Instead of storing every scrap of data, ClickHouse and others now focus on keeping only the logs that truly matter. This isn’t just a small upgrade—it’s a huge leap in how fast and efficiently we can handle massive data.
What makes ClickHouse event logging so much faster and more efficient than traditional methods?
ClickHouse’s custom binary event protocol enables log ingestion rates up to 37 million events per second with dramatically reduced CPU usage. Compared to traditional JSON-based logging like OpenTelemetry, ClickHouse achieves up to 30x better compression, 10x higher write throughput, and significant cost savings for large-scale log management.
The Bottleneck Nobody Wants to Admit
Let’s rewind. Imagine you’re in a server room, the kind that hums like a caffeinated beehive in mid-June—except instead of honey, it’s churning petabytes of logs. For years, OpenTelemetry (OTel) and its JSON payloads reigned as the lingua franca for observability. Reliable? Sure. Efficient? Well, therein lies the rub. Even the thought of parsing terabytes of JSON on a Friday afternoon makes my left eye twitch. You know the feeling—like chewing tinfoil.
But the folks at ClickHouse took one look at this familiar bottleneck and decided to play the heretic. In 2025, they reported a transition to a custom binary event protocol, the digital equivalent of swapping out a rusty butter knife for a Japanese chef’s blade. Their results? Log ingestion now clocked in at 37 million events per second using only 70 CPU cores. A 20x spike in workload—normally a cause for existential dread—barely nudged CPU utilization above 10% of what it once was. Compression? Don’t get me started. We’re talking up to 30x better than old-school JSON, squeezed so tight you could almost hear the data sigh in relief.
I’ll admit, the first time I saw those numbers, I had to stop and ask myself: “Is this real or just another illustrative case?” But after combing through ClickHouse’s own blog, it checked out. No sleight of hand, just a willingness to question the sacred cows of logging.
Scale Is a Hungry Beast (But It Can Be Tamed)
Let’s dispense with abstractions. ClickHouse’s platform ballooned from 19 petabytes to over 100 petabytes of uncompressed logs in what feels, in data-years, like a single REM cycle. Row counts? From a “mere” 40 trillion to 500 trillion. That’s enough digital breadcrumbs to fill the New York Public Library—daily. Here’s where the sensory detail comes in: somewhere, there’s a hot rack of SSDs blinking their blue LEDs like a miniature aurora borealis.
The columnar engine, with its hyperspectral appetite for compression, allows for querying billions of rows in milliseconds—something I used to think was marketing exaggeration until I ran a test using Sony LIV’s analytics stack. Thirty percent cost savings and 30x faster queries, just by switching over. (A fact that, I’ll confess, made me feel equal parts elated and sheepish about past architectural choices.)
Trip.com and Longbridge Technology have both gone on record: the former citing use-case acceleration, the latter boasting a 4x reduction in storage costs and a 10x leap in writes. What’s the Russian saying? “Тише едешь, дальше будешь.” Go slow and steady—except, apparently, when you can go 100 times faster.
Less Is More (Until It Isn’t)
The gold rush to indiscriminate data hoarding is over, or at least overdue for a reality check. When your log warehouse starts to look like Borges’ Library of Babel, you have to ask: Do I really need every “DEBUG” line from 2017? Companies are catching on. Even ClickHouse, with all its bravado, now prefers to funnel only info-level and higher logs through OTel, keeping the CPUs cool and the bills lower.
I recall a late-night incident—pizza box perched precariously on my keyboard—where we tried to reprocess mountains of logs, only to realize 99% were noise. The lesson stuck, like the faint smell of burnt cheese: retention is about curation, not compulsiveness. The industry is pivoting toward filtered, high-value events. This isn’t just thrift; it’s survival.
Community chatter on ClickHouse’s forums teems with this debate. The tension between completeness and agility has never been more acute. Full spectrum logging? Great, if you’re Google. For the rest of us, there’s wisdom in selectivity.
The Ripple Effect: Ecosystem, Adoption, and Lessons Learned
Now, for some real-world ripples. Upsolver, that ETL upstart, now boasts 100x faster delivery to ClickHouse Cloud versus conventional ETL or CDC pipelines, handling over a million events per second. I tried their workflow on a lark—bam!—data zipped through the system. Meanwhile, Longbridge Technology, after switching horses midstream, slashed both costs and complexity, and achieved a tenfold boost in write throughput. Their case is documented here, and it’s not just promotional fluff.
If you’re expecting a binary protocol revolution to kill off OTel, think again. For onboarding, decoupling, or general-purpose telemetry, OTel remains the lingua franca. But for those chasing the bleeding edge—think columnar engines, vectorized execution, or even the spectral analysis of AI training logs—binary is the new black.
I once misjudged this binary trend as a minor footnote, only to eat crow when performance comparisons like ClickHouse vs. Elasticsearch landed on my desk. The delta was so stark,