Kelsey Hightower on Engineering at Scale
“There’s a saying I like: ‘Some people have good ideas. Some ideas have people.’ When your idea outlives you, that’s success.” — Kelsey Hightower
Kelsey Hightower’s career is a perfect example of that. His ideas have taken on a life of their own, extending far beyond his work at Puppet, CoreOS, to KubeCon and Google. And he continues to scale his impact with his signature unscripted keynotes as well as the definitive book, “Kubernetes Up and Running.”
We were thrilled that Hightower joined ScyllaDB’s Monster SCALE Summit to share his experiences and advice on engineering at scale. And to scale his insights beyond those who joined us for the live keynote, we’re capturing some of the most memorable moments and takeaways here.
Monster SCALE Summit 2026 will go live March 11-12, featuring antirez, creator of Redis; Camille Fournier, author of “The Manager’s Path and Platform Engineering”; Martin Kleppmann, author of “Designing Data-Intensive Applications”) and more than 50 others. The event is free and virtual; register and join the community for some lively chats.
Fail Before You Scale
The interview with host Tim Koopmans began with a pointed warning for attendees of a conference that’s all about scaling: Don’t just chase scale because you’re fascinated by others’ scaling strategies and achievements. You need to really experience some pain personally first. As Hightower put it:
“A lot of people go see a conference talk — I’m probably guilty of this myself — and then try to ‘do scale things’ before they even have experience with what they’re doing. Back at Puppet Labs, lots of people wrote shell scripts with bad error handling. Things went awry when conditions weren’t perfect. Then they moved on to configuration management, and those who made that journey could understand the trade-offs. Those who started directly with Puppet often didn’t.”
“Be sure you have a reason,” he said, “So before you over-optimize for a problem you may not even have, you should ask: ‘How bad is it really? Where are the metrics proving that you need a more scalable solution?’ Sometimes you can do nothing and just wait for scaling to become the default option.”
Ultimately, you should hope for the “good problem” where increasing demand causes you to hit the limits of your tech, he said. That’s much better than having few customers and over-engineering for problems you don’t have.
Make the Best Choice You Can … for Now
The conversation shifted to what level of scale teams should target with their initial tech stack and each subsequent iteration. Should you optimize for a future state that hasn’t happened yet? Play it safe in case the market changes?
“If you’re not sure whether you’re on the right stack … I promise you, it’s going to change,” Hightower said. “Make the best choice you can for now. You can spend all year optimizing for ‘the best thing,’ but it may not be the best thing 10 years from now. Say you pick a database, go all in, learn best practices. But put a little footnote in your design doc: ‘Here’s how we’d change this.’ Estimate the switching cost. If you do that, you won’t get stuck in sunk cost fallacy later.”
Rather than trying to predict the future, think about how to avoid getting trapped. You don’t want dependencies or extensions to limit your ability to migrate when it’s time to take a better (or just different) path.
“Change isn’t failure,” he emphasized. “Plan for it; don’t fear it.”
In Hightower’s view, scaling decisions should start on a whiteboard, not in code.
“When I was at Google, we’d do technical whiteboard sessions. Draw a line — that’s time. “Today, we’re here. Our platform allows us to do these things. Is that good or bad?” Then draw ahead: “Where do we want to be in two years?”
He continued, “Usually that’s driven by teams and customer needs. You can’t do everything at once. So plot milestones — six months, a year, etc. You can push things out in time for when new libraries or tools arrive. If something new shows up that gets you two years ahead instantly, great. Having a timeline gives you freedom without guilt that you can’t ship everything today.”
Are You Really Prepared for a 747?
Following up on the Google thread, Koopmans asked, “I’d love to hear practical ways Google avoids over-engineering when designing for scale.” To illustrate why “Google-scale” solutions don’t always fit everyone else, Hightower used a memorable analogy:
“I had a customer once say, ‘We want BigQuery on-prem.’ I said, ‘You do? Really? OK, how much money do you have?’ And it was one of those companies that had plenty of capital, so that wasn’t the issue. I told them, ‘That would be like going to the airport, looking out the window, seeing a brand-new 747 and telling the airline that you want that plane. Even if they let you buy it, you don’t have a pilot’s license, you don’t know how to fuel it. Where are you going to park it? Are you going to drive it down your subdivision, decapitating the roofs of your neighbors’ houses?” Some things just aren’t meant for everyone.”
Ultimately, whether it’s over-engineering or not depends on the target user. Understand who they are, how they work and what tools they use, then build with that in mind.
Hightower also warned against treating “best practices” as universal truths: “One question that most customers show up with is, ‘What are the best practices?’ Not necessarily the best practices for me. They just want to know what everyone else is doing. I think that might be another anti-pattern in the mix, where you only care about what everyone else is doing and you don’t bring the necessary context for a good recommendation.”
How Leaders Should Think About Dev Tooling
“Serializing engineering culture” (Hightower’s phrase) like Google did with its google3 monorepo makes it simple for thousands of new engineers to join the team and start contributing almost instantly. During his tenure at Google, everything from gRPC to deployment tools was integrated. Engineers just opened a browser, added code and reviews would start automatically.
However, there’s a fine line between serializing and stifling. Hightower believes that prohibiting engineers from even installing Python on their laptops, for example, is overkill: “That’s like telling Picasso he can’t use his favorite brush.”
He continued: “Everyone works differently. As a leader, learn what tools people actually use and promote sharing. Have engineers show their workflows — the shortcuts, setups and plugins that make them productive. That’s where creativity and speed come from. Share the nuance. Most people think their tricks are too small to matter, but they do. I want to see your dotfiles! You’ll inspire others.”
Watch the Complete Talk
As Hightower noted, “Some people have good ideas. Some ideas have people.” His approach to scale — pragmatic, context-driven and human — shows why some ideas really do outlive the people who created them.
You can see the full talk below.
Fun fact: it was truly an unscripted interview — Hightower insisted! The team met him in the hotel lobby that morning, chatted a bit during a coffee run, prepped the camera angles … and suddenly Hightower and Koopmans were broadcasting to 20,000 attendees around the world.
