Scaling Engineering Teams

Growing engineering teams is a challenging task that requires a mix of technical and organizational skills. As an engineer working in fast-growing startups, I've had the opportunity to scale engineering teams from a handful of people to dozens of engineers. Here are some of the best practices and solutions I've encountered along the way.

Derisking Product Rollouts with Feature Flags

In most successful companies, products and their underlying infrastructure are constantly evolving to changing needs and growing demand....Systems Architecture, Engineering Management, Product Development

Why Documentation Matters

Software is still eating the world, and there’s no end in sight....Documentation, CodeTrail

Making Architects work in Software Teams

As teams grow, managers invariably start to think about dividing the former team into smaller groups working on specific features or areas, in most cases supported by people or small teams that focus on tooling, infrastructure, and other “meta” tasks that enable engineering....Engineering Management, Product Development

Steady State means Continuous Rewriting

Most software companies don’t ever get to see a steady state, where the core product has been built, the rate of innovation decreases, and maintenance and minor improvements become the primary workload. In this world, priorities shift from figuring out the product, to ensuring customers are happy and keeping the lights on....Engineering Management

Choosing an Implementation for Unique Identifiers

Picking the right underlying system for generating unique identifiers (read ID generator implementation) throughout your application is an important task early on. Even if it doesn’t seem that important, having to revert a bad decision made early can require a lot of time and effort you’d gladly spend on a thousand other things....Systems Architecture

Reducing SaaS Complexity

Building software is hard. Typically, engineering teams use dozens of services for building, deploying, monitoring, and analyzing their product, as well as providers for billing, internal tooling, customer service, and so on....SaaS, Infrastructure-as-Code (IaC), Automation

Preview Environments for every Pull Request with Pulumi and Vercel

Testing the changes in a pull request isn't intuitive: checking out the branch, waiting for Docker to launch all service containers, keep waiting for another bit, then getting to test the actual PR. By the time you reach the last step you might have forgotten what you initially wanted to do, or another distraction came up in between. Does it have to be that way?...Systems Architecture, Infrastructure-as-Code (IaC), Automation, Pulumi, Terraform, GitHub Actions, Containers

Product Versioning for Fast-Moving Teams

Recently, I’ve gotten to wrap my head around versioning again. If you’ve been following this blog for a while, you might remember that I wrote a post about API versioning in mid-2021. Back then, I focused on versioning API-first developer products. This problem was relatively contained, we used a single versioning scheme to keep systems running for older clients while deploying a breaking change....Systems Architecture, CI/CD

Versioning Features for Sustainable Product Evolution

When you're working on a product that is constantly evolving, adding, deprecating, and deleting features not only has to be communicated with your customers in advance, but often you have to maintain multiple versions....Evergreen, Systems Architecture, Product Development

Improving observability with annotated build information

In my recent posts on managing a commit-oriented infrastructure, I explained the benefits of deploying services of one specific version. This version would be the hashed Git commit reference, or commit SHA....Observability, CI/CD, Docker, Containers, Git, Systems Architecture

Improving releases with service-splitting

Last week I published a piece on how running a CI-driven commit-based release infrastructure can improve your time-to-release drastically. I noticed that it wasn't super clear on how to get services running that aren't typically bundled or packaged up like container images, for example, frontend deployments. The issue here was that running all workflow steps in one routine every time something changed would take a long time, so usually, people put them into asynchronous flows, to run deployments to completion somewhere in the background, making it really hard to observe if something did not go to plan....Continuous Deployment (CD), Infrastructure-as-Code (IaC), Terraform, Pulumi, Github Actions, Systems Architecture

Running a commit-based release infrastructure

We spend a huge time thinking about building our products, but less about the pipeline that delivers features from engineering to production. Ideally, we want to test the latest changes before they're released to all our users, even if we carefully hide them behind feature toggles....CI/CD, Git, Infrastructure-as-Code (IaC), Systems Architecture

Preventing Increasing Fragmentation

How many SaaS applications do you use frequently? If you're working for a medium-sized company, chances are, that your number of subscriptions ranges from tens to hundreds. So many, that you wouldn't know which tools people on the other end of your org are using, with dozens of inactive subscriptions that drain budgets....Software-as-a-Service (SaaS), Company Budgeting

Boring Technologies, Reliable Foundations

In the ever-changing world of software engineering, keeping track of, not to mention specializing in, a broader range of areas and technologies has become increasingly difficult in the past years....Systems Architecture

Reaching Remote Acceptance

For millions of people, the past months have radically changed their lives. In my case, one of the most impactful changes was the gradual transition into working fully remotely, or to working from home....Evergreen, Remote Work, Engineering Management

Rapid TypeScript Development with esbuild and Estrella

With codebases growing steadily, so does the time spent waiting for pipelines of builds to finish, type checks to run, and in many development environments, services to be started and stopped. This classic approach to developing TypeScript services has led to much frustration over the years, as the size of projects only ever grew, but speed improvements couldn't hold up....TypeScript, Build Tools

Fundamental Design Decisions for Scalable Systems

When building customer-facing systems, especially in the realm of SaaS businesses, it's important to get a couple design aspects of your system right early on, if you want to plan at scale and grow without unexpected side effects....Evergreen, Systems Architecture, Engineering Management

Documenting Design Decisions using RFCs and ADRs

If you work on a software project, no matter your role, you'll make and a lot of decisions and witness even more being made. Ranging from minor choices between code style A or B to rather impactful ones like where to store data, and why that should be the case. Recalling the reason behind a decision can be painfully hard after months of ongoing engineering, often because you're missing the context that led up to it....Evergreen, Engineering Management

Scaling Release Systems

As products grow, so does the requirement to ship more often, which is desirable as it enables continuous feature delivery in a low-friction scenario....Containers, CI/CD

Building a Testing Culture

Writing software tests is one of the most important steps throughout the software development lifecycle. Preventing regressions and validating that business logic works as expected are much-needed guarantees when building a software product. Customer feedback should never be the primary source of learning that something does not work, at least this is what we're always telling ourselves. But too often, we're lacking a sufficient testing infrastructure and above all, actual tests to run. Why is that, and how can we solve it?...Engineering Management, Testing

Boost Your Team's DX By Formatting All Git Commits

Imagine the following: You've just completed a polished new feature, upgraded some dependencies, found some code paths that needed refactoring and worked on several other things you're about to release. Unfortunately, your existing commit messages look something like the following:...Engineering Management, Git