War Stories - How I matured a chaotic startup into a professional development team
A story of a dev-ops adoption and an organization’s transformation from waterfall to agile to lean.
The Beginning of the End?
Engineers were leaving by the droves. We lost seven engineers in months due to a single project. Our turnover had hit a high of a 38%.
It was the beginning of an exodus and an organizational collapse was on the horizon.
What was going on?
We were supposed to be on a solid path. Product ran two-week sprints and performed all of the scrum ceremonies to the T. We had 1:1s with all of the engineers. By any organizational standard, we should have been doing well.
Our problem was that, while on paper we were agile, in spirit we were far from it. We had highly centralized, top-down command-and-control, driven by product management that told the developers exactly what to do and when to do it.
Product’s initiatives weren’t at all aligned with the business. The developers ended up creating feature after feature that the business never used or even asked for.
The project that broke the camel’s back was a multi-month waterfall technology project that provided zero value, was started with little to no planning, and touched every core part of our software systems and business processes. Despite multiple warnings raised by engineers, they were overruled and the project proceeded.
It was an intense and focused effort by the organization to get it done, with the promise from Product that it would solve all of the business’ problems by the end. It was an empty promise but we labored on.
On the one hand, it was great to see that level of organizational alignment. On the other, developers hate death marches. Sure, we ran it in two-week sprints, but a two-week waterfall is still a waterfall. By the end of it, the developers were burnt out.
Enough was enough. The rate of departure was unsustainable and not yet over. As an engineering manager close to team, I knew that more than a few developers were actively looking for other jobs.
We had to make a change.
Who Are We? Product or Engineering?
The organizational structure at the company hadn’t changed much from its startup roots. While the company had scaled by nearly 600%, its processes and structures hadn’t grown to match. The structure was very much the same as its startup roots.
There was no “Engineering” department. It was simply “Product,” which included all of the product managers, designers, and engineers under a single organizational entity. Engineering didn’t have a separate budget, OKRs, KPIs, or even roadmap.
As a result, Product was the sole authority on everything.
This led to a few challenges
It was difficult to prioritize or advocate for Engineering requirements. Things like addressing technical debt would always lose against “the next big product idea” that would (in an ideal world) generate millions of more dollars in revenue.
As a result, the system accrued a significant amount of technical debt over the years, making the development of even basic things like changing the color of a button across all of the pages a monumental effort. We had a joke for it — we called it “load-bearing CSS” because we knew somehow, somewhere, there was critical business logic depending on the color of that button.
It made resourcing and planning difficult. We shared a fungible budget with Product, which meant when an engineer left, that allocation often disappeared, going to either Design or Product, or disappearing outright.
It made it hard to give people raises. Our developers had significant pay disparities as a result of ineffective hiring practices during the company’s hyper-growth phase. Many were nearly 30–50% under market, some were 50% above market. In some cases, people who shared the same level had as much as a 60% salary difference!
Finally, it made engineering difficult. We were subservient to the Product organization, which meant we always had to do what the product manager said, without exceptions. We ended up with a team of ticket takers building tiny things the product manager specified, instead of leveraging our collective problem-solving skills to find a great solution to a problem.
Lacking voice
As far as the rest of the company was concerned, Product was the technology team and, as a result, the product managers were the voice of the technology, design, and engineering.
Unfortunately, the product managers simply didn’t know the technology as well as they thought they did. They had a lot of assumptions about how it was built and how it functioned — these assumptions were incorrect.
Part of the issue was their skill level. Novice product managers ended up drip-feeding information or projects to engineers with no cohesive end vision. As a result, things were built as separate items or as minor add-ons to existing systems. When it was time to make the pièce de résistance, tying them together into a synergistic whole, it was impossible because they weren’t built to be integrated or separated from other parts of the system.
Part of the challenge was the team structure. We had teams that operated at the layer level — front end, back end, and mobile teams. They weren’t cross-functional and each initiative required significant cross-over and synchronization. The result? Unnecessary hand-offs, gatekeeping, and delays on things that could have been executed rapidly with a single cross-functional team.
This happened repeatedly. When product managers communicated their features to the rest of the company, they were assumed to have wildly different capabilities than what was actually built. However, because product managers were viewed as the voice of the technology, their inaccurate statements were taken as gospel by other parts of the organization, leading to significant confusion.
Engineering needed a distinct voice.
How did we get there?
Keep reading with a 7-day free trial
Subscribe to Joseph Gefroh to keep reading this post and get 7 days of free access to the full post archives.