Tag Archives: Engineering Management

Mastering Execution in Engineering Teams: From Formation to Delivery

Posted on by

In the fast-paced world of software development, execution is everything. It’s not just about writing code; it’s about forming effective teams, collaborating across departments, focusing on outcomes, and managing technical debt. Let’s dive into these crucial aspects of engineering execution.

The Art of Forming Teams and Structures

When it comes to team formation, the old adage rings true: “Adding manpower to a late software project makes it later,” as famously stated in Brooks’ Law. This counterintuitive principle reminds us that team dynamics are complex and that simply adding more people doesn’t necessarily speed things up.

Understanding the stages of team formation is crucial. The Forming/Storming/Norming/Performing model, developed by Bruce Tuckman, provides a useful framework. In my experience, the Forming and Storming stages usually take a minimum of 2-3 sprints. If you’re struggling with these initial stages, consider reducing your sprint cadence to give the team a short reflection period on their working process to drive process change faster.

Here are some key principles for effective team structure:

Longevity should be a priority when structuring your engineering teams. Teams should be viewed as long-term investments rather than temporary assemblies. Even when headcount calculations don’t align perfectly, resist the urge to disband established teams. The relationships, shared knowledge, and mutual understanding that develop over time are invaluable assets that can’t be easily replicated. A team that has worked together for an extended period will often outperform a newly formed team, even if the latter looks better on paper.

Independence is another crucial factor in team effectiveness. Strive to create teams that possess all the skills necessary to execute their projects without constant handoffs to other teams. This autonomy not only boosts efficiency by reducing communication overhead and wait times but also increases accountability. When a team has end-to-end ownership of a project or feature, they’re more likely to take pride in their work and ensure its success.

Lastly, system ownership plays a vital role in team engagement and performance. In my experience, teams should have clear ownership over specific systems or components within your technology stack. This ownership fosters a deep understanding of the system and a sense of responsibility for its performance and evolution. Conversely, teams without any system ownership often struggle to appreciate impact of the technical debt they introduce and may lose respect for the value of the systems they interact with. By giving teams ownership, you’re not just assigning responsibility; you’re teaching a team about how to responsibly manage technical debt, as they are ultimately going to be then one’s responsible for it in their own system.

The Diplomacy of Inter-Team Collaboration

Working with other teams is an essential skill in any large organization, and it requires a strategic approach rooted in understanding human behavior and organizational dynamics. One crucial concept to keep in mind is what I like to call “Game Theory in Action.” When seeking collaboration with other teams, always consider the question, “What’s in it for me?” from their perspective. It’s a natural human tendency for individuals and groups to act in their own interest, and engineering teams are no exception. By anticipating this mindset, you can proactively address the needs and motivations of other teams, making collaboration more likely and more fruitful. This doesn’t mean being manipulative; rather, it’s about finding genuine win-win scenarios that benefit all parties involved.

Another key aspect of successful inter-team collaboration is the cultivation of informal networks within your organization. As a leader, one of your roles is to help your team build what I call an “irregular social network” that extends beyond the formal organizational structure. Encourage your team members to connect with colleagues from other departments, attend cross-functional meetings or events, and engage in casual conversations with people outside their immediate circle. These informal connections can be invaluable for smooth collaboration and problem-solving. They create channels for quick information exchange, foster mutual understanding, and often lead to creative solutions that might not emerge through formal channels alone. By building these networks, your team will be better positioned to know more about what’s going on within the org, and share more in common solutions to problems, in small organizations this isn’t as important as in large ones.

Shifting Focus: From Output to Outcome

It’s easy to get caught up in metrics like story points, sprint completion rates, or hours logged. However, these are merely measures of output, not outcome. Your true measure of success should be the business value your team delivers.

I once had a conversation with one of my engineers about changing the way the calculate carrier over work, I told him a half done story is “not done” and should count to zero for sprint completion, ultimately making their completion rate lower and closer to actual “completion”, his response was “But my points!”, he was fixated on his story points being his sole measure of success and was ignoring the actual value the team was delivering to the business.

Keep your engineers connected to the value they’re creating. Don’t let product management focus solely on “feature” or “milestone” success without tying it to measurable business value. If you do, you risk falling into the trap of DDD (Deadline Driven Development).

Remember Dan Pink’s insights on motivation: autonomy, mastery, and purpose are key drivers. By connecting your team’s work to real business outcomes, you’re providing that crucial sense of purpose.

Dan Pink, what motivates people

The Balancing Act of Technical Debt Management

Managing technical debt is a critical part of long-term success in software engineering, and it requires a strategic approach. One principle I’ve found effective is what I call the “30% Rule.” This involves allocating about 30% of your team’s time for technical improvements. While it might seem like a significant investment, especially when faced with pressing feature demands, this dedication to ongoing improvement pays substantial dividends in the long run. It helps prevent the accumulation of technical debt that can slow down development and increase the risk of system failures.

Why 30%? I asked Yaron Zeidman this once, who taught me this, and his response was, “Joel, I’ve worked in companies where we tried 20%, and we found that we weren’t able to keep on top of debt and technical improvements we needed, and i worked in companies where we tried 40%, and we found we weren’t able to execute on product enough, so 30% seems to be the happy middle ground.”.

Time-boxing is another powerful technique for addressing technical debt. One approach I’ve seen work well is the use of “Mobathons” – intensive periods focused solely on tackling technical debt or improvements. See this post about them.

Another instance, I once worked with a team that implemented a “60% leap sprint,” where the majority of a sprint was dedicated to making significant progress on technical debt, and every other sprint was 100% product work. These focused efforts can create momentum and visible progress, boosting team morale and improving system health.

If you try to do every sprint exactly 70/30 split, it almost never works out well.

One of the most important principles in managing technical debt is to finish what you start. It’s all too easy to let the tail end of technical migrations drag on for years, but this approach can be costly. The longer legacy systems remain in place, the more their costs grow, and the more significant their impact becomes. By seeing migrations through to completion, you can fully realize the benefits of your work and avoid the compounding costs of maintaining legacy systems.

When it comes to system design and development, thinking small can yield big benefits. Building small, modular systems allows for incremental improvement and quicker realization of value, for example framework upgrades such as reactjs or other frameworks, need to be done at system level, for a single large system it becomes an all in effort, if you have 10 smaller systems you can do one and measure the value in an increment, validate assumptions, to help you re-prioritize before continuing. This approach not only makes it easier to manage and update your systems but also allows for more frequent deliveries of value to your users and engineers.

While technical debt may seem like a purely engineering concern, it’s crucial to include product management in these discussions. Getting buy-in from product managers on your technical work can be tremendously beneficial. Not only can they help you ask the right questions about the business impact of technical decisions, but they can also become powerful allies in advocating for necessary technical work.

Finally, don’t hesitate to escalate when necessary. If technical debt is severely impacting your team’s ability to deliver, it’s time to have a serious conversation with product management and leadership. Work together to pitch for more headcount or resources. Remember, addressing technical debt isn’t just about engineering preferences – it’s about maintaining the health and efficiency of the systems that drive your business.

Conclusion

Effective execution in engineering teams is a multifaceted challenge. It requires thoughtful team formation, skilled inter-team collaboration, a focus on meaningful outcomes, and diligent technical debt management. By mastering these areas, you can create a high-performing engineering organization that consistently delivers value.

Remember, the goal isn’t just to write code or complete sprints. It’s to create systems and products that drive real business value. Keep this north star in mind, and you’ll be well on your way to engineering excellence.

System Building Manifesto 

Posted on by

It’s hard for highly technical people to not dominate conversations about tech. But in a role of Engineering Manager it’s important to not do this, ownership should be with the people doing the work not their managers.

So how do you manage people with less experience than you and not become a dictator?

Something I’ve been working on with my teams lately is coming up with High level Guidelines to give them work with. Highlighting common pit falls and encouraging best practice that come from the experienced people in the organisation. Having a common understanding of what’s good or best help people move in the right direction while giving them the freedom to design and build as they like, as long as the guidelines are not too specific and leave room for interpretation that maybe be slightly with each team or engineers individual context.

For example, I would not give my teams a guideline of “Code Coverage >80%”, this is too specific, and based on a team’s application they are working on they maybe happy with 70 or even 60%, and that’s ok. A better way to phrase this if coverage is important to you would be “Team’s should value and have high test coverage”.

This again though is too specific, If you have poor assertions, it doesn’t matter what % coverage you have right? Code coverage has a higher purpose, and it alone does not serve this purpose, it’s better to focus on the higher level goals.

Code Coverage, for me, is a part of Test Automation, the goal of test automation is to reduce bugs, production issues etc. So these in my opinion are better to focus on. In my example below

Systems should have test automation that brings confidence and inspires courage in engineers

Where I mention test automation i mention the behaviour I have seen in high performing teams specifically. I’ve worked in teams where the “deploy” button is pressed with little regard for the impacts, because the Engineers are confident in the pipelines, monitoring and rollbacks that are in place. This for me is the high level goal i want my engineers to strive for, Real Continuous Delivery.

So here’s the full list I have in Draft, feel free to comment, I’ll do some follow up post with dives into some of them.

I used the word “Manifesto” because when i showed them to another manager it’s what he called it, I thought it was cool 🙂

Guiding principles for Systems

  • Systems should be Domain Specific, responsible for one or few domains
  • Systems should be small in the overwhelming majority of cases. Small systems limit complexity
  • Systems should be consistent in design in the overwhelming majority of cases
  • Systems should be easy to build
  • Systems should have test automation that brings confidence and inspires courage in engineers
  • Systems should be easy to contribute to, not require extensive training
  • Systems should have Cross Cutting concerns addressed and shared in an easy and consistent way
  • Systems operate independently for the purpose of testing and debugging
  • Systems have consistent agreed upon telemetry for monitoring
    • Telemetry is a solved cross cutting concern for non-domain specific metrics
  • Systems are built on Modern up-to-date frameworks and platforms
  • Systems use Continuous Integration as a principle not a tool, merge and deploy often and in small increments
  • A System scales horizontally, both within a site and across multiple site. With this comes redundancy, Users experience zero downtime for instance and site outages
  • Systems have owners, who are responsible for the long-term health of the systems and who have contributors as customers

How to Promote an Engineer

Posted on by

To understand how to promote we need to understand why we have titles, and what they are for.

Most modern tech companies (Amazon, etc) have the IC (Individual Contributor) level track concept, so I will use this as a basis. It works in roughly these levels

  • IC1 – Associate Engineer
  • IC2 – Engineer
  • IC3 – Senior Engineer
  • IC4 – Lead
  • Etc

Titles are important for recognition of people’s achievements, to set them targets to drive personal improvement, they also help with reconciliation of compensation to make sure people are paid what they deserve but I don’t personally believe compensation is their primary purpose.

They can also have a negative culture impact if used in the wrong way, sometimes people can use their title to boss or lord over others, but my advice on this is that it’s not a problem with the title, it’s a problem with the person, and this is toxic behavior, if they can’t fix it, show them the door.

Titles are also used implicitly to set the expectations of others. When people work on a team with someone of a higher title than them, they should hopefully be inspired to be a better engineer, and in turn help drive their own career progression.

Usually titles come with a defined list of Qualities that can be quiet subjective and high level, like “Practices the Latest in CI/CD Technology”, while useful as a guide, these aren’t very actionable or objective that you can give to an Engineer to do.

Many Managers I talk to about career progression tend to look at goal setting as a method for Engineer to prove themselves, I like goal setting and I do it a lot, but when it comes to career progression I think it’s a bit flawed and I’ll explain why.

Some examples people used with me for goals for Engineers were:

  • Do a blog post
  • Lead a project to completion
  • Do a tech talk

Goals like this are fine, but if used for career progression you can effectively create a checklist list for a promotion, and after the Engineer has done X,Y,Z on the list we promote him, this doesn’t mean after being promoted they will continue to do this. If we take the example of an Engineer who is set the above three goals, does them in a Quarter or two, gets promoted to senior engineer, then goes back to doing the same thing he did before. He’s not likely to inspire those around him who are doing the same job now, but don’t have the title. In fact, it may even have a negative impact on the team.

And when you are asked by someone “why is he senior?” is your response of “He did a design review and a tech talk 2 years ago” going to be a good answer?

So when are goals ok?

Goals I believe are good for short term, they are good to push someone out of their comfort zone to give them a taste of something, or to defeat fear. A bit like young children and swimming; children are usually worried about getting wet and will cry and complain, but once you finally get them in the water it’s hard to get them out. Goal setting is good for pushing people out of their comfort zones, and also for giving people a taste of something new that they other wise would never have tried, perhaps in the example of cross training, or opening conversation of new career paths, is my opinion.

But back to career progression

If we want people to be doing the above things, they should be self-motivated to do them not doing them because they are led by the promotion carrot stick. So what we are after more, is a change on mindset as opposed to a “To Do” list, so that it becomes are part of their day-to-day thinking.

How to change or measure people’s mindset?

You can’t measure that I’ve found, but the best proxy I’ve found is the behavior people exhibit. The advantage of using behavior is it is a day-to-day thing. The way people conduct themselves when dealing with others, specific to engineering scenarios, on a day-to-day basis is something you can set goals around, or more so, expectations.

Setting expectations of behavior is something that Ben Horowitz talks about, he wrote a blog a long time ago called “Good PM, Bad PM” applying this to Product Mangers in the 90s and 00s.

If we promote people based on their day-to-day behavior the exhibit, they are likely to continue this behavior as it part of their routine, they are unlikely to degreed their behavior over time, and if more goals are set around improving behavior then they will most likely progress.

Taking the example of the “Inspiration from the Senior Engineer on my team”, if we assume that behavior is consistent over time then the answer to the question about “why is he senior?” becomes easier to answer in that he acts in fashion A,B,C on a day-to-day basis.

I have some example of what I set, to try to explain the method:

  • A Senior Engineer identifies and helps with skill gaps in his immediate area, escalating when they are too much for him to handle. He is the guy that says in a stand up “hey Bob, you haven’t had much experience in system X how about you pick up that task today.” He encourages continuous improvement of the team in the day to day.

The above is an expectation around collaboration and system ownership, this is from my Senior Engineer expectations, you can see how its worded that it’s day-to-day behavior expectation around being a positive influence in the team.

The thing missing from this that’s present in the Horowitz article though is the “Bad PM”. Horowitz remarks on calling out explicit negatives in behavior as well. This is very useful for calling out common bad behavior people pick up within the organization (or in the industry in general) that might be common and help to correct them.

Here’s an example from my basic Engineer Expectations:

  • An Engineer Tests. They employ automation to do so and understands when to use Unit vs System vs Integration Testing. An engineer does not have a “Tester” on their team whose responsibility it is to do the testing.

This is a common pitfall from the industry, especially from older engineers who used to work on teams where they did have “testers”. Engineers like this that have any form of Quality role attached to their team think they have a “tester”, and this is very bad for not only cross functional teams but also the correct use of automation. So by calling out this negative behavior we help to correct this by setting the expectations.

Be careful though, the expectations I have here are specific to my context, not everyone should have the same expectations, there will be things unique to your company, team, etc. that they should change. From the example above, maybe you do have “Testers” on your team, and that is ok for you.

In closing though, I would recommend trying to Set “Behavior Expectations” around your career levels as a method to drive the right change, in your staff, for promotions.