Necessary but not sufficient

The progression of software engineers is like this:

• Junior You have theoritical knowledge of software but most of it is obscure for you. The focus here is on learning a framework, learning how development processes work, and delivering code that works. You are not yet able to fully develop applications but can handle minor tasks.
• Senior You have launched a few applications and can autonomously complete 95% of tasks. You are not battle-tested ready yet, but can resolve most urgent issues. You are also able to onboard junior developers.
• Staff / Lead Engineer Like a compass, you are able to set a direction for the engineering team and navigate complex challenges, often involving product teams and other departments. You can tell horror stories – about the time the prod database was lost, or when the team had to spent shipping a hotfix on Friday 7pm for a rare but critical issue.
• Executive position You contribute to achieving key business milestones for the company. Most of your time is spent meeting, hiring, managing i.e maximizing producitvity. And then there’s the odd monthly code contribution.

For Juniors and Seniors, the key to success is technical expertise. Common work issues is resolved by whoever has the fastest or most secure implementation. Numbers and hard facts dominate decision-making. This is where you understand tech expertise is necessary.

This is not the case for staff engineers, the next step. In other industries such as civil engineering or mechnical engineering, you can rely on mathemetical formulas and sheer tech expertise. Software engineering, on the other hand, model human processes. Human is complex and messy, and so is software. As a result, any leadership position in software requires talent in resolving human issues: what do people want? what do people care about? what issues did my team overlooked? how do I resolve the conflict between programmer A who doesn’t agree with programmer B? how do we deliver this software on time,even if my lead programmer is going on vacation? So a conclusion for this stage is that tech expertise is not sufficient.

So it is critical for staff engineers to sit down and make plans. Can we list risks? Are there unknowns? What are our margins? If there are unknowns, what is the contigency plan if the main plain fails? Do we want to allocate all the team’s resources to the main project or hedge by having someone work on the backup plan? Experience can help answer a few of these. Often, answers comes from within the company. Stakeholders will tell what is best for them and point out threats. Even junior views are appreciated – their fresh minds or naïveté can help re-evaluate the challenge.

Model

Where:

• E: Effectiveness as a Lead Engineer. Your ability to lead and deliver complex projects
• T: Technical Expertise. The depth and breadth of your technical skills and knowledge, which are necessary but insufficient alone. For a junion or senior engineer, E = T
• H: Human Skills. Ability to resolve conflicts, understand human motivations and team dynamics. This depends on personal history, experience, motivation but also on company culture.
• F: Strategic foresight. Your ability to foresee risks, plan contegencies, and align diverse parties towards a shared goal. This comes from experience, from working on difficult projects, and also working with the company.

The Process As A Video Game Quest

When mentoring other engineers, I sometimes use the metaphor of US Congress(wo)men and their constituents. You go door-to-door, and gather feedback from constituents to understand their need. Another metaphor I use is from adventure games such as Mass Effect or The Witcher 3. In these games, you have sub-quests. You navigate complex dialog trees, initating communication with different people, choosing trade-offs, even sometimes taking to cryptic NPCs (Non-Playing-Characters) for hints. You build relationships with NPCs (your crew), earning trust and alignement across departments and companies, and making H progress slowly. You don’t mind going through complex dialog trees and grinding for gear because you know it will allow you to finish the level and the game.

In a typical project, NPCs can be product managers, paying or free users, coworkers, or even far off teams like legal or marketing. Ask questions like: Why? Why are we doing this? What do we want to get out of it? Why do you propose we do it this way? What are priorities? All these questions are meant to increase F.

If you ask the questions above, you will be able to get the team adopt your suggested solutions. Or deliver a complex project, even when dates, constraints and requirements were fuzzy.

When you are a staff engineer, can you get away with just being the best ever technically? If you have a Ph.D in machine learning or authored a published paper in Computer Science, yes – you can just be “the best ever” with your Technical Expertise T significanlty higher (») than other T in the team, thereby making H or F unnecessary. For the majority of software engineers, however the usual path involves understanding that tech expertise is necessary but not sufficient.