‍Completely unrelated to the unfolding events at a prominent social media company, we thought it is a good idea to highlight some anti-patterns to measuring productivity, and a few things that should be common sense to do instead.

Anti-Pattern to Engineering Metrics

Lines of code written: While lines of code can be a proxy metrics for how much code you have to maintain (and therefore costs to incur) it has been proven to be a terrible metric for productivity. This shouldn’t need much explanation but: Output does not necessarily equate to outcome. Different programming languages have different verbosity, and e.g., just including some open-source packages or pasting in some code from Stack Overflow does not make you more productive. And, of course, solving hard problems often does not require a lot of code, but a lot of thinking, exploration, and collaboration.

‍Complexity or salience of code: Complex code is not good code. Someone else has to read, understand, and maintain it. Salient code (let’s interpret this as “most notable”) can be worth noting, but that does not make you a good team player or show any consistency. One key point here: Software development is a team sport. However, every company has some rock stars who are able to do things that others cannot, but that does not mean you only want rock stars who cannot work with others. This is especially true if you have more than on star with similar traits.

‍Number of PRs/Reviews: Now, this is a bit more on the grey side. Of course, if you can churn out a lot of pull requests (PRs) this typically means it is work associated to some feature spec (at least one would hope so). This means, it is at least going into the direction that is perceived as customer value. But on its own, PRs are again a proxy of the amount of work that requires maintenance and a metric that is prone to gaming. Sure, you can do a lot of small PRs and “clog up” the review and merge pipeline, or do a lot of cursory reviews (LGTM) or worth add confusing and useless comments. So, adding value to some customer feature is good, but for the sake of showing activity it is not.

‍Bug fixes: Someone fixes a zillion bugs? This can be great, but two questions come to mind: Why are there that many bugs in the first place? And: do all of these “bugs” need fixing? The unfortunate truth is that no software is perfect and there is always way more to do than that there is time for.

‍Number of hours worked: This shouldn’t need much explanation either, but hours does not equate to time well spent. Sure, if you only work 1h a day you must be really exceptional (and not hardcore) to produce continuously high value. But if you work 14h it does not mean you are at your most productive. Especially running out of steam/coffee. All good things need the right effort without going to extremes. Sometimes this might be more coding, sometimes more exploration/thinking/discussing.

‍

‍

‍The Value of Data-Driven Engineering

‍Now, we are the last ones not to promote some metrics (aehm ….), but metrics are not the goal, insights to what is going on in the organisation and continuous improvement loops are.

Firstly, metrics are almost always poor to manage individuals. If you need metrics to understand if an employee gives you the value you are expecting than you are already in trouble. We understand that large organisation need some sense of objectivity, but using any of the above is probably poor. Also, metrics in itself do not solve principal issues. Which brings us to the next point:

Metrics are of value to highlight trends, anomalies, imbalances and bottlenecks. This is in particular true for processes, workflows, and team/group aggregates. It would be unthinkable to run your ops without watching some numbers, same for sales or marketing. But it is always intended to improve and streamline processes and create less disruption. As such, metrics can provide great signals to manage organisational engineering productivity and to remove friction in the system.

For development teams it makes sense to measure some numbers and trends. For instance: Where in the development process do we get stuck or waste time? Does our build system hold us back? Are we overloaded with reviews? Do we have QA issue? Do our sprints get re-scoped all the time mid air? Are there too many context switches? All these things are essential, especially for good managers to assist their teams. Managers should remove roadblocks and distractions, not micromanage lines of code.

As for trends there are some indicators whether teams are “in the flow” or not? How is the cycle time/lead time for features/PRs tracking over time? How much work do we usually get done (feature tickets, even PRs as a proxy)? Do we follow process (reviews and approve those)? Do our security checks run? How does our build reliability perform? The question then is: Do we see spikes, anomalies or unhealthy trends? These indicate issue with infrastructure, processes and team health. Those warrant fixing and in turn creates happier and more productive teams.

Just our 2ct. Expected to be updated as “new inspirations” roll in.

‍

Reducing Friction not Micro-Managing

The key to using metrics is to gain insights and confidence into processes, teams, and workflows that are otherwise hard to obtain. The engineering productivity objective should be to minimize any friction in the software delivery pipeline. This means, the steps from planning a feature to delivering it to the customer, and the reaction time to customer feedback/request should be quick, smooth and reliable. Any process or principle bottlenecks significantly outweigh a single contributor's shortcomings and should therefore be the major focus.

Modern engineering leaders focus on the 4 Key Metrics to structure their engineering measurements to get better visibility into their teams and delivery processes. These metrics are:

1. Velocity of delivery (how fast)
2. Throughput of delivery (how much)
3. Quality of delivery (how good)
4. Impediments to delivery (how risky)  

Recently, a new wave of engineering intelligence platforms supports teams and their engineering managers to get the required visibility and insights to create such healthy and productive organisations. By shifting the focus away from individual performance metrics to reducing friction towards productivity goals, companies have achieved double-digit percentages of cost savings after a short period of time. We will dive deeper into this in a future article.