video • 10MIN

Engineering Screw Ups: How (not) to approach metrics

Chris Downard is a stellar VP of Engineering at GigSmart - but even he makes mistakes. Hear from Chris how he screwed up metrics - and what he did to fix his challenges with LinearB.


Conor Bronsdon:
Chris, thank you so much for joining us today for INTERACT.
We're incredibly stoked to have you as a part of this. You've been an incredible partner to Dev Interrupted throughout the lifetime of the Discord, one of our best users and regular contributors. You've got some amazing talks for us in the past and appeared on the podcast. And for those who don't know in the audience, Chris is a.k.a The Panda. He is the VP of Engineering at GigSmart. And Chris, thanks so much for joining us.

Chris Downard:
Thanks for having me. I'm glad to be here.

Yeah. It's our pleasure having you here.
And I know you got a bit of a cold, so I really appreciate you soldiering through and making it here.


So I hear you have an engineering leadership screw up with, like,
maybe a bit of a lesson that you want to share with us today.

Yeah, sure. It's tough being in engineering leadership,
and sometimes you roll out things and show people all kinds of cool stuff that you get access to, and sometimes that works out great, and sometimes you can overwhelm people. So when I joined GigSmart, we were using some tools to pull metrics and all those things, and we were really kind of trying to fine tune our metrics, too. What was the thing that makes the most sense that we should measure and pay attention to? And when I was rolling it out to the team, like, we all sat down for an hour meeting and I showed them all the data they were collecting. Here's all this stuff, and I kind of realized after the fact I got some direct feedback from some people, too, after that meeting, but I showed them everything, which is great. But if you show people everything and you don't say here's the things that are important, and here's why they're important. Data is awesome, but until you turn it into context and information, people don't always make the best decisions with it, right?


Which is really tough. As an engineering leader,
we have all these cool things that we can pull out of thin air, right? How many points are we delivering each branch or per cycle? What's our cycle time? If we're looking at it from some of the Dora metrics, it can be even as simple as how many commits are we pushing through this week? What does our commit activity look like? And I think that one in particular is one that can be really tricky, because when you attach words like activity, it makes it sound similar to productivity, and those are not. Those are not always the same. In fact, they can be mutually exclusive.

Totally. We don't want to be measuring individual
developers by how many commits did you make a week. It needs to be more holistic. And frankly, we'll be measuring teams and identifying where there are issues that we can help with on projects.

Exactly. So I did learn a lesson of this because I got feedback
from one developer that was like, that was a really overwhelming amount of stuff, and it makes me a little bit nervous for what's being measured, and I was looking at it and I was like, well, for me, it's a whole bunch of information that tells us at any given point if we're having trouble in one area, is there supporting data to point us to a solution? Not I'm looking at you as an individual and paying attention to all these things on you because you're not the whole picture, right? Like when you're part of a team, it speaks volumes about what the team is doing as a whole. And if you're, for instance, like a senior engineer who is onboarding a junior engineer and spending a lot of time pairing with them and making sure that number one they're driving when you're pairing, which is important, then all the commits that are being pushed through that are going to be in someone else's name, even though you're spending a whole bunch of time and data doesn't capture that, right? So one of the outcomes I ended up having is there was a developer over the next several months, I didn't realize that I had caused this anxiety, but their commits, they could see their own commit activity. That was the metric that they drilled into and that they really attach themselves to. Even though I wasn't driving, I didn't care. There was just more data that was interesting about our team. And so this particular developer would push commit after commit to commit because it represented being highly active, high producing, etc. And it wasn't... It was a really good example because it wasn't actually married to delivery. Like a really high commit volume did not translate to a really high acceptance and low rejection count type of thing. So it gave us an opportunity. This isn't our first evolution of metrics before we found LinearB as a partner, and it changed the way that I presented data to the team after that because we have access to it. I'm a big fan of open access to everything. You can pull anything you want, but I'm very intentional as a result now about here are the things that we're looking at. Here are things that are indicators that we're struggling. Here are things that are indicating that we are having success. And so it's always more than one data point that relates to that. It is never specific to an individual. It is never showing everybody the full teams metrics on a giant thing showing like here's all the commits that everybody's pushing through, here's all the points they're going through, etc, and so and so's name. Everything is much more of an abstraction now in dealing with the team. And even though all of the team has access to all that and all the leadership team has access to that, I also don't show the leadership team that information that way. Unless there's a particular reason to dive into it, which we haven't really had a need to.

To be honest, it sounds like you learned a couple of really great lessons from it.
I mean, I'm definitely hearing you say having the right tooling and visibility in the right way is important, but beyond that from a leadership lesson, it's very clear that you took this to heart and said, okay, I don't want to overwhelm folks with data, and I want to make it clear that I'm not trying to measure them on arbitrary statistics. I want to talk to them about, like, what can I do to enable them and use these tools in the right way?

Yeah. Absolutely.
It should always be used as something to support. And I think one of the things that I got out of that was is is raw commit data important? And over time I've decided, yes, it is, but not volumes of commits, right? But you can look at that with a lens, like, I can work with our data person and I can say I want to know how many people are really pushing a lot of commits and off hours or on weekends. Are those people I need to be paying attention to because they're putting in too much time and they need a break, right? Those are things that I can use as a leader that I don't have to make visible to the team, but it allows me to show up and act as a coach of like, hey, I see you're putting in a lot of time. Do you need a break? Do you need somebody to help support you? Is there something that we've messed up planning during the week so that I can get you additional assistance so you don't have to do that?

I love that. You're taking that visibility and saying, okay, how can we apply this
to make sure we avoid burnout and enable people to do their best instead of saying we're going to push you to try to hit this number of commits or do X number of things, which is like you said, really arbitrary.

Yeah. Absolutely. I think if it's not committed,
who's getting the most poll request through or something like that, right? But if every single week or every two week cycle whatever it is, you always have the same people that are at the top and they're like, three X, the level of everybody else. Are they spending too much time? Are they working too hard? Are you getting your best work out of them? How do you know that if they're pushing through a huge amount of volume, are they really getting the rest and recovery that they need? Or is that an indicator that they're not actually thinking about the long term things? Like, what's the effect of this? How are we going to be using this in six months? Are they designing the system to the best that they can? It's atypical to even have your top performers always beat the top of the list, right? They just shouldn't be. You should have cycles because you're human and you probably manage humans, I'm guessing. They should have ebbs and flows, and that's totally normal. And you should recognize those cycles and support them.

I like that. You're giving the people contact to the numbers you're seeing
you're saying, okay, the numbers can help me identify where there might be blockers or where someone may be on the verge of burnout. But you need to have the people context because we are, as you point out, we are managing teams. We have to understand how people work and not just try to push them into these arbitrary number statistics. So I love that you kind of brought that context, and as you put it, different iterations about how you're using metrics and how you're explain that to folks, it's clear you evolved and how it does like very excellent sense of what you're going for with them.

Yeah, absolutely. As a leader, your job is to unblock people
and at a high level, that seems really simple. We need to make this plan in order to execute or something like that. But when you really get down to managing an individual, blockers can be a lot more complicated than just technical blockers or getting information from a team. It can be a lot more about this team has been working really hard. These individuals have been really pushing themselves because they're really into this thing, and they really want to get it completed. But now they need a recovery cycle. So how do you work with your product team to give them a catalog after that really hard push that allows them to kind of recover, right? It's like running a really hard race. After you're done, you need to go into recovery. You can't just run another really hard race. You shouldn't.

That's a great perspective. Well, fantastic.
Chris, this has been really interesting, and I'm sure folks are going to find a lot of value out of this. And thank you for sharing this screw up and how you learn from it, because I love that you really took this to heart and had a lesson from it. So thanks for taking the time. We really appreciate it, always great having you here.

It was a pleasure. I hope everybody enjoys their INTERACT.


Explore your dev team's metrics in less than 3 mins!

I'm interested