317. How Managing Software Development Goes Wrong

Is waterfall or agile better? How long should a standup be? Are story points the right way to evaluate developer time? Why is spec driven development not a good thing? These are the questions we will answer in today's episode of DevQuestions.

Website: https://www.DevForge.com/ 
Ask Your Question: https://suggestions.iamtimcorey.com/
Sign Up to Get More Great Developer Content in Your Inbox: https://signup.iamtimcorey.com/

Transcript

Copy Transcript Copied

SPEAKER_01 0:00

Software development is a hard field to quantify. Building great apps that are performant, secure, maintainable, and that meet the need of the customer, it's hard to do all that well. It's as much an art as it is science, which is why managing software development projects is so difficult. Over the decades, different approaches have been tried to manage software projects, from waterfall to agile to extreme programming. The longer you're in software development, the more you see. Each of these systems, as well as many others, have their benefits. But too much of a good thing is often a bad thing. So let's talk about what happens when good systems go bad in this episode of DevQuestions.

SPEAKER_00 0:51

Welcome to the Dev Questions Podcast with Tim Corey.

SPEAKER_01 0:55

Software development is more than just writing code. So let's talk about the rest of it. Specifically, let's talk about bad development systems. And let's start with the foundational principle that almost anything good in moderation is bad in excess. For example, salt can really elevate the flavor of dish. But too much salt ruins it. I just did that recently. The same is true for techniques around managing software development. So let's let's talk about a few. I'm gonna step on your toes, hopefully not too hard. Um, but just think about what I'm saying in these things. All these things can be good, but they're often done to excess. And let's talk about the one we talked about last week because we can just touch on and move on. But spec-driven development. So, you know, I did a whole video on why I dislike spec-driven development last week. And there was some questions, some pushback. And one of the things that people were confused on is the difference between writing good specifications and spec-driven development. Because writing good specifications, that's a good thing. Knowing what your app should do, putting that down in writing so that you can say, this is what it should do, that's a good thing. Being very clear about this is what it'll do, this is how it'll work, these are the inputs you'll give us, these are the outputs we'll give you. These things are really important to do well. However, we often see this gone too far, where what you end up doing is basically you end up doing all the development on paper, where you have such a detailed spec that what you're actually doing is designing your application, but on paper. And the problem with that is, first of all, you're doing all the design up front, and then you're having it developed. And if you think about that, what is the what is the development term for doing everything up front, all the plan up front, and then doing all the all the development next? That's waterfall. And we've moved away from waterfall in most projects. We'll talk about that next. But that's what you suddenly slip into because what you're doing is you're going too far. I have seen specs that are hundreds of pages long. That's no longer a spec. That is a management nightmare. You have spent so much time and so many meetings trying to hammer out the exact details. And what the problem is, is that you spent all that time when it would have been faster to get the structure you need, the overall picture, and then develop a part of it and say, hey, how about this? And let's modify a little bit and let's go back and figure out what we missed in our specification, and then let's do the next thing and the next thing. And that sounds a bit more like agile. And that's more of what we've done. That's why agile development even came about because we saw that when you do all the work up front, all the planning up front, and then you do all the work, what happens is it's a long process. It's months or even years in the process from the idea to when you see the first thing. And the problem is when you start to see things, you start to go, oh, well, it'd actually be great if we did this, right? And when you start to see it, then you go, oh, I want something changed. Because the end users don't know exactly what they want. And so by taking it too far in Spectre and Development, what you're doing is you're saying, you have to know up front, or we have to get it out of you up front. But even developers don't know everything up front. Because there are always going to be pitfalls or things that you didn't anticipate. And that changes what the spec says. And so then you have to say, okay, well, now let's go back and change the spec. And if you're not careful, you don't do that fully, but you just make the changes in the code, and before you know it, the spec is different than the code, and you have a whole mess. And what you're doing is creating a whole bunch of management instead of getting the job done. Now I mentioned waterfall, and that's you know, number two in this list of examples is that waterfall development has got a bad name, but really it's not a bad process. So put that out of your mind. Don't think waterfall equals bad. The reason we think waterfall equals bad is because we're thinking about it in terms of larger projects where you have months of planning before you do months of development before the customer sees anything. But that's not true of every project, right? Some projects are smaller projects. They might entail a week or two of work. Well, that's that can be waterfall. It doesn't have to be some type of other process with more management. But what happened was that waterfall went from it's good in certain circumstances to it's the way we do things. And that may have worked in certain size projects. But then you get larger projects. And if you stick with the same techniques, we were saying, no, what we do is we plan things out and we design the whole plan. We break the plan down to pieces, and then we we work on each piece, make sure it's all right. And then we take all of that and move into the development phase. I was a project management professional, PMP, uh, for a number of years. I've actually given it up now. I didn't want to keep um maintaining it. But I trained on this, and there was a whole series of sections. You know, so the the initial part of the planning was was a two-step process with things underneath that. And then there was the the actual like detailed planning. And that was, I think, 21 steps um worth of work. And then there was, I think, 14 steps of of actual of actual work, and then you did another 12 steps of follow-up, like testing and those kind of things and validation before the final two steps. And so most of the work went up front. There's a lot of these steps you went through to get to the point of even doing any work. And there's a whole bunch of management. And the problem is that when you do all that work, you have to basically have a full-time person maintaining all that planning and design and management, all that meeting work, right? Um, before you even get to the point of doing the job. And this is where if you ever, you know, if you're in the US, you ever drip past a road crew, sometimes it feels like there's, you know, eight guys standing around a hole and one guy digging. And this is what it feels like when you put all these layers of management on is that there's a whole bunch of people having a meeting about work and only you were doing the work, or you know, half the time you're doing the work while you're in meetings the other half the time talking about the work that you did or were going to do or why the work you did was took too long. So this is where, again, waterfall kind of was messy because it had a whole bunch of these planning things up front, and it takes a whole bunch of work to get going. And then once you get going, again, the real world asserts itself because the real world is not this nice digital world with ones and zeros and perfection. It's messy. Things don't work right. Things that you think are going to interface together need something else because they just don't quite mesh. Or you have a you find a bug in a software to you know, switch gears and try something else. There's so many things that can go wrong in the real world and you have to adjust. So, waterfall is another one of those things where there's a whole bunch of management before you even start work. But then we get into the agile world because the agile world is the solution for waterfall. The waterfall downsides, which again, there's upsides, but the downsides of waterfall, we say, okay, we need to do agile instead. And so then out of agile comes a number of these management things that put a whole bunch of ceremony around doing the work that aren't bad in and of themselves, but they can often be taken too far. For example, again, I'm gonna step on toes, sprints. The idea of a sprint is that you have a set amount of time. And let's just choose a two-week sprint because that's a pretty common one. And so you say, okay, we're gonna do work in two-week chunks. So when you work on something, you have to have it done before the end of the two-week chunk. That way you can show the customer, show the user, whoever, get feedback on it, and iterate for the next sprint. And you can overlap these and other things. But that's the typical process. So, what do you do? Usually you spend the first day or so planning out the next sprint. And then you work on the sprint maybe three or four days that week. And then the next week, you can only work through Wednesday because then you have to have everything locked down. So you can do QA and so you can do testing before you launch on Friday or the next Monday. That's the typical process of sprints. So what happens? You start to get more and more management around that, where there's some things that just don't fit that mold. Where, yes, the idea behind this is break the problem down into small enough chunks so that you can do it inside of a sprint. That way you know that you're not doing as massive changes as it takes six months to do, kind of falling back in waterfall. Um, but some things really are gonna need to be that larger thing. And so you have to think about do I break the sprint process for this? Where do I do this out of band and do it over multiple sprints? But then you have this whole process of am I doing that too much? And you have to have meetings about is it okay to go outside the sprint process and for how long and in what branch and how does that work with the other things that are going on, and so on. Before you know it, you're doing lots of meetings to talk about development. And then with that comes the the connecting bit, which is story points. Um, this is one that's that's always is fun to deal with in different organizations. Because story points points, if you're not familiar, the idea is that it's a way of estimating how long something will take. And so you have these points, and the different points mean, again, different organizations do do it differently, but they mean different things. So, you know, one point might be a, I don't know, a half hour, an hour change, two points might be a half a day change, three points might be a full day change, and so on. And what you do is you say, okay, I can do a three and uh and two ones in this sprint or whatever that that might be. Um so what you do is you spend a lot of time figuring out how much is this gonna take? How much is this gonna take? It if if I you know look into this a little bit more, I can probably figure out a little bit better what it's gonna take and do some research on this. And so you start down the path of even investigating what the problem is in order to figure out what a good estimate of the time is. And so then you can figure out the time, but then you stop doing the work and and context switch to the next task instead of just doing the work because that's not how it fits in the sprint. You've got to make sure you get the right number of points on everything. And so what ends up happening is you have bidding wars and you have other things that just make this process a management process that takes up a whole lot of extra time instead of actually adding value. Now, with this is the last step in Agile, or another step in Agile that is another area that's often abused, even though it's a good thing. And that's stand-ups. Where the idea behind stand-ups is that for five minutes, 15 minutes at the beginning of the day, or at the beginning of the shift, what you you all get around, you literally standing because you don't want it to be a long meeting, but you're saying this is what I'm working on. That way the rest of the team knows what's going on without having to go really in depth. And you can go, oh, I can talk to you at that, or or I can collaborate on this, or whatever. It keeps everyone in the loop. That's the idea. The problem is that's not exactly what happens. Daily stand-ups turn into daily 45-minute meetings or an hour meeting. And instead of being a standing meeting, it's a sitting meeting because we can't stand for that long, which is the whole point of a stand-up, is that you want to make sure you don't encourage long meetings. But then you have to get into like, well, let's talk about this thing or that thing and make sure that we're on the same page and let's work through the details and have this whole meeting around this topic. And what you've turned your standup into is just one more meeting. And, you know, not everyone comes in at the same time. So let's let's make sure you delay the stand-up by an hour. So instead of starting at, if it already starts roughly at eight, we're gonna do a standup at nine, which means that people come in, they don't really get a whole lot of work done because they're waiting for the stand-up. And so you're basically wasting an hour, and then you take another hour for the stand-up, and now it's 10, and then lunch is coming pretty soon, and you're thinking about that, and you can't get too deep into something. What you've done is taken a good thing and turned it into a lot of maintenance that probably isn't helping your process. What it's probably doing is just adding another layer. So I can go on, but each of these situations illustrates the point. A good thing can be ruined by doing it to excess. I'm not saying that specifications are bad, they're good. I'm not saying that waterfall is bad, it can be good. I'm not saying sprints or story points or stand-ups are bad. They can all be great. But if they're taken in excess, if they're done too far, then what happens is they become a management nightmare where it is a drain on the team, it slows the process down, and it's more about managing the process than actually doing the process. So what's the solution, right? Uh number one, keep the main thing the main thing. This too often is one that is overlooked. What's the goal? What's the point? Why are we even doing this work? Why are you doing the management? Why are we doing this design, the specification, the the standups, why are we doing this? It's to deploy quality software. So the system needs to serve that. If this, if the thing you're doing is helping you deploy quality software faster and better, then keep it. But if it's dragging you down, if it's slowing you down and restricting you from deploying quality software, then you need to re-evaluate it. The things that you're doing probably did help you deploy quality software better and faster in the beginning. But over time, they tend to degrade and make it worse instead of better. This is why, number two, apply the phrase it depends to managing software projects, not just to the software itself. And this is where I come back to the illustration of waterfall. Waterfall is not inherently bad, it's just not for every project. And quite frankly, for enterprise projects, it probably isn't the right solution for the overall project, but it might be for a feature. So what you do is say it depends for every project. That might mean that you do one feature in a waterfall style and another feature in agile, which that can have its own problems and drawbacks too. I'm I'm not saying it won't. It may be that you say, yeah, it'd probably be beneficial, but for our overall process and for making sure we're doing things right, we might say the same, do the same thing for both of them. That can be okay as well. But that's where it depends comes in. For some teams, you could do different things for different teams. For others, you're like, no, we've got to have in lockstep because it just makes it easier to not think about. But you should look at every situation that you're working in and say, you know, should we have stand-ups? Well, it depends. Are they helping? Are they doing what we want them to do? Or is it just that that's the thing you do? Number three, cut as much management as possible. Too often management tends to creep in and accumulate, right? It just kind of builds up over time. Um, Microsoft, unfortunately, just announced a whole bunch of layoffs to the Xbox division. And one of the things in that note was that there are some uh workers that had 14 layers of management between the decision maker and the person doing the work. That's insane, right? That but that is a very common thing. When you see any industry look at the layoffs they do when they're trying to get more lean, they're trying to be more efficient, what do you often see? Middle management gets laid off. And the reason why is because slowly we accumulate more management. And it isn't a bad thing at first. Management is necessary. It's necessary to have a person that's got a bird's eye view that makes the decisions that are more business focused or focused on the customer, that are in the meetings, talking to the customers, understanding the big picture, and then passing that down the line. That is important. But if you're not careful, what happens is you start putting more and more layers in, which are beneficial at first, but over time aren't. And it's not just people, it's processes. Processes are probably worse because they do it more often. We don't think about it. And even when we're trying to cut down, be lean, we often don't look at our processes. There are businesses that I've worked in. I worked as a consultant for a number of years, and so I've been over a hundred companies working on their team or with their team. And there are some businesses where the majority of their time, or it feels like that at least, is spent in meetings. You know, I joke that it's you have a meeting in the morning that talks about what you're gonna do. You have the meeting in the mid-afternoon that talks about what you've done so far, and then you have a meeting later to discuss why you're not getting enough done. And it's like, because I'm in meetings, right? I could have got the job done, but I've I've been in all these meetings. And I look at some people's schedules and I'm like, man, your your schedule is filled with meetings. And yet I don't feel like that's actually helping you get the job done. And this is why, you know, when I when I worked with people, whenever I try to work with people, I try and say, okay, Monday is my meeting day. And so if we're gonna meet, we're gonna meet on Monday, if possible. I'm gonna try and eliminate as many meetings as possible. The people that work for me, they don't have meetings hardly ever. Because I'm like, I'm gonna let you do your job. My goal is to empower you. If you need something, let me know. But otherwise, do your job. I'll give you what you need, get it done. Because as much as possible, I want to stay out of the way and not have the hovering over the shoulders, you know, layering management upon management onto them in order to try and control what they do. Instead, I want them to do their job. That's really what I want that want done. That's my end goal. The main thing is for them to get the job done, not to be in meetings talking to me about what job they're gonna do. Now, again, there is value to meeting with people, setting expectations, making sure that you've laid out ahead of time what you're looking for. This is all good things. But too much of that turns into a bad thing. Uh, number four, hold loosely to your systems. The system you're using right now may work great. That's awesome. If you're hitting all cylinders, that's awesome. Great. But don't think that's going to be the case forever. There may be different situations come up where your current system does not fit. Or it may be over time your team will change and it does not work well for your new team. And so you need to make sure that you don't hold so tightly to your systems that they become this mantra you have to do. This is the this is what we do, right? You've probably heard that a lot. This is what we do. Why? Hold loosely to those systems. So the bottom line is this that um good management systems often morph into burdensome, bureaucratic nightmares that slow down development that involve more ceremony than actual development. And the way to prevent that is to continually reevaluate what you're doing and adjust as necessary. You have to keep cutting back. Remember that your primary goal is to deploy good software. Keep adjusting your system to help you best accomplish that goal. That way, you'll set yourself up for success no matter what you're doing. Thanks for listening. As always, I am Tim Corey.

SPEAKER_00 22:45

Thank you for joining us for this episode of Dev Questions. When you're ready to learn to think and code like a professional developer, head over to IamtimCorey.com and enroll in a course.