160 Why Do Software Development Projects Fail?

What are some reasons why custom software projects fail? What should we avoid when building a custom application? How do we ensure success for our project? These are the questions we will answer in today's episode of Dev Questions.

Website: https://www.DevForge.com/ 
Ask Your Question: https://suggestions.iamtimcorey.com/
Sign Up to Get More Great Developer Content in Your Inbox: https://signup.iamtimcorey.com/

Transcript

Copy Transcript Copied

SPEAKER_00 0:00

Welcome to the Dev Questions Podcast with Tim Corey. Join us each episode as we tackle the questions you are asking about a career in software development, understanding the industry, and new technology. If you're just starting out or you want to grow stronger as a developer, this is the place to get your questions answered. Now, here's your host, expert developer and online educator, Tim Corey.

SPEAKER_02 0:25

Why do software development projects fail? What are some tips to improve the odds of success for your project? This is a question that came up on the suggestion site, and it's one I want to tackle in today's episode of DevQuestions. Now, if you have a question that you want to get answered, go to suggestions.imtimcorey.com and ask your question there or upvote a question that seems relevant to what you want to hear answered in the DevQuestions podcast or in other types of mediums. Okay, so what makes software development projects fail? Because quite frankly, a lot of projects fail before they even get off the ground. It's tricky to get a software development project right. And so there's been a lot of systems that have attempted to make it more reliable. So we've heard things like, oh, waterfall is bad, and you shouldn't use the waterfall technique anymore. That's that technique of you you figure out the design of a project, you you map it all out, you you figure out all of the design stuff, you design everything up front, and then you build the code, and then you give it to the customer. And that's bad now because the fact we have things like Agile. And the idea behind Agile is it makes, in theory, it makes your project more reliable. It makes it more likely that you'll get something to market successfully. Because you create small little pieces and then give those to the customer right away, and they can say, hey, I want, I don't like that. I was thinking this. And so it allows you to make changes more rapidly. That's the theory. And so that's just one of many tools out there that have come about in order to make projects more successful. Other things are test-driven development, the idea that we write tests before we write the code, so that when we write the code, we can ensure more success in the outcome and not have to wait until it gets into production before we find out there's these odd edge case bugs. Or if it does get into production, we have a way to change things more safely, where we feel more confident in the change we make. That's a little piece of overall project success, but it's also things like issue trackers and Kanban boards, where we can figure out what are the priorities here? And do I want to adjust how much goes into this particular sprint or this particular section of development and how you want to move things around and prioritize things and give numbers to them and all the rest. There's also things like continuous integration where we're constantly testing our code and putting it into the overall code so we can make sure that our changes over here don't affect something over there. There's design patterns, things that we have developed over the years to figure out how to better write applications at size. And then there's things like system diagrams and workflow mapping, and all these tools are designed in theory to help your project succeed. But here's the thing that's really important. None of these guarantee success. And in fact, all of these have been used on failing projects. And also, projects who have done none of these things have succeeded. So these are not magic, magic beans we can throw around and say, okay, this is going to succeed, this will work, this will make our project successful. So, what's the secret? What's the secret to a successful project? The secret is that tools and techniques don't replace good project management. That's something that people often look at is what tool can I buy to kind of buy success? That's not how it works. The secret is that those tools can help you succeed, they can be of value, but the reality is that you need to implement some good project management techniques. So let's talk about those. Number one, start with good requirements. Assumptions kill projects. They're brutal. I've got an example of this that's not in software development, but I figured it kind of illustrate. Okay, so if I come to you and say, or maybe you come to me, you come to me and say, I want to rent a vehicle, and I want to spend no more than $100. And that's your that's your requirements. I might give you a bike. And you're like, no, I don't want a bicycle. I I want something that has that's fast. Okay, that's another requirement you just added to that budget. Okay, I'll give you a small plane. Well, no, no, no, no. That's not what I was expecting. What I was expecting was a car. You asked, I said, I want to rent something, and I in my mind was thinking car. But me as the renter was like, hey, your requirements are cheap. Okay, bicycle. No, no, no, fast and cheap. Private plane, small little plane. Well, no, no, no, no, I want something that drives on the road. Oh, okay. So you want a motorcycle? No, no, no, four door. Like, see, until you really specify really good requirements, what you think and what the other person thinks can be totally different. Because when you said, I want to rent a vehicle, in your mind you were thinking car. You didn't say car, but you were thinking it. And that's where projects can really start out of the gate wrong. Is you have in your mind car, but I don't have that in my mind. And so what I'm thinking through is based upon what you have said, what you've communicated, and then I fill in the rest. I assume things. And so that's a problem. That's something we need to get on the same page about. So start with good requirements that do not have assumptions. You've got to kill as many assumptions as possible, if not all of them. So that's number one. You've probably heard it before, but it's a great place to start, is if you don't start well with good requirements, you're not gonna have a good project because what's gonna happen is you're gonna deliver something the other person wasn't expecting. This is why Agile is seen as such a great process. Not be it's not because it really is that much better necessarily. There are some benefits there, but the the real, you know, the real killer feature, the real thing that's the most important is that you get something in the customer's hands sooner. Because that's when it removes some of those assumptions, where the user actually sees it and goes, that's not what I was expecting. And you can very easily or very quickly pivot to what they were expecting, because you haven't put a lot of effort in yet in the project. So that's a theory behind Agile, why it's such a you know, a popular idea. There's some definite downsides there, and I think people get Agile wrong, but um, but there are some benefits there, and that's one of the things is it's it helps you eliminate some of those assumptions by putting something quick into their hands where they can say, no, I wasn't expecting this, and then you can have a conversation. But if you just have a conversation up front and can talk through, you can sometimes eliminate some of those expectations by being very clear about exactly what they're going to get, or about exactly what you want if you're on the other side of the project. Number two, establish version one deliverables. This is dates and features. So establish exactly what you will deliver, when and what. Okay, so when, what date, and what features. Now, you may have to be careful about giving dates because estimation is always hard. But think of it this way: let's go back to our rental idea. I say, you know, you say to me, I want to rent a car. Okay, now we're on the same page. We got a car rented, I go, cool, no problem. You show up on Monday and say, where's my car? And I say, I don't have it until next month. Well, it's too late now. Like, so if you talk about, and this comes back to assumptions though, too, if you talk about what's the delivery date and how important is it to hit that date. Let's just say that you're building software that has to be running for a specific holiday. Well, if you get delivered a week late, a week after the holiday is over, what's the point? So you have to know is that a hard date or a soft date? Is that a date that can slip, or is that a firm? We have to have it by this date. And that will help drive some of your other assumptions about what has to be in the project, and then also what you do or do not have for features. Because if that's a hard date, then you say, okay, well, we're gonna get this absolute core done by that date. We wanna have these other features done, but if something has to slip, it's gonna be those extra features that we might not have time for because that date cannot slip. So establish a version one deliverable. Understand exactly what that is. Number three, stop feature creep. This is hard. This is really hard when you don't really control the features. Okay, maybe you're just the lowly developer, the person who's just doing the work, and you have a boss or you have management or the project owner that has power over you, has the ability to say, no, you will do this. That's hard, but you need to stop that feature creep. And there's ways of doing it even when you're not in control. So, feature creep is the idea that we're gonna build this one-story house that has two bedrooms and one bathroom. And then as you're building it, so you've poured the foundation, you've you've started building the framing of it, and then you go, you know what? I kind of want to have a second edition, a second story. So it would kind of be almost too late, but you know what? We'll figure it out. And you put somebody on, you know, you know what? I really want to have a bigger footprint for the house. Let's let's add another room out to the side. Well, now it's not under the foundation, but that's okay. We got some rocks we can stick under there and get that to work. And that's kind of what happens when you build projects. You start off with two bedroom, one bath, one-story house, and you start adding these weird features that kind of hang off. And you know, you're already adding bedrooms, so what's one more bedroom, right? It you know, you're we're gonna move these outlets around, but that's not a problem, right? We're gonna move the plumbing around, but that's fine, right? It's not. You know, there's gonna be some consequences of that. Well, that's what happens in software development, is when you start adding these features, you know, you're rerouting plumbing, you're rerouting the electrical, you're rerouting stuff in your code that all of a sudden now you've introduced bugs because you have workarounds to get things from here over to there because it wasn't designed to do that originally, and it makes things messy. So, how do you stop feature creep? Well, the way you stop it is first of all, having that very clear deliverable for version one. And saying, well, that sounds like a not version one thing because it's not part of our original deliverable. It's not part of what we agreed to for this date with these features. And they may say, yes, but we need to have it. And that's when you talk about, okay, that's going to make these changes. Because this is reality, right? If you are building a house and you said, I want to add another room, at the very least, you're adding more building material. You're also adding more time. Because, again, at the very least, you have to cut out more boards. You have to pour more foundation, you have to run more electrical. So, in the same way with a software development project, it's not just a okay, we add more things. We have to figure out, okay, that's either going to take more time or incur more cost, or it's going to remove other features. So if you have a drop-dead date, if you have a firm deadline, you say, okay, what do we remove? Because we're adding something, we have to remove something. This is balancing the scales. You cannot just add to one side and expect the scales not to go out of balance. So, how do you remove, what do we, what are we going to remove? And it's had that logical conversation about, you know, this is not a great time to add a feature, but if it's really important to you, we can do it. And if it's really important, then we'll try our very best to put this in. But we're going to figure out what goes out. Is it that the timeline gets extended, or is it to remove some features? What would you like to do? And put it on their lap. And if they say, well, none of those things, you can't add and not take away. Okay, it's it's a math problem at that point. If it's going to take you five weeks to do this, and you have this plus something else, then that's no longer five weeks. You've got to add some time as well. So have the logical conversation about feature creep, but do your very best to stop it before it even starts. Number four, communicate the danger of changes mid-development. Okay. So that's you're going to have to do that. You have to communicate ahead of time. Hey, listen, this is what's going to happen. So feature creep, remember from previous point, well, plan and communicate ahead of time to say, listen, this is our version one. We've talked about this. Okay. Now, if we make changes to this, this is what's going to happen. You prepare them for the conversation you will have if they try to bring new features in. All right. But on the other side of this is that as a software developer, plan for changes. And this is kind of what you call defensive programming. The idea that, listen, we can't plan for everything. We do not want to write code for every eventuality. But we know that people are going to try and change their minds midway through. Now, some things might be small. Hey, I want to move this button over here, or let's make that button green instead of blue. Not a big deal. But sometimes it's, hey, you know what? I know you build this whole desktop app, but we'd love to have it in the web. That's no big deal, right? So there's small things and there's big things. But if as a software developer, you can kind of prepare yourself for that. Maybe you see that they want the latest flashy thing, or maybe they want to do all this cool stuff and you've backed them down to something simpler, but you know that it's coming back at some point. Maybe you decide, hey, let's put as much code as possible into the class library so we can change out that UI more easily. Or let's build an API for this, because it would help our front end, but we kind of know that that mobile thing that's probably coming back. Like we pushed them off for now. We said, you know what, let's do a progressive web application. But we know at some point they're gonna want a dedicated, installed uh iOS and Android app. Then maybe you plan for that ahead of time a little bit, a little bit of defensive programming. Do not just write for all the eventualities. Okay. Don't put an API in if you aren't pretty certain they're gonna need it. But think that through and try to kind of predict a little bit and maybe even put some slack in the schedule so that you can adjust to it. You can handle some unknown stuff that they throw at you without badly trashing the schedule. Okay. Be a little bit offensive in how you plan out the project. Number five, keep the project small. The bigger the project is, the harder it is to keep in check. Okay. So if you keep it small, you can do more. Okay. Um, this is why microservices, for all their faults, and I do not think that microservices is the right call for most organizations, at least not full scale. But this is why microservices can be a real benefit. Because they're meant to be 100 lines of code, 150 lines of code, that's it. Like that's this little thing. So if you keep it small, you can build one of those in a day or two. You can have it into production in a day or two. You don't have to have as long drawn out process. Well, when you do that, then you can make changes to, you can iterate rapidly, you can grow it over time, you can change behind the scenes what happens or where it logs to or what database it uses. All those things can happen more quickly. Whereas if you say, you know what, I'm gonna build this complete uh CRM, complete CRM, and you decide we're gonna put all the features in here and kind of cram everything in, and you're trying to build it all at once, that's a problem because it's going to take years to get even close to the version one where everything's working. All right. So try and keep it small, which leads into the next point, which is break your product into parts. All right. The example here is when you're building a car. I know you don't build cars very often, but imagine a car manufacturer. If they said, we're gonna build a car, and they start with some metal and some rubber and some plastic and try to assemble the car from that, where they start building gears and then they build that's too big. Instead, what they do is they design the parts they need and they have each part built separately. That's a separate process. And then at some point, they have the assembly line, which assembles all those things together. And yes, some things may get built on there, but very little. It's mostly taking what's already built in pieces and putting it together into the hole. So they have each piece tested, each piece working, and they put it together. In the same way, your application, maybe you don't need to have the entire application from end to end working. Maybe you can have a little piece working and then another little piece working and then another little piece and link them together, hook them together, whether it's adding a new form or whether it is adding a new service. Again, microservices are not the answer to everything. But there is the idea of a modular monolith. The idea that you can break apart your monolith application a little more easily, add things to it over time, grow it over time, instead of trying to build everything all at once. So break it down into parts, see which parts you can build that don't have to rely on every other part being done too, and start from there. Number seven, don't build for Google Scale. This is a trap that people fall in, especially as new developers, but even as experienced developers. You fall in this trap of I need to build it so it works for huge influxes of people. We need to build it so that a million people can hit this all at once. No, don't do that. Build it efficiently. Don't waste efficiency because you can, but at the same time, worry about scalability later when you need it. For instance, Blazor Server has a connection back to the server. So it uses Signal R to talk from the server to the client side. And it's A very small little connection. And a virtual machine can run 10,000 to 20,000 of those connections concurrently. And that's the big thing people talk to me about is well, that's not good enough. That's not good enough. I might have 50,000 people on my site at the same time. Really? Really? No one knows who you are. You haven't launched your domain name yet, let alone have an actual product yet. Let's start with five people at once, 10 people at once. I have over 300,000 closing on 400,000 subscribers. I use Signal R, not a problem. I've never had a scalability issue with Signal R with any of my services that people talk to. The suggestion site uses that Blazor server with Signal R. Not a problem. Even when I announced it and launched it and it was the heaviest hit because everyone's going to it, I'm not having issues like that. Because I'm not really that scale. I'm not Google scale. I'm not the scale where I have, you know, I'm not a public university where we're doing registration on the same day for all 50,000 students. That's not me. Now, if you are, then yeah, plan for that. But until you have that, plan for what you need and then how you can grow, just have that in your mind. For instance, for me, if I were to need to scale up my signal R connection, I can do that using Azure's Signal R service. Well, that's not a big deal, not a big change. I don't have it in place now. And it would take me a little bit to get there. But by the time I start realizing, hey, we're at 5,000, 6,000 concurrent people, I'll start thinking about what happens when you hit 10,000. Okay? So plan to be efficient. Try to write efficient code, but don't make so complex your code because you're trying to figure out how to be super scalable. Worry about scalability later and just plan now for an efficient application. Number eight, learn to successfully build small things before you build bigger things. This is one that people hate to hear, especially when you're first starting into development. But it's important the first application you build should not be a learn learning management system or a customer relationship management system. These are not, these are very large systems. A student information system, no, too big. Pick something small. Build a small project first. Build a very small project first. Build a Windows Form application with no back-end project first and grow over time. Your first project in the real world, your first project where you're building something should not be huge. And you know what? If you're getting hired into a job, you might have to, but practice first. Learn how to build smaller projects first. Because the best teacher for a lot of these things is experience. Now, number nine. All right, balance your optimism with reality. This is what all developers struggle with. The idea that, you know, when you, you know, in the US, we have a holiday called Thanksgiving. And for Thanksgiving, it's basically a food holiday. It's it's great. Um, you know, my mother was always about feeding the whole, you know, the whole neighborhood, basically. We had tons of people over, had tons of food, had tons of pie, everything else. And so there's lots of great stuff on the table. And I could have huge portions. And when you look at all the food there, you're like, I want all of this. I want to have six different kinds of pie. I want to have the three different kinds of turkey and ham and all the two different kinds of stuffing and potatoes and all the rest. And you start filling your plate and you realize I don't have enough space in my plate, I want a big portion of everything. And the expression in US is your eyes are bigger than your stomach. The idea that what you're looking at, what you want, and you start envisioning I can eat all of this, I want all of this. When you start actually eating it, you realize I can't eat all of this. I should save some for later. All right. Or maybe a lot for later because I can't eat all of this. The same thing happens with developers when you look at projects. We look at projects, we say, yep, that's easy. I know how to do that. I can do that. That's a weekend project. I love weekend projects. They last about a month. Because when you start saying that's a weekend project, then you start getting into them, you realize that, oh, well, I have to create a new database. Oh, I have to figure out where to put that database. Okay, now I gotta worry about permissions. Now I gotta worry about setting that up and monitoring that. Okay, now I gotta put logging in place and get that set up and tested and integrated in my rest of my logging. And oh, now you gotta figure out how to connect this to Azure and CI CD. And before you know it, all this stuff, even just around your code, is taking up more time than you expected the whole project to last. But then you start getting the code and you're like, oh, well, how do I do that? And you start figuring that out and testing things out, and oh, that's a little more complicated. And oh, I couldn't do things that way, I have to do this. And now you're starting to write more code, and it takes you know a week to write the code or two weeks to write the code. And before you know it, the whole month has gone by for this weekend project. So balance your optimism with reality. Balance your optimism with what you've seen before. And this is where, again, having an experience of going through and building projects is really important. That's why you should build practice projects. Because if this is your first project, you don't know how to gauge how long things will take. And trust me, after 100 projects, you still struggle with that. But you can say, hey, you know what? My last weekend project took a month. Maybe I should plan for more time than just a weekend. Maybe I should think through in lessons learned and figure out how to better estimate for next time. And it's always going to be something you struggle with because software development is hard. It's not the cut-and-dried, you know, cut these eight two by fours and put them in this order. That's not how software development works. There's a lot of creativity and a lot of problem solving, and you know, banging your head against a desk because of that one little bug. So timing is hard. Practicing and figuring out what it's going to take will help, but at the end of the day, it's still going to struggle to figure out what the time is. So balancing your optimism with some reality of what you've done before and try and get a better time frame or a better estimate of how long the project will take. That way, you won't be setting expectations poorly for those who are depending on the project. So those are nine things that will help with project development. But when it comes right down to it, you need to apply all nine of these things and work hard to communicate clearly, to estimate properly, and to protect what you've said you will do with that, you know, against scope creep and all the rest, so that you can do your best to meet expectations for the project. Because a successful project that gets deployed too late can be seen as a failure. And a project that may deliver on time but doesn't have the features that were expected can be seen as a failure. But that same project, if communicated properly, can be seen as a success. So I hope that helps. I hope that gives you some sparks of ideas to how to make these things better. Remember, you can't just buy a tool to make this go away, to make the project better. Now, tools can be helpful, tools can make your life easier, and tools can help you succeed, but you have to apply these nine principles in order to really have a good chance at success for your project. All right, thanks for the question. Thanks for listening, and as always, I am Tim Corey.

SPEAKER_01 30:41

Thank you for joining us for this episode of Def Questions. Tim is committed to making it easier for you to become a developer. If you would like to help make more content like this possible, please like, subscribe, rate, and share Dev Questions. You can also send your questions to questions at IamTimcorey.com. Until next time, remember, you are too smart and your time too valuable to waste it making all the mistakes Tim did. When you're ready to learn to think and code like a professional developer, head over to imtimcore.com and enroll in a course.