135 Should I Start My Project Over From Scratch or Update It?

What are the criteria for deciding to start a project over? Is there a time when it is better to throw the old application out and build a new one? Should we just upgrade our application instead? These are the questions we will answer in today's episode of Dev Questions.

Website: https://www.DevForge.com/ 
Ask Your Question: https://suggestions.iamtimcorey.com/
Sign Up to Get More Great Developer Content in Your Inbox: https://signup.iamtimcorey.com/

Transcript

Copy Transcript Copied

SPEAKER_01 0:00

Welcome to the Dev Questions Podcast with Tim Corey. Join us each episode as we tackle the questions you are asking about a career in software development, understanding the industry, and new technology. If you're just starting out or you want to grow stronger as a developer, this is the place to get your questions answered. Now, here's your host, expert developer and online educator, Tim Corey.

SPEAKER_02 0:25

Should I start my project over from scratch or should I update it? This is a question that's asked on a suggestion site. I thought we had a great one to tackle in today's episode of DevQuesti. Now, if you have a question, go to suggestions.iamtimcorey.com, leave it there, or upvote an existing one that fits your question. Hopefully, you'll see your question answered on a future episode of Dev Questions. Okay, so should you start a project over? Let's say you have a project that you know it's kind of old, it's got a mess in there, and you're like, should I just abandon it and start over, or should I try to fix this thing? Let me start with where I start. And that is I start from the perspective of we're going to upgrade this. And let's talk through why I think that, what the pros and cons of each are, and then hopefully I'll give you some better criteria for identifying when it's time to just replace the entire project versus upgrading the project. Okay? So we're gonna start off with a scenario. And this is a scenario that I've changed a few details to protect the innocent, but this is a real scenario that I really did work on for a company. And the scenario is you've got a project that is in.NET Framework 3.5. It's a web forms project with lots of third-party dependencies. Okay. It's a pretty common scenario, actually. And then the actual issues, besides the fact it's web forms, not you know, um, something more modern, and it's on.NET framework even 3.5, not even uh 4 or 4.5. Um it's got spaghetti code. It's got a mess of code all over. There's no test, there's no logging, there's no best practices, there's no class library to house our business logic and data access. It's kind of all just in the code behind for web forms. So that's our scenario. And the question is: do we upgrade this or do we do a uh build a second application where we build it up and then swap it out at one point, you know, overnight, something like that, where all of a sudden it's a brand new application with all new bells and whistles. So let's talk through how to upgrade it. Since we know how we we replace it, right? We'd rebuild the application in a brand new code base and redo everything. It'd kind of be like the a waterfall approach, probably, where we'd say we already have our requirements, let's go ahead and build it. Or even worse, you get into the place where you're like, well, since we're already here, let's go ahead and add these features and change these things, and all of a sudden you're in a real nightmare. But that's the the build process. We don't have to build an application. But how would you upgrade something like this? Let's just talk through the basics of this, and then we'll get into the pros and cons of upgrade versus replace. So what I would do, what I did, um would be I would extract as much code as possible into a class library. So you have.NET Framework 3 or 3.5. Um, you extract as much as possible into a.NET 3.5 class library that you um you know move everything as much as possible out from underneath the user interface. Now, UI code stays in the UI, but business logic, data access, all that stuff, try and get it into a class library. Well, then I'd want to reduce as much as possible the dependencies on third-party libraries because those third-party libraries probably won't upgrade cleanly throughout our entire move. Because we probably want to move to.NET 7. Well, most dependencies aren't going to just move from old.NET framework to brand new uh.NET 7. So we want to reduce as much as possible the dependencies and get down to as few as possible. Next, I'd try to upgrade the entire thing to.NET Framework 4.8, the latest.NET framework. That's still.NET Framework. It's still the older technology, not.NET Core, but we try to move it to.NET 4.8, 4.7, somewhere in there, if we can get it that far. Okay. That'd be the goal for that upgrade process. And that would be a lot of trial and error. And it would be a bigger upgrade just in the standpoint I would have to test the entire application and all the rest, but that's where I'd go next. And then what I would do is I would move as much as possible in that class library, if not the entire class library, over to a.NET 7 API. And then I'd call it from the user interface in web forms. Okay, so now we have web forms calling an API instead of web forms calling our class library. And all of a sudden now our API can be on the latest technology because an API doesn't matter what language it's in, if it doesn't even matter what language it's in. Like you can write it in, I don't know, node, and that's fine. It will still run with.NET framework. So I would try and do that. And then I would try to create replacement pages in probably, I don't know, ASP.net core Razor pages. And then I would slowly create, you know, page or two in Razor Pages, and then replace that page or two over the other site and try and mix those two together. And I'd mix the whole site until finally we don't have any web forms left. We have just Razor pages. All of a sudden, we have a brand new application, really, but at the same time, a lot of the stuff behind the scenes is still a little kludgy. We're still working through it and improving logic and all the rest. But we kind of move that over. Okay. So that would be the high-level process that I'd probably take in a scenario like that. That's a lot of work. And it's a lot of interwork, okay, where you have to have things working together that might not necessarily want to work together. And you're doing some things that, like the upgrade.NET Framework 4.8. Well, that's a that's an upgrade, but not as far as you want to go. You still want to go to.NET 7, which is a far sight different than.NET Framework 4.8 as far as underlying architecture. But you know, the code might be similar, but why do you stop there? Well, because that's where you kind of have to stop to keep web forms running. So you'll do some upgrades like that where it might not give you everything, might not give you all the way there, but you get closer. So now we have that scenario in our heads. Let's talk about the cost to upgrade. Okay, the first cost of upgrading as opposed to replacing, it's gonna take a very long time. It's not gonna be something overnight that you do. All right. It's also means you're gonna work with older technology for a while. So you're gonna stick with the.NET framework for a long time. Until that last web form page is replaced at the very end of the process, you will still have.NET framework code. That's a long time. All right. So you're gonna have to be okay with your team's gonna have to be okay with the idea of we work in older technology, we use older versions of Visual Studio, we use the older versions of C sharp, and we don't have all the latest and grayest stuff because we're using.NET framework for some of our code. So you'll have that dual life for a long time. It's ultimately harder to debug because now you have, you know, when you get to the point where you have an API that's.NET 7, you have your web forms, which is.NET Framework 4.8. Well, now you've got to debug that connection piece as well. And you have to have two applications running and seeing, you know, is is the the API causing the problem, or is the front end, or is it how the front end you know figures out the API and all that kind of stuff? It just gets harder to debug. All right. And it might even make your application worse for a while. So your application might be slower, it might be less responsive, it might not have as many features as it used to. Um, when I went through this process, the the company I was working for was heavily into web forms and they used session for everything. So they put everything in the session object, megabytes of information in the session object that the user would pass from page to page, which was very, very convenient because every uh web app page would know about everything that came before. Well, you can't do that in MBC or Razor pages. There's they're not they're session-let or stateless. So that state you're passing around, it didn't really have a good way of knowing about that. And you could turn on state, but it would cause problems because it's not designed to do that. And now you're trying to force it to do something that you shouldn't want it to do long term. And so we had to create a way to basically look up state, the state that you needed for a particular page, instead. And so now it's a lot more database calls, there's a lot more uh latency as opposed to just having the session object have everything in it. So that caused some degradation of the application's performance and the some degradation of what information you got and how fast and how it worked and all the rest. So there are things that will happen that will could intend could make your application work a little less good, a little a little more poorly if you um try to do this upgrade process. But what are the benefits? Because this sounds horrible so far. Why would I ever do this? Well, there are some benefits. Um the benefits are let's just say you are starting down this process, and you've got to the point where you have the API and the web forms in.NET Framework 4.8. But then your boss comes to you and says, Oh my goodness, we need to add a new feature to the site. You can do that. You can either create a new web forms page and have it talk to the API, add new API functionality, or if you're at that point, you can write that feature in Razor pages and have it be the first page that gets released because it's a brand new page or a brand new feature, a brand new section. So updates can still happen throughout this process, which is important because typically businesses do not say, okay, we're good. We're gonna pause all changes for the next two years while you upgrade. It just doesn't happen. In fact, it's hard to get businesses to pause changes for a week. Because they're like, well, yeah, but this is a bug, I need to get this fixed. Or I really need this new feature, or you know what, we just took out a new client and they need this functionality. It's going to happen. And so being able to make changes to the system while it's being upgraded is huge. Now, also, as you make changes, you can change and implement, or I'm sorry, you can test and implement a change as you make it. So as you decide, okay, we're ready to upgrade to.NET Framework 4.8. Well, you put it in, you test it, and then you do it. So something has improved right then. Instead of waiting until the very end and getting all the changes, you get changes in increments as you go. And with that, work can be done in small chunks. Maybe that's a long process, like I said, but maybe you don't do it continually. Maybe you say, okay, our goal for this year is to get to.NET Framework 4.8. And you do that by March. And you say, you know what, we're gonna go back to get any other updates done, other things we're gonna do where we're not gonna continue to do this upgrade process until next year. And so you can do that if you want to. You can do small chunks of work and say, okay, we're done for now. You can do something as simple as, hey, every time you fix a bug, try and fix some code around it as well. Or write a unit test for that code. You know what? When you start, you'll have no unit tests. After you fix the first bug, you might have one or two unit tests. And for a million lines of code, one or two unit tests seems like nothing. But you've made progress and you've made an improvement, and that small little chunk of improvement will grow and grow until the point where you have dozens and hundreds of unit tests and you have more confidence in the changes you're making. So there's some benefits to be had by even making small changes to your system to improve how things work. So that's the benefits. Let's talk about the other side. What if you decide, no, you know what? I'm not going to upgrade, I'm going to replace. What are the the costs of doing that? Well, it's going to take a long time. You might say, well, Tim, that was the cost of upgrading. And it sure was. But the cost of replacing it is it's going to take a long time. It may almost certainly will take longer. So it's going to be a long time to replace it. And then another cost is going to be that any work done on the old system is going to need to be done on the new system as well. This creates a really scary scenario where you have a moving target for your end goal. So let's say that you are, I don't know, a small little application, but you know, small meaning 200,000 lines of code. And you know, you do X, Y, and Z, you create these widgets. And then, you know, as you're in the upgrade process, a new company comes in and says, Hey, we want to increase your business by 30% if you would do this. Okay, whatever this is. And of course, the company executives are like, this is awesome. We can hire more people, we can we can expand, we can all get raises, whatever. And so they say yes, and they come to you and say, okay, but we need this new feature added. Well, now you've got to add that feature to the old system. And you've got to look at the new system and figure out how to change the new system to also incorporate that new feature, but it has to be created in the new way. So you've got a moving target where the where done is for your new system. And that's a really scary scenario because all of a sudden it feels like you're never going to catch up to done. Because the more changes you make in the old system, which remember, always gonna be changing. You're always gonna have things that are gonna be changing. The more changes you make over here, the more change you have to make in the new system. And all of a sudden, your goal gets a little further out every time. So along with that, another cost to replacing is business logic can be lost. Remember, I said that the system has spaghetti code where it's just complicated. And you say, okay, let's take a step back. I got the inputs, I've got the outputs. Let me figure out if we can just work from that and figure it out and not have to deal with this spaghetti code. And so you do that. And you say, okay, I got this form, it takes in this information and it puts this data in a database after it crunches some stuff. Um, cool. And you get it to work. But I can tell you for certain that unless that is brand new spaghetti code, there's probably some hidden stuff in there because you've probably had bugs over the years where they say, oh, well, that works for everything, but in cases where X, Y, and Z happens. For example, um, if a user is from Alaska, we have to do this stuff. If a user is from Canada, we have to do this stuff. You know, normally all of our users are in the continental United States, um, the 48 states, but every once in a while you get a person from Alaska. We have to take care of that, but it's got some tweaks to it. Well, that you put that in the spaghetti logic. That's why it becomes spaghetti logic. Because you say, okay, you know, in there buried somewhere is if the user is in Alaska, then also do X, Y, and Z. So when you write your code where you have the input and the output, and you figure out I can do this, the problem becomes where you do that and you miss Alaska. You miss Canada, you miss whatever other little things that are buried in there that were exceptions. And what you do is reintroduce the bugs that you've fixed over the years. And all of a sudden, your code is going to move backwards a lot. And you don't even know it because you don't have unit tests and you don't have tests for those edge cases. And so all of a sudden now you've you've got a system that looks like it works. It's the worst kind of system, a system that looks like it works because you start to get confidence and you say, okay, you know, I've I've put in a person from Washington, a person from Texas, a person from Iowa, a person from Pennsylvania, a person from Maine. It all works. And so you're good to go, and you forget about or don't realize that Alaska is a problem or Canada is a problem. And then all of a sudden, your application, if it ever does get launched, when it gets launched, it has those bugs in it, and you'll find those right away. But you'll find a lot of bugs really quickly because you bring this application online in all those places where spaghetti code was there because it was hiding a number of exceptions or edge cases and the fixes for them. All of a sudden you have to fix all of them all at once, and it feels overwhelming, and it can actually cause your application to be rejected and people to roll back to the old application. I have seen this happen. I have seen brand new applications rolled out and rolled back in less than a week. And all that work gets scrapped because they say, you know what, we just can't rely on it. Our business needs to keep running. And it does. And we can't rely on a system that's so full of bugs, this is just a failure. So you gotta be very careful that you don't lose the business logic, you don't lose the bug fixes, you don't lose the institutional history or institutional memory of all the things that need to be addressed for edge cases. All right. And then another cost is that work needs to be completely done before anything can be used. Now, you can kind of, I don't want to say cheat because that's not right. But you can say, oh, well, we're gonna take over this whole department and replace the entire application with a new application. You could do that. But um, you know, doing just that still is build the entire application and put it in place. You're not doing the the upgrade process where you can change features, you can change the the version of .NET, you can change to use an API instead of using the the class library. You can't do those things. You gotta upgrade the entire thing. And that leads to bigger bugs and more bugs and and more cost and risk. Okay. Another uh cost here is actual cost. Okay, your cost in dollars, your cost in time will be higher than expected. And you may say, Tim, I'm great at estimates. I don't care. You're not, you don't understand the future. You don't know the future. You can't predict the future with any certainty. You don't know what bugs are in there, you don't know what hurdles you're gonna have to overcome. You don't know what issues will come up. The cost will be higher than expected. Okay, so I've seen it over and over again people confidently say, well, we've got an extra 30% of buffer here. We're we're sure that even with even the unexpected things come up, we have that buffer. We have 50% of buffer. It almost doesn't matter because it will be higher than expected. You will run into problems. And unfortunately, developers tend to be optimists. They tend to look at a problem and say, I can fix that. Um Example of this was when Twitter was having problems and get the switch over to Elon Musk and all those things. And people were saying, oh, well, I could rewrite Twitter in a weekend. And they were fighting and arguing over, yes, I could. It wouldn't be this big of scale as Twitter, but I could definitely rewrite Twitter in a weekend. No, no, you couldn't. Because it is much more complicated than you think. On the surface, it looks super simple. Just put 280 characters on a screen, you know, mix those together, you're good to go. I saw a flow chart just of how to notify a person of something on Twitter. So, you know, if you're tagged on Twitter, if a person likes your post, whatever the thing is, you know, how do you notify a person? And the flow chart was about 50 steps long on figuring out if you should notify a person or not. Because you might have notifications turned off. Okay, well, then don't notify, right? Well, yes, but you might have notifications turned off only for a group. Well, but what if it's personally tagged to you? Well, and there's the list goes on of all the exceptions, all of the times when you should or shouldn't. And the waiting of that and figuring that out in the flow chart of that was pretty impressive. And that's just for notifications. And when you look at a system from the outside, you tend to say, oh, it's just 280 characters in a list, and we just kind of merge them all together from everybody, and we're good to go. And it's not that simple. In fact, it's much, much more complex than that. So when you're looking at your system and you say, oh, it just does X, Y, and Z, you're forgetting about all the edge cases, you're forgetting about all of the institutional knowledge that went into it, and you're forgetting the fact that it took developers years to come to this point. And yet you in the hubris have this idea that you could do it in six months or three months or a year. Whatever the case. If it took developers 10 years to develop to this point, they weren't just slacking off. Now, yes, they were upgrading systems, they were improving things, they were taking things out and changing it with something new. And so it's not going to be 10 years to replace it, but it will still be a lot of time. So the cost in dollars and time will be higher than you expected. And then there's also the danger that it will never happen. Like I said, it may be that you lose some institutional knowledge and some of that business logic of edge cases where you put it in place and you have all these problems, and so management gets freaked out and just pulls the plug. And that could absolutely happen. But another thing that can happen is that those cost overruns occur and that time overrun occurs. And at some point the business says, we just can't afford to just keep throwing money at this thing. I've seen this happen a number of times, where companies invest one, two, even three years into a project and it just never comes to fruition. So that's a big danger when you're talking about just replace. So there's a lot of dangers when it comes to just replace, but there are some benefits. We can't skip over the benefits. The first benefit, and we're gonna put some asterisks on this, um, you get a clean code base when you're done. So, you know, as you build up your application, you start to have some legacy stuff in there and some things that might not be used anymore and all the rest.NET went through this. The.NET framework was pretty bloated. And Microsoft said, hey, we're gonna start over, and they did, and we're gonna create.NET Core. And so they recame about and looked at how do you make this efficient? How do you, you know, lessons learned, how do you make this a clean code base that is more efficient and more effective, and yet still uses the same C sharp code or as much as possible. And they came up with a result where the.NET Core code, which is now just.NET, um, is so much faster than.NET framework code. It's the same C sharp code itself, but.NET Core executes it so much faster because it's clean. It doesn't have all the legacy of, well, if you're on XP, Windows XP and you're you know using a 32-bit op, forget all that. It's kind of cleaned out all those exceptions. If you're on Internet Explorer, forget that. You know, it's kind of cleaned out all this old stuff, this legacy stuff that was kind of patched and fixed and and you know, jury-rigged to get to work, and it does, but it's not as efficient as it could be. So you can have a clean code base when you're done. Now, I want to stop right there. Like I said, asterisk here. That's a bold claim. You can have a clean code base. That does not mean you will. Because when you're building an application over again, you start off with the best of intentions. I'm gonna do everything perfect. I'm gonna have dependency injection, I'm gonna have separation of concerns, I'm gonna follow all of solid. I'm going to, and then the reality of the world hits, and you start to have to make things work, and you have to implement a new feature that was implemented in the old system. And so now you've got to rip things out and put things back in. Before you know it, you still have a bit of a mess, but it's at least in the newer version, right? So that's what a clean code base will really look like. It won't be perfect, but it'll be better. Okay, so that's the first benefit. The second benefit is you can have two different teams. You can have a maintenance team and a rebuild team. And you can have them both work in parallel. And so you can say, you know what, we're gonna put two people on maintenance only. The people that kind of know the system and know how to keep it working, we're gonna keep communication open so we know when new things happen. But we're gonna put six people on the rebuild team. And that way we can kind of get that done quickly and get finished and complete and implement all the changes that two people are doing, and then eventually we have that new system only. So you can have two different teams working on this. You also get to re-choose your architecture. Let's say you were really tied into um, I don't know, Microsoft Access. I've been there. I worked on a system where it was completely tied to not one, but multiple access databases. C sharp code, actually, I think it was VB code, but your code tied to an old database structure with multiple databases and messy stuff where the databases have been modified and changed and huge tables with lots of columns and all this stuff. And you can say, you know what? Let's pause that. Let's take a step back and say what's best for our team. What's the best architecture look like? And maybe it's MongoDB and we have this document structure instead. Or maybe we use Microsoft SQL and we move to that and have a different database schema and we optimize that. And you know, you get to re-chose your architecture. Maybe you say, you know what, we're gonna prepare for Azure or move to Azure right away so that we have this scalable architecture instead of this limited on-site-only architecture. And you can get to re choose, and that's a big benefit of just completely restarting your application. So this is the time when you get to choose. Do I upgrade or do I rebuild? And hopefully you've seen there's costs and benefits to both. But at the end of the day, I start from upgrade. That's what that's the starting point. So when I come into a meeting where that's what we're gonna decide, or we're gonna start working on, is how do we move forward? I always start from the default of upgrade. Because you get to upgrade in place, you get to see your changes happen as they as they are done. You get to you know keep some of that logic in place so you don't have those bugs. There's smaller changes over time, and all this good stuff that comes with it, even though there's a lot of pain that comes with it. But just because I default to that does not mean I only do that. There have been times where I've come in, I've looked at the situation, I go, you know what? You need to do a restart. You need to rebuild. And there's things you can do and tricks you can do to try and have both systems work at the same time. But at the end of the day, you need to rebuild your system because it's just too far gone, or it's just too difficult to work with, and there's not enough institutional knowledge to make sure we get it right. So there are times when a total scrap it and start over makes sense. Like I said, Microsoft decided that the scrap it and start over method was what was best for.NET. And that's why they built.NET Core while still upgrading.NET Framework. And they kind of have a period of time where.NET Core was being improved and features are being added to it at the same time as.NET Framework was having features added to it and being the primary thing used in production. So they kind of ramped it up and then they stopped.NET Framework at a point once.NET Core was complete enough to be a replacement. So it there are times when it is the right choice. And I think that was the right choice for Microsoft. But just because there are times does not mean it's the primary thing you should do. Instead, I would again default to upgrade your systems and plan for that and plan that journey so that you can keep your management happy, you can keep your business moving forward, you can not have that big leap off a cliff moment where you put it in, hope things work. That's just not a good place to be. So there's a lot of benefits to upgrading your system, even if it does mean you have older technology in place, you keep some of that spaghetti logic, you have to deal with some bad architecture, and you know it's gonna take you a long time to kind of cleanse the system of those old processes and ways of doing things. So that's my advice. Um, I would encourage you to look at your situation as unique and evaluate it honestly. Just be careful that you don't use a bad estimate as your basis for moving forward. Don't just say, oh, well, it's gonna take us six months to rebuild, so let's do that. Because it will not take you six months. Okay, so don't use a bad estimate as your foundation for the reasoning behind making the replace choice over the upgrade choice. All right, thanks for listening, and as always, I am Tim Corey.

SPEAKER_00 34:01

Thank you for joining us for this episode of DevQuestions. Tim is committed to making it easier for you to become a developer. If you would like to help make more content like this possible, please like, subscribe, rate, and share Dev Questions. You can also send your questions to questions at Iamtimcore.com. Until next time, remember, you are too smart and your time too valuable to waste it making all the mistakes Tim did. When you're ready to learn to think and code like a professional developer, head over to imtimcore.com and enroll in a course.