288. The Best Approach for Building a Large Web Application

What is the best approach for building a large web application? How do you approach it so that you have a successful outcome? Do you do it page by page? Feature by feature? Are there other ways to approach the process of building a web application? These are the questions we will answer in today's episode of Dev Questions.

Website: https://www.DevForge.com/ 
Ask Your Question: https://suggestions.iamtimcorey.com/
Sign Up to Get More Great Developer Content in Your Inbox: https://signup.iamtimcorey.com/

Transcript

Copy Transcript Copied

SPEAKER_01 0:00

What is the best approach for building a large web application? How do you approach it so you have a successful outcome? Do you do it page by page, feature by feature? Are there other ways to approach the process of building web applications? These are the questions we'll discuss in today's episode of Dev Questions.

SPEAKER_00 0:26

Welcome to the Dev Questions Podcast with Tim Corey.

SPEAKER_01 0:29

Software development is more than just writing code. So let's talk about the rest of it. Specifically, let's talk about the best approach to building large web applications. Now, I've done this a number of times for multiple different companies, both as a total rewrite as well as a brand new web application. After having done it multiple times and gone through the process of completing and maintaining these sites, there are some best practices that I've seen work really well, as well as some practices that we should really avoid when building web applications, especially large ones. So let's talk about it. Number one, have very clear, written plans for what the first version of a site will be, as well as what future enhancements are likely. And I'm very deliberate there. So very clear written plans. I have done this before. I've made this mistake where we've had a discussion about what the site should be. We talked about it, and then I went off and built it. And I'm very certain I had the right notes. But the client thought something different. And a lot of it comes down to assumptions. And so when there's nothing written or not enough written, then it's a, well, I remember this, well, I remember that. And the person who may be misremembering things is also the person that's responsible for paying you. So you know what? You have to go back and figure out how to make that work. So written plans will definitely help. And this includes whether you're a consultant or whether you're working for a business. Get it in writing. But then it's also about the first version of the site. Because when you are talking to someone, say, we're going to build this brand new web application for you. They want the moon. They want everything. They start dreaming about all the things they'd like to see in this web application. And it this list keeps getting longer and longer and longer. And you can cut that short and say, no, what's the basics? But what will happen is they'll keep adding to the basics over time. And they'll have in their mind a picture of what that basics might look like, but it's a bit more fleshed out than what you were anticipating. So again, in writing, but talk about the first version as well as potentially what future versions might have for enhancements. So this way you're not saying, no, we'll never have that, but you're talking about what's important to have right off the bat. What's the bare minimum? We talk about the minimum viable product or MVP of an application. This is important and something you strive for, but also version version one should be not much more than that MVP, if not just that MVP. You don't want to have something where you have all the features necessary. What happens is when you try to get all the features necessary in version one, it never gets released. And I've seen this happen where a company will take six months, a year, 18 months, two years, and the product isn't released. And what happens? Your product is out of date before it even has its first user. Don't do that. Don't get caught up in that. You want your application out in front of people being used as soon as possible. So you want to have that minimum version first. But then talk to the customer and say, hey, what might you want to have in future versions? Or when they say, hey, I would love for it to do this, you go, you know what? That'd be great. I don't think it's for version one, but let's put that in a list for a future enhancement. You can include the future enhancement as part of the process, but version one needs to be something that can get out the door that works. Get that out the door first and then add to it. Having that future list, though, will help you with shaping how the application is built today in preparation for those next steps. Don't go too far down the road of anticipating what might be, but still have an idea of where you might be going. That way you can make sure that you don't make change changes that kind of force you into a corner when it comes to future enhancements. That's number one. Number two, this is why I see people skip a lot or do in the wrong way. Draw the site by hand or using a sketch style application. I prefer by hand in a legal pad. I am not an artist. I am nowhere near close to an artist. And that's actually a good thing for this step. What you want is an extremely rough design of what the site might look like and what features it might have and how it might work. What you want to do is give the user something that they know is not the final product that they can look at, they can physically touch, or virtually with the mouse touch and see what would happen. So I will say, okay, if you click this button right here, I will flip three pages and say, you go to this form. This is what it will look like. And talk them through the process. Get their sign-off on that because what you have in your mind and what the customer has in their mind, again, whether this is internal to your company or as a client for a consulting job. Either way, what they have in their mind will not match up with what you have in your mind. So get it on paper. But do not get caught up in using fancy tools that look too close to good or that even are drag and drop site builder type things that make it look like a real thing. You want to be at the most in a sketch style digital drawing. Excala does this really well. So if you use Excalibur, it's the drawing tool I use online. It does really good at giving you sketch style boxes and making sure that people know this is just a back-of-the-napkin type drawing. The reason why is because if it looks too good, then your client or your boss or whoever is going to think that the site is almost done because they see it working. And they don't know that that's the tip of the iceberg and that you still have 90% to do. And so the expect is hey, that 10% that they think is 95%, that took you three weeks to do. I'm expecting this done to be done by the end of the month. And that's nowhere near realistic. But it sets a bad expectation because the user doesn't know any better. Even if you tell them this is just a this is just a sketch, the fact that it almost works or it works kind of is a really bad thing. Avoid it at all costs. That's why I like the piece of paper idea. People understand that a sketch on a piece of paper is nowhere near done. The closer you get to done, the more likely they are to misunderstand your potential timeline. Number three, build a vertical slice. So in game development, a vertical slice would be one single maybe level or one single part of a level where you have the controls working, the character working, you have some enemies involved, if it's a shooter or something like that like that. You have maybe a puzzle or two, if it's a puzzle game, you have a little bit of everything. So you can see what this game might look like. Well, when you're building a big web application, you want a vertical slice that gives you an idea of what this application might look like, how it might work. So you want to have authentication, you want to have a little data access, you want to have a little bit of display, you want to have a little bit of your style and structure for the site in place. You want maybe one page, a login page, and then one actual page that's got a little bit on it. This is not to have a complete thing. This is to have just enough to show you the process of what your site might look like. This also gives you the ability to test out how are we gonna do auth? How are we gonna do data access? How are we gonna do CSS? How are we gonna structure the site? How are we going to lay out our pages? What are the things we're gonna do for each of these various parts? That gives you the patterns that you will then follow for the rest of your application. It allows you to test every part and it gives you the pattern to follow for building everything else. Once you have that vertical slice in place, then get feedback on that. Make sure that people are understanding internally to your team as well as externally with your users. They understand, yes, this is what we're pursuing, this is how it's gonna look, this is how it's gonna work, these are the processes we're gonna do. And so they can use that little vertical slice to go, yeah, I like that, or ooh, I don't like how it's doing this or that. You want to change it when it's the least amount of code, the least amount of design, the least amount of pieces in place. Because the more pieces you put in place, the harder it will be to make foundational changes. This is where you can see if those foundational choices you made after steps one and two were right, and if you need to adjust the foundation before you start building on top of it. So build a vertical slice. Number four, start building up parts of your slices. So start handing out to your various team members, say, hey, start building these pages, start building these features. And you can kind of divide up the work from there because now you have a central idea to look back on, a central pattern you can look back on and say, hey, yes, this is how we're going to build the application. So I know how authors are gonna work, I know how data access is gonna work. So if I my page needs authentication, cool. I can pull from that. If my page needs data access, I know how that is gonna get done. If my page needs to display data, I know how to do that. So you know how to do the various parts. Now you can have multiple teams or multiple people working on various parts at the same time while still working in the same direction. So this is how you get a lot of work done because you can hand out more work and do more things in parallel now that you have that vertical slice to use as your North Star, as your direction. So that's the next step. And after that, you start putting these things into place and getting things up to speed. Now, are you gonna do Scrum or are you gonna do waterfall? It doesn't really matter at this point. Although I encourage you not to do a long waterfall process, but at the same time, don't get so caught up in Scrum that you lose sight of what you're trying to do here, which is build small parts at a time. Find what works for your team and your situation and get it done from there. But that's how I would build a web application. Now let's talk about best practices in pitfalls. So, best practice, put as little as possible on the page. Too often I see people fail at this. They get caught up in the idea of I need to have a grid. And that grid needs to have all of our user records in one spot. So I can filter and sort and all the rest on that grid be great. Until you have 100,000 users. And then it's gonna be a problem. You might say, but I need all that information. No, you most certainly do not. Work harder at figuring out how to fake it. Yes, you may look like you have all than 100,000 users on that grid, but you don't need to have all 100,000 users. Also, why are you including all the information you are? Reduced to just the information the user needs. I have seen so many data grids, and this is a big one. Uh, I've seen so many data grids on web applications that have hundreds, thousands of records with a dozen different properties or more per row. And yet the user, all they're doing is saying, okay, give me the next one and clicking on it and doing work on that. Why do they need to see or have downloaded a hundred thousand records to pick the next one in the list? They don't need that. Why are you showing them 12 pieces of information or more when all they really need to know is what's the next ticket to work on or what's the next user to process? All you need probably is if it's a user, you just need the first name, last name, maybe, um, and the date that it came in. That might be all you need for a ticket for a help desk. You probably just need the title and the date. That might be all you need on that grid. Now, you'll need more information than that when you load the rest for when you actually show a ticket or when you work on it, et cetera. But you don't need all that information in the grid. Keep as little as possible on the page. Number two, keep things simple. Too often I see people that love to create complexity. And part of this has to do with inexperience in building these full-scale applications. And part of it has to do with learning on your own or even in colleges or boot camps where you learn how to do these complex things. You learn all about design patterns and you learn about these algorithms and other things that they can really enhance your code. Well, yeah, in certain circumstances, not all the time. Here's a here's a really simple example. For 99.9% of your applications, you need to build one monolith. And probably you don't need to have the full application be a single-page application with all these bells and whistles. You probably don't need that either. You need something simple. And bringing in all this complexity for the we might need it when we become the size of Google, just not necessary. And in fact, it's hurting your customers, it's hurting your process, it's hurting your ability to get more features out faster. Keep things simple. That's practice number three. Keep your dependencies light. It's really tempting to just use the libraries that are available. This came to a head a couple of years ago, I believe it was, when there was a dispute over the package left pad. And the author said, fine, I'm going to remove it from npm. I'm going to destroy it, get rid of it. And half the internet went down because half the internet was dependent on left pad. What is left pad? It's a way to pad a string in JavaScript on the left. It really is like one or two lines of JavaScript. That's it. And yet people were using that instead of writing the one or two lines of JavaScript. So when that NPM package went away, it crashed applications. It crashed build processes. So what did they do? They actually restored the package and told the author, tough luck, we're taking over. That all came about because a bunch of applications were depending on a package that did practically nothing. Keep your dependencies light. Focus on the things that you really can't do yourself. There's a need for dependencies. Not every web application can build everything on its own from the ground up. It's just not practical or efficient. But you don't need a dependency for everything. Rethink how you do things, try and trim those down to as few as possible. If you're working with a UI package for one particular thing, you need to have it, try to work with that same vendor or that same open source maintainer for other packages in the same line. That way you're dependent on one thing or one vendor or one maintainer instead of one here and one there and one another place because you've now tripled your potential for problems. Number four, get feedback as quickly as possible. Don't wait until you're done. Get feedback as early as possible in the process. So this is where that vertical slice is really helpful because you get to show off as much as possible in the across the board things that might happen. So you've got your login form, you've got your one page, you might have a grid on there or something to show off how you're going to do data access and working with data. So you've done a vertical slice, and now the user comes across and says, Boy, you use pink everywhere, and I hate pink. I want purple. Okay, you've learned early. You can make a change. Or they might say, you know what, I really don't like that font. Okay, not a problem. We can make the change now, and we don't have to review a whole bunch of controls and systems and pages and all the rest to make sure that font change didn't affect spacing and sizing and all the rest. Or they might say, you know what, I don't want to use that login system. You're using this login system, I want to use a different one. That could be a big deal unless you're checking things early. So get feedback as quickly as possible. And then number, what was it, number five, I think? Yeah, number five. Utilize the documented plans as guidelines and protection. So I said earlier to get things in writing. Use that as your checklist for are we done yet? And as your protection when your customer, your boss says, Well, I wanted it to do this. Come back and say, I didn't know that. We talked about it doing this, this is what it does. So if you want to add that, we can do that. It's going to take extra time. And talk about that. Have a meeting about adjusting the time frame because of that change. And if they push back and say, no, no, no, I want to have that included because that's that was included. Say, here is in writing what we said we were going to do. Where does it say we also do that? Can you show me that? And they can't, then you say, so what? Not a problem. We can talk about adding that, but now we're adding something new. That was, we did not have an expectation of this. And so these are our expectations. We are meeting those expectations. So if you want to add new expectations, we'll have to readdress how that affects the timeline. So it's both guidelines for you and protection for you. So make sure you utilize that documented plan. Now, those are the best practices. Let's talk about some pitfalls. The biggest pitfall for any project, and especially in a bigger project, is scope creep. It's so easy to do. It's insidious how easy it is to have scope creep, where you can do it as a developer. You say, you know what, we didn't talk about this, but it makes some things faster. So let's put that in. That's still scope creep. Even though you might say, well, it's gonna make things faster. Is it though? Have you thought through the long-term ramifications of that? Is there something else down the road that it might take more time to do? Are you sure? Talk about it. Because that can actually cause some problems down the road. I've seen that happen. Okay, another thing is when the customer says, boy, I would love for it to do this. Or, well, isn't it going to do that? Again, coming back to your protection, your in-writing stuff, make sure that you say, we hadn't talked about that. And if they say we want to add that anyway, not a problem. But let's talk about how to make an adjustment. Or let's talk about putting that in the next version. Put that in a list. So having that extra list can be really helpful for avoiding scope creep. Put it in the next list. Don't put it in a list for now. Put it in a list for after we deploy, then we'll talk about the next version. Version. Okay, so watch out for scope creep. Be really careful about it. Next thing to avoid is changing tech midstream. This can be super tempting. So you start working on a project and then out comes a new version, or there's something better that comes out. And you're not that far down the road. So you know what? Long term, it's getting better. Again, that's kind of scope creep, and it's also problematic because you might be removing or removing things that you did or changing things. And all of a sudden, now you're throwing out work that you've done in order to have the latest and greatest thing. And especially a big project, you're going to run into the case where your project is not complete before it's outdated. And the temptation is to always make it up to date or change things or bring new things in or change how things are done. There may have to be some of that. But if at all possible, get the first version done. Then look at do we need to upgrade or change? The exception to this would be if you find out midstream that there's going to be major breaking changes in your technology. I had this happen with uh Angular One. So I built a whole project in Angular One. I was about done when Google came out and said, you know what, for Angular 2, we're making everything breaking changes. Basically, it's a whole new thing. So Angular One became Angular JS. And then Angular 2 and beyond became Angular because it was that much of a break. And I had a decision make. Do I basically rewrite everything and redo how we structured things in the whole nine yards? The answer was no. I stayed with AngularJS and just finished building that because we needed to get something out the door. Our customers needed this application. We needed to get it done rather than fight with the latest version of the next thing. And it took them a number of years before they moved to, they actually moved to React instead of Angular, whatever the version is now. So they changed technologies, but it took them a while to do that because it's more important to get the application out and working. And in fact, that application worked for a number of years and was useful for a number of years. It made the company money for a number of years before they ever upgraded to React. So changing midstream is a dangerous thing to do. Be careful not to do that if you don't have to. If you're building a.NET application and you build it on, let's say you start on.NET 8 and you're building on.NET 8 and .NET 9 comes out. The temptation is, let's upgrade .NET 9. I would encourage you not to until you have version one done. And maybe you get to the point where you're like, boy, it's going to be.NET 11 before we're done. Well, maybe that time you can think about because.NET's upgrades are so easy. Maybe you can think about seeing what it's going to take to upgrade a nine and then a 10. Because it might be easy enough that it's worth it in order to make sure you don't have things that are deprecated, make sure you're still in a support, et cetera. But there may be a case for saying, no, we're not going to upgrade to those new versions. We're going to try and get this out earlier on the current version. Maybe you can get a.NET 8 application out by the time.NET 10 comes out, which is out by the way now, but maybe you can get that.NET 8 application out by the time.NET 10's out. And then the very next thing you do is upgrade to.NET 10. Because if maybe if you upgrade to.NET 9 and upgrade.NET 10, maybe your application is not even out until.NET 11. That's a problem. So get it out earlier, which means avoid changing tech midstream. Now, pitfall number three is doing too much in the first version. This is another one of those things that's very common, very easy to do, is you say, well, but this is essential. But that's essential. But we have to do this. We have to do that. Before you know it, you have your full application because we have to have all of this stuff. No, you don't. So cut that back as much as possible. Don't do too much of the first version, or the first version will be the never version. Get something out the door that works, that does the job, and add to it. Because adding to it is much, much easier than trying to put all those things in before anything works. Because otherwise what happens is there's always one more thing to add, one more thing to change, one more thing to debug, and your application never deploys. The final pitfall to avoid is complexity without enough added value. I talked about this earlier on the positive side, but this is the negative side of things like microservices. They can be valuable, but you probably don't need them. They add a lot of complexity. They add a lot of overhead. They add a lot of performance hit. And you might say well no microservices make your application more performant. No, they don't. They make your application slower. It's just that you can scale them separately so you can put more resources towards making your application faster by paying more money. You don't get a faster application by using microservices. You get a slower application because an application that talks internally to itself is always going to be faster than an application that talks across a network to another application. So that right there shows you that microservices will make your application slower. It also makes it harder to debug. Instead of debugging one application you're debugging hundreds. That makes things harder. So don't add complexity without making sure that it adds enough extra value. So this is not just a oh will that be valuable that's not good enough. You have to figure out does it add enough additional value? So avoid that pitfall of complexity. Your app should be as dead simple as possible. Don't overengineer it you're not going to need it. Building a large web application can be a hard thing to get right. At first you're enthusiastic and optimistic but if you aren't careful the reality will be depressing and the problems will bury any chance you had for success. So follow these tips and you'll give yourself a great chance for that success. Thanks for listening and as always I am Tim Corey.

SPEAKER_00 28:58

Thank you for joining us for this episode of Dev Questions. When you're ready to learn to think and code like a professional developer head over to Iamtimcorey.com and enroll in a course