285. AI is not a Source of Truth

Why do some people hate AI so much? Why won't AI replace software developers? Why is vibe coding not a real thing? These are the questions we will answer in today's episode of Dev Questions.

Website: https://www.DevForge.com/ 
Ask Your Question: https://suggestions.iamtimcorey.com/
Sign Up to Get More Great Developer Content in Your Inbox: https://signup.iamtimcorey.com/

Transcript

Copy Transcript Copied

SPEAKER_01 0:00

Objectively, we know AI isn't always correct. Unfortunately, it's very confident, even when it's wrong. And this, more than anything, represents the danger of AI. It also shows us why AI will not replace software developers. The problem is that the better AI gets, the more dangerous it will actually be. Let's talk about it in today's episode of Dev Questions.

SPEAKER_00 0:36

Welcome to the Dev Questions Podcast with Tim Corey.

SPEAKER_01 0:39

Software development is more than just writing code. So let's talk about the rest of it. Specifically, let's talk about why AI cannot be a source of truth. Societally, we are in a really dangerous place. More and more people are turning to AI as a source of truth. If you're on Twitter and you see someone making a claim, you'll often see dozens of replies underneath asking Grok if that statement is true. ChatGPT has overtaken Google as the first place people turn to get information. In the developer realm, Stack Overflow visits have crashed in favor of various LLMs. The problem is that we're making more and more decisions based upon the word of an AI. A few months ago, I created a course called Navigating AI, where I teach you what AI actually is, how it actually works, and why it hallucinates. We also cover how to work effectively with AI and how to not be replaced by AI. Now, I'm not going to recap the entire course here, but I want to focus on one bit that we cover in the course in more detail. So when AI responds to you, it is guessing based upon patterns it has seen before. It's not sharing something it has learned. It is not creating something new. It's guessing, word by word, what you want to hear. Each token, which a token represents a word or a part of a word, each token that the AI spits out is actually fed back into AI in order to generate the next token. So it's learning from itself, learning from its own answer to make sure that answer is coherent. So that represents an informed guess. But then how that guess is made is also important. The operator of the AI has to make a decision on how creative it wants the AI to be. Not creative enough, and the AI will respond the same way every time you give it the same prompt. That's no fun. Too creative, and it will return bad data. In my course on embedding an offline AI into a C application, we had to play around with the temperature, which is one of the ways you identify how to make the AI more or less creative in order to get different data without getting as much junk data. Once you truly understand this process, it will change how you look at the results coming out of an AI. But let's look at it from another angle. AIs create really good output most of the time. When you ask an AI how to do something, that information comes back in a customized to you way that's almost always something that you can follow. When Grok evaluates something, it almost always gives a coherent, correct answer, even if the ask didn't like the answer. But that's what makes these answers so dangerous. Let me give you an example. Let's pretend I'm an AI. You asked me if sodium was okay for your diet. And I say yes, in a healthy amount, since sodium is essential for life. And then I tell you about the chemical symbol for sodium was Na. And then I recommend that you use Na or sodium in moderation in the food that you prepare. That statement is mostly correct. But if you follow it, you could be seriously injured or even die. You see, sodium is the actual chemical we have in our bodies and other places, but typically we don't get it from, well, we never get it, just from the pure element sodium, because sodium reacts violently with water. Typically we get it from NaCl or table salt or from other foods that have Na as part of their makeup. So technically, yes, we have Na or sodium in our diet, but we never use pure sodium when cooking or when creating things because of its violent nature and the fact that that's not how we need to consume it. So the answer is practically correct. It's practical, it's 99% correct, and yet we could die if we follow it. That's the danger of mostly correct information. You see, it's not that the information is, well, it just got a little mistake. No, no, no. That's a serious issue. It's not, oh, we probably we said, you know, 20 teaspoons of tape of table salt instead of 20 tablespoons of table. That's a mistake, that's a bug. But when you're talking about life or death, when you're talking about major issues, that's more than just, oh, it's mostly correct. As software developers, allowing AI to build software for us based upon our specification is similarly dangerous. The worst thing that can happen is that it works. So this is something that I came across. Recently, I saw an application that was built as a web application that worked. It was a web application that had authentication, so you can't get to it unless you have the proper credentials. Once you got in, it was able to access some data from various sources, collate it, and allow new data to be created or modified. So it worked. It worked correctly. It did the task it was asked to do. It accomplished the goal. And so it seemed like it was a good application. It was secure, it was protected, all the rest. But the reality was that it had exposed multiple API keys. It also had hard-coded uh credentials in the code. There's other problems with it as well from a performance aspect as well as security. You see, it worked and it did the job, it met the specification, and it looked correct. And that's the most dangerous place to be. Because when it looks correct, then you don't replace it, right? You don't fix it. You think, well, it's it's correct because it did the job and it's it's working. But that's a problem. So the common pushback I get on this whole AI is mostly correct and that's dangerous, is that humans aren't 100% correct either. And that's true. But the difference. So humans make mistakes based either upon a lack of knowledge or an unintentional flup. It happens. We do it all the time. In fact, as developers, our primary job is often to track down bugs, bugs that we often create ourselves. So it happens. We make mistakes. We work to mitigate this by having more experienced developers who lead projects. So that's why you see an awful lot companies asking for experienced developers or developers with a number of years of experience. They're looking for people who have seen these things before, have experienced the problems, have worked it out, have filled in their lack of knowledge and learned how to round themselves out and learn how to avoid these bugs in the future. These kind of people we pick to typically lead development projects. And we also do code reviews. So even with lead developers, we have up to read their code often because we need to make sure that you didn't make an accidental flub or you missed something and you have a gap in your knowledge. However, that's not the dangerous type of mistake AI makes. AI will leave things out. It will make assumptions or it will just do things wrong because it's guessing. But it will always make things look right. It makes the logic sound correct. So it's worse yet, it's going to create a working solution often that just looks right. So back to that example of that web application. It looked right. The logic looked right. Things worked. It met the specification. It was even protected. But it was badly, badly wrong. Wrong in a way that a developer wouldn't have done. Or if they did, it was because they had very little experience in building applications. Okay? But because the fact the AI did it and it did it in a way that looked right, it actually made it worse because it enabled something really bad to go into production. So that's the type of problem that AI creates. The bottom line is this AI is a powerful tool and it can do great things. Just be very careful not to start leaning on it as the source of truth. Don't stop your search when AI gives you results. Use that as a starting point. Recently I asked AI, we had a discussion in the comments on my video on random, where one person said, well, point out the fact that random that has been instantiated with a seed was slower than random that was instantiated without a seed. And so I went to verify that. The first thing I did is I asked ChatGPT, is that true? ChatGPT said, no, it's not true. It uses the same logic for both of them. The only difference will be that during the instantiation, it has to initiate that variable and set up and use that seed you pass in. Therefore, it's a tiny little bit of difference in the instantiation. Well, I use that as my starting point. I went into code and create a benchmark where I benchmarked those two and found out that it's about five times slower to use a seed. So it was totally incorrect. But it sounded right. It gave me the algorithm it used, all this information that sounded right. Well, I go back to ChatGPT and I said, this is what I found. And it said, you're exactly right. Of course it did, right? But it said, in.NET 6 this changed. This used to be this way, but now this is the way, and now this is the algorithm, et cetera. So if I had used ChatGPT as my source of truth, it was confidently wrong with great information. Information that sounded right. It was, it had all the information to document what algorithm it used and how it worked and how it created and what the tiny differences might be. That's a problem, right? So verify your results. Use the results coming from the AI as the starting point. Make sure you can validate there are no hidden flaws in the logic or its assertion. For the people who start to get lazy and ignore this warning, they're gonna find themselves making a serious mistake before too long. Make sure you don't fall into that trap. Don't use AI as your source of truth. Thanks for listening. And as always, I am Tim Corey.

SPEAKER_00 11:57

Thank you for joining us for this episode of Dev Questions. When you're ready to learn to think and code like a professional developer, head over to IamtimCorey.com and enroll in a course.