I think most people start out using AI the same way they would use a search engine like Google. Ask it a question, get an answer back. This is where AI was notoriously bad in the early days. If the model couldn’t find an actual answer in its training data or by searching the internet, it would just make up a plausible one. This was a pretty common hallucination problem that plagued models only a year or two ago.
One solution to this problem is to simply not use AI for search problems. If you are trying to look up something like who sang a particular 80s song where you only know a few of the words, search is tailor made for that. If you have a question that has a single, simple, discrete answer that is likely on the internet, an LLM can probably find it for you. But in this case, the model is often just doing an internet search for you and handing you the results. It is easier and more straightforward to just cut out the middleman and conduct the search yourself. An added benefit is that a Google search will not hand you the wrong answer with confidence, which is a problem we have all experienced with AI. However, Google is pushing their AI results more and more prominently at the top of just about every search, so it will be interesting to see how search and AI merge over the coming years.
In my experience, the problem with hallucinations has diminished dramatically over the last 12 months. Models have gotten much better at internet searches and finding specific context in their training data. You do need to be careful, because AI models do not always do the best job of vetting internet sources. If you ask a model to cite its sources after it gives you an answer, the source of the answer can often be from a Reddit post or from some other online forum. That doesn’t mean it is incorrect, but it does mean that it probably shouldn’t be taken at face value. Another reason that, for questions with discrete answers, AI should probably not be your first stop.
The best way to get clean, consistent output from an AI model is to make sure it has adequate context to produce the output you are looking for. By building a robust context layer first, you can shape the output in a meaningful way and avoid some of the traps that come from leaving it too open ended. Let me explain with a concrete example.
One of the things I have used AI for in a client setting is assisting me with research for software selection. If a client is looking for a specific piece of software to do a specific job, AI can tremendously speed up the research process. How you execute this can directly drive the quality of the results. For example, a quick and dirty prompt could look something like this:
I have no doubt that Claude, ChatGPT, or Gemini would all produce a meaningful and useful report on this topic. However, what that prompt would produce would also be pretty generic. By doing a little legwork up front, you can dramatically improve the results.
Suppose instead of just prompting it cold, I go to the client and interview several stakeholders. I record these interviews with one of the many AI note takers that are common today. Then I take all of those stakeholder interviews and utilize AI to quickly extract common themes and pain points with the current system. I acquire valuable business context from those interviews by probing for some key questions. Some of those questions might include: how many different SKUs does this manufacturer produce? How quickly do they turn their inventory? How large will their user base be? What are the pain points with the current system? I then take the answers to all of these questions and structure them into meaningful context that AI bases its research on. Prior to doing any of this, I always discuss with my clients how their information will be used and consumed by AI, and make sure we are aligned on confidentiality. There are techniques you can use to anonymize or genericize the information before processing it through an AI model as well.
Modern AI models can hold a pretty large amount of context at a time. As of this writing, Claude Opus 5 (the model I currently use most) can hold 1 million tokens in context. That is roughly 750,000 words, or the equivalent of 7 to 8 novels’ worth of text. The context window includes any text the model is referencing, all of the prompts you are typing in, and all of the responses the model is providing during your back and forth. So 20-30 pages of context is trivial in comparison to the total, but can really help sharpen your results and move them from generic AI slop to something much more tailored to your desired output.
Instead of just jumping straight to conducting research, I would first use AI to create a much more focused profile of the company I am doing the research for. I would then prompt AI with something like this:
I think it is pretty plain to see how this would shape the results in a positive fashion. There still needs to be a human in the loop for the final selection, and I still have run into issues with AI not always getting specific details such as pricing correct on the first pass. For example, one software tool I was evaluating had a lot of module add-ons that were buried in their pricing page. These modules ended up being critical, but also doubled the per-user cost for this specific platform. AI does not always read the fine print.
I always verify the discrete and fact-checkable claims that AI makes before presenting anything to a client or declaring it final. Over the last year, it really is remarkable to see how much less correction I need to do. But at least for now, human review is still a critical step in the process.