AI-powered chatbots like EQUAL ChatGPT And Google Bard there is certainly a moment—the next generation of conversational software tools that promise to do everything from dominate our web searches to create an endless source of creative material to memorize all the knowledge of the world, so we do not need to do so.
ChatGPT, Google Bard, and other bots like them, are examples of big language model, or LLM and it’s worth learning how they work. That means you’ll be able to make better use of them and better appreciate what they’re good at (and what they really shouldn’t be trusted with).
Like many artificial intelligence systems—such as those designed to recognize your voice or create pictures of cats—LLMs are trained on massive amounts of data. The companies behind them have been quite cautious when it comes to disclosing exactly where that data comes from, but we can look at certain clues.
For example, research articles introduces the LaMDA (Language Model for Conversational Applications) model that Bard has built on, referring to Wikipedia, “public forums” and “code documents from related web sites” programming like Q&A sites, tutorials, etc.” Meanwhile, Reddit want to start charging to access its 18-year written chats and StackOverflow just announced There are also plans to start charging. The implication here is that LLMs have been making extensive use of both sites up to this point as sources, completely free and supported by the people who have built and used those resources. It is clear that a lot of publicly available content on the web has been collected and analyzed by LLMs.
All of this text data, no matter where it comes from, is processed through a neural network, a commonly used type of AI engine consisting of many nodes and layers. These networks are constantly adjusting the way they interpret and make sense of data based on a variety of factors, including the results of prior trial and error. Most LLMs use a specific neural network architecture called transformer, there are some tricks that are particularly suited to language processing. (That GPT after Chat stands for Generative Pretraining Transformer.)
Specifically, a transformer can read large amounts of text, detect patterns in how words and phrases relate to each other, and then make predictions about which words will appear next. You’ve probably heard LLM compared to turbocharger autocorrect tools, and it’s really not that far off: ChatGPT and Bard don’t really “know” anything, but they’re good at finding figuring out which words follow which start to look like real thinking and creativity when it reaches a sufficiently advanced stage.
One of the key innovations of these transformers is the self-attention mechanism. It is difficult to explain in a paragraph, but in essence it means that the words in a sentence are not considered in isolation but are related to each other in many complex ways. It allows a higher level of understanding than is possible.
There’s some randomness and variation built into the code, that’s why you won’t get the same response from a morphing chatbot every time. This autocorrect idea also explains how bugs can creep in. At a basic level, ChatGPT and Google Bard don’t know what’s correct and what’s not. They are looking for answers that seem logical and natural, and are consistent with the data they were trained on.
