Today, let’s talk about the advancement of Bard, Google’s answer to ChatGPT, and how it solves one of the most pressing problems with chatbots today: chatbots’ tendency to make stuff up.
From the time chatbots were introduced last year, their makers have warned us not to trust them. The text generated by tools like ChatGPT is not based on a database of established facts. Instead, the chatbot is predictive—making probabilistic guesses about which words appear to be correct based on a large corpus of text trained on its underlying large language model.
As a result, chatbots often "confidently make mistakes," in industry parlance. This can fool even highly educated people, as we saw this year with a lawyer submitting citations generated by ChatGPT - not realizing that every case was made out of thin air.
This situation explains why I find chatbots mostly useless as research assistants. They'll tell you anything you want in a matter of seconds, but in most cases won't cite their work. Therefore, you need to spend a lot of time researching whether their answers are true-which often defeats the purpose of using a chatbot in the first place.
When Google's Bard launched earlier this year, it featured a "GoogleIt" button that submitted your query to the company's search engine. While this allows you to get a second opinion on the chatbot’s output more quickly, it still puts the onus on you to determine whether it’s true or false.
But starting today, Bard will do more for you. After the chatbot answers one of your questions, click the Google button to "review" your answer. Here's how the company explained it in a blog post:
When you click the "G" icon, "Bard" will read your answers and evaluate whether there is content on the web that corroborates your answers. If the claim can be evaluated, you can click on the highlighted phrase to learn more about supporting or contradictory information found by the search.
Close inspection of the query turns many sentences in the responses green or brown. Replies highlighted in green link to the referring web page; hover your mouse over one of these pages and Bard will show you where the information came from. Replies highlighted in brown indicate that Bard does not know the source of the message and the highlighting may be an error.
For example, when I double-checked Bard's answer to my question about the history of the band Radiohead, it gave a lot of green-highlighted sentences that matched my own knowledge. But it also turns the sentence brown: "They have won numerous awards, including six Grammy Awards and nine Brit Awards." Hover over this sentence and Google search results reveal conflicting information; in fact, Radiohead have never won a single Brit Award, let alone nine.
"I'm going to tell you about a tragedy that happened in my life," Jack Krawczyk, Google's senior director of product, told me in an interview last week.
Krawczyk was cooking swordfish at home, and the smell from the cooking process seemed to permeate the entire house. He used Bard to look up ways to deodorize, then scrutinized the results to separate fact from fiction. It turns out that a deep clean of the kitchen doesn't solve the problem, as the chatbot originally said. However, keeping bowls of baking soda around the house may help.
Krawczyk told me that because people use Bard in so many different ways, double-checking is often not necessary. (You wouldn't typically ask it to double-check a poem you wrote, or an email it drafted, etc.)
While double checking represents a clear improvement, it still often requires you to pull out all citations and make sure Bard is interpreting those search results correctly. At least when it comes to research, humans are still holding AI’s hand, just as AI is holding ours.
However, this is a welcome development.
"We may have created the first language model that admits it made a mistake," Krawczyk said. "Given the importance of improving these models, ensuring that AI models accurately admit mistakes should be a top priority for the industry."
On Tuesday, Bard got another big update: It can now connect to your Gmail, Docs, Drive, and a few other Google products, including YouTube and Maps. The so-called extension allows you to search, summarize and ask questions in real time on documents stored in your Google account.
Currently, it's limited to personal accounts, which greatly limits its usefulness, but as an alternative way to browse the web, it's sometimes fun -- for example, when I asked it to show me some great videos about getting started with interior design, it did a great job. (It's nice that these videos can be played online in the Bard answer window.)