The Bard Exam

Google's answer to ChatGPT and Microsoft Bing is rolling out, and it's pretty good.

I don’t think I can start this newsletter with any other story than the one Google has been driving for the entire week: the release of Bard. Announced back in February, Bard is Google’s answer to ChatGPT and the explosion in popularity of Microsoft Bing. It’s the search giant’s take on an AI chat bot, integrating its LaMDA language models to power the experience. The initial rollout is centered around public figures, Google colleagues, and folks in the company’s Pixel Superfans program.

That’s the group I fall into, and over the past couple of days, I’ve gotten a chance to play around with it. Here’s my super early verdict: it’s good, but it doesn’t answer any outstanding questions people have about the future of AI chat bots.

A lot of early testers agree that Bard is a bit more friendly than ChatGPT or Bing, and they’re mostly right. Speaking with Google’s bot feels a lot more welcoming than with Bing or ChatGPT; there’s a sense that you’re interacting with something instead of just typing commands into a text field. That’s very on brand for Google, a company famous for its friendly approach to hardware and software.

Because Bard isn’t powered by OpenAI’s GPT model, many were curious to see how fast, accurate, and powerful the chat bot is. After all, it’s not like there’s another program using LaMDA we can compare it to. Fortunately, it seems to be just as powerful as GPT, but I’m not sure if it’s all that accurate.

Many of my fellow tech reporters tried typing their names into Bard and asked it for details on their person, and the responses were pretty wild. I especially enjoyed Cherlynn Low’s results that suggested she was a part of the “Diversity and Inclusion Council” at Engadget and has taken part in an interview for The Verge, two things that simply aren’t true.

Cherlynn Low @CherlynnLow

i love that Google's Bard knows I'm passionate about Diversity and Inclusion, but I'm not so sure I like how it's completely made up shit about things I've supposedly said or written. I have not written these blog posts nor been interviewed by The Verge on this topic🤣

There’s an expectation that the information Bard responds with will be highly accurate, given how the chat bot is connected directly to Google’s pool of web data. However, we’ve seen Bing slip up many times despite having a similar setup with Microsoft’s web data, and let’s not forget that Bard famously reported false information on the James Webb Telescope during Google’s launch event for the service.

Because Bard uses various systems and servers to pull information from and compile sentences as its response, the odds of inaccuracies popping up in its responses are particularly high. Over time, this will improve—as evident by Google’s explicit labeling of Bard as an “experiment”—but for now, you have to anticipate it won’t get everything right all the time.

For the record, during my time with Bard this week, I wasn’t able to surface any hateful comments or misleading information, but Google notes that it’s entirely possible to generate that with its model. Again, it’ll get better over time, but it’s worth highlighting nonetheless.

On the bright side, Bard is really useful. You can use it for all the normal AI chat bot things like generating poems, starters for articles, emails, and answers to complex questions. You can ask it to help solve math problems, plan a cross-country vacation, recommend new coffee blends, and more. I tried all of this and I was given useful responses in return.

Of course, because I am who I am, I tried to provoke it with a few questions about controversial figures. None of the responses surprised me or caught me off guard—each was pleasant, informative, and well detailed.

I also had Bard generate a short story on a gangster living in New York, and while it’s by no means the story I was expecting, I still found it pretty entertaining.

So yeah, that’s Bard. It’s a fun tool to play with, and that’s pretty much it. Google will need to continue working on it with improved accuracy and data pools to pull from, much like other chat bots floating around the web.

Is it better or worse than ChatGPT/Bing? I’m not really sure. Bing has a more capable feature set, recently being updated with AI image generation. But it’s not like Bard won’t get there eventually, and each product is still in its infancy. It’s hard to say where both bots will be a year from now, let alone how they’ll each define the future of consumer technology. If there’s one certainty, it’s that they’re here to stay—the form they take is another question.

Let’s just hope by then Google changes its name, “Bard” is pretty awful.