At the MadeByGoogle event on Wednesday, US time, Google launched a new version of its mobile smart assistant, Google Assistant, called Google Assistant with Bard. This new version is a combination of Google Assistant and chatbot Bard, which can answer a wider range of questions and perform more tasks. In the past, it could only answer simple questions such as "What's the weather like?", "Set an alarm" or "Send a text message to Jenny", but now it gets smarter answers from BardAI.

Google Assistant with Bard can access multiple Google apps, such as Gmail and Google Drive, and provide personalized responses to queries based on user consent. For example, you can ask it: "Look at the important emails I missed this week," and it can find the emails you need to know about.

The new feature builds on a mid-September Bard update that allowed the artificial intelligence chatbots BardAI and ChatGPT to integrate with Google’s own apps and services, including Gmail, Docs, Drive, Maps, YouTube, Google Flights, and Hotels, through “Bard extensions.”

Users who have already authorized Bard to access Gmail, Drive and Docs do not need to authorize again when using Google Assistant with Bard. However, those who haven't tried the extension will need to give Bard permission to get responses to personal queries in the assistant app.

Google says that in addition to finding content in your inbox, extensions can be used for personal tasks like making travel plans, creating shopping lists, or writing captions for social media. With this trial, Google aims to study how people use the Assistant in Bard before broadly rolling out the feature to regular users on Android and iOS.

Bard is now available on mobile devices, and users can interact with it in a variety of ways.

Sissie Hsiao, vice president of Google Bard and Assistant, talked about this new feature in an interview. "It can hear sounds through the microphone, it can talk to the user through voice responses, it can see things through the camera, and it can even take actions to help you. It's all on the device you carry with you, your phone," she said.

The executive sees the expansion as a major leap forward for the company's digital assistant, which previously could only perform the most basic tasks.

Xiao added: "Over the past seven years, Google Assistant has helped hundreds of millions of people complete tasks through conversation, such as setting an alarm, asking about the weather, or making a quick call with a simple 'Hey, Google.' Now, with the advent of generative artificial intelligence, we have the opportunity to create digital assistants that are smarter, more personal, and more intuitive. We believe it should go beyond voice."

There are actually three ways users can interact with Google Assistant with Bard. They can ask and follow up with their voice, enter a query, or utilize Bard's Google Lens integration to capture or upload a photo to accompany the query.

Xiao Qianxi said that people use this feature in many unique ways, such as taking pictures of their clothes and shoes and then asking Bard how to design them, taking pictures of the application and then asking Bard to write code scripts. "We want Bard to be multimodal. It can see, it can hear, it can talk to you," she explained.

In addition, on Google Pixel devices and some Samsung phones, you can long press the power button or the home button respectively to bring up a floating window that provides a conversation overlay on the page you are browsing, allowing Bard to respond to what you see on the screen. For example, you can open a picture of a hotel on Bard and ask it if you can book a room at that hotel this weekend.

Bard's integration with Google Assistant makes it no longer restricted by the web version. This means it can now double-check your answers to avoid problems with AI hallucinations. A big problem facing modern artificial intelligence is constructing wrong answers based on wrong information.

Google said it will initially launch Google Assistant with Bard in limited markets, not just English-speaking countries. However, the company has yet to determine which markets or languages ​​will receive support for the update first. Over the next few months, it will be rolled out broadly to iOS and Android phone users, and then explore the possibility of bringing upgraded Assistant features to other platforms. (little)