In order to solve the pain point of users frequently switching between AI conversation windows, work documents, and emails, Google recently released a major update to the macOS Gemini application, aiming to create a more seamless cross-application desktop office experience.
In earlier versions, the Gemini macOS client attracted a lot of attention with its shortcut keys to bring up floating windows and instant screen sharing functions. In this update, Google has introduced a new voice and screen-aware interaction. Users only need to long press the Fn key on the Mac keyboard to quickly activate global speech mode and turn on the "Smart Dictation" feature.
Different from the traditional word-for-word dictation mode, this function is like a portable real-time editor, which can automatically filter out pause words and slips of the tongue such as "um, ah" when the speaker is at a loss for words, and directly fill in neatly formatted and clearly expressed text into currently active windows such as Slack, Apple Notes, or Mail.
In addition, users can also manually turn on the "Gemini Inference" switch in the application settings. Once turned on, Gemini has the ability to sense the current screen content. Users can directly select multiple PDFs, images, or local documents on the desktop, and use voice commands to extract key information and compose summary emails; they can also highlight text in any application and ask for secondary polishing, or point to an image on the screen to directly initiate fine-tuning and variant generation.
Currently, this voice and cross-application collaboration function has been gradually rolled out to macOS Gemini users around the world. It initially supports English and will be expanded to more languages in the future. Mac users can go directly to Google’s official channels to download and experience:
http://gemini.google/mac
