The Google I/O 2024 keynote is behind us, and Google had plenty to say during the show. The keynote lasted around two hours, and Android was just a tiny portion of it. Google will talk more about Android separately tomorrow, so we’ll add that to the article when it happens. For the time being, we’re focusing on the Google I/O 2024 keynote, and that’s plenty, trust me. Google talked a lot, as it mentioned AI 121 times. This didn’t really surprise anyone, but what did surprise people is the fact Google actually counted and shared the info at the very end of the keynote. The company’s CEO even joked about it, as he knew someone out there was keeping count. In any case, let’s just straight into it. We’ll cover new Google I/O 2024 announcements here, briefly, and link to dedicated articles where needed. You’ll find new announcements separated by headings.
Google Photos search getting smarter thanks to ‘Ask Photos’
Google has announced a new feature for Google Photos during the keynote, the ‘Ask Photos’ feature. This feature will basically help you find images with far more precision. AI will be a part of it, and it should make the search function on Google Photos far more useful. The option will be pinned down to the bottom bar, and the results won’t be based only on the location where the photos were taken. Gemini will use its judgment to show you what’s relevant.
AI Overviews coming to a billion people
The AI Overviews feature will become available to a billion people in Google Search by the end of 2024. AI Overviews, for those of you who don’t know, have been known as the ‘Search Generative Experience’. The feature is coming to users in the US, and will soon roll out to the rest of the world. You’ll be able to see an AI-generated s ummary at the top of many of the search results you get.
Gemini AI features coming to Gmail
Google is bringing Gemini AI features to Gmail, making the service smarter. The company announced a trio of features is coming. The ‘Summarize’ feature will be able to summarize longer threads. The ‘Gmail Q&A’ feature is also coming to make sense of emails and help out. While ‘Contextual Smart Replies’ will look through emails and provide more customized and detailed suggestions.
Gemini is coming to NotebookLM
Gemini AI is also coming to NotebookLM. It will be able to generate audio discussions based on text material. It will allows students to join the session in order to grab examples pertaining to the course content. In other words, students will get a truly interactive learning tool with this integration.
Gemini 1.5 Pro improved & expanded
Google announced that it has both improved and expanded Gemini 1.5 Pro. It has now expanded to 1 million tokens to all developers, globally. It’s now available with Gemini Advanced in 35+ languages. The company is also expanding the number of tokens to 2 million, but in a private preview to developers. Gemini 1.5 Pro is also getting injected into Google Workspace, as it’s now available in Labs.
Gemini 1.5 Flash, a trimmed-down Gemini
In addition to talking about Gemini 1.5 Pro, the company also announced the Gemini 1.5 Flash. This is basically a lighter-weight model of Gemini 1.5 Pro, as Google referred to it. It’s optimized for tasks “where low latency and cost matter most”. Developers can use it starting today with up to 1 million tokens in Google AI Studio and Vertex AI.
Project Astra, probably the coolest demo we’ve seen today
Google managed to surprise many with its demo of ‘Project Astra’. This is basically Google’s new project in which it utilizes the camera interface and combines it with Gemini AI in order to create AI with eyes, kind of. In the demo Google shared, the person was able to converse with Gemini AI while walking around with the camera UI and pointing it at things. The person was able to mark down some items and ask questions about them, or get other answers based on what’s in the viewfinder. It’s kind of Google’s competition for GPT-4o.
Imagen 3, the most advanced image generation model yet
As part of Google I/O 2024 keynote, Google also announced Imagen 3. That is Google DeepMind’s “most capable image generation model yet”. It is able to understand the way people wirte, and create more photorealistic images, says Google. The company said that it “produces visuals with incredible detail, realistic lighting and fewer distracting artifacts”.
Music AI Sandbox, simplifying music creation
Google wants to help artists create music, and that’s what Music AI Sandbox is here for. The company partnered up with some creators from YouTube in order to demo how this will work. Creators can request specific patterns to be provided for them, in a specific way, as a tool to help them create music. Based on the demo, it works really well. Google also says that it worked “closely with musicians, songwriters and producers” in order to make this happen.
Veo, a new generative video model
This is also a part of the Google DeepMind announcement. It allows users to create and edit HQ videos with use of a text, image, or a video prompt. It allows for video creation of over 60 seconds in different visual styles. When we say high-quality, we mean 1080p clips, just to be accurate. Here’s an example.
Gemini-boosted Google Search
Google has also announced boost for Google Search, thanks to Gemini AI, of course. Google will soon bring “multi-step reasoning capabilities” to Google Search. It will essentially be able to break down your questions into parts and figure out what problems to solve. Google says that this could enable you to save a lot of time on research. The company also announced that you’ll soon be able to ask questions with video, right in Google Search.
Improved communication between Workspace apps
Google has announced that it’s boosting communication between Workspace apps, apps like Gmail, Docs, and Calendar. Gemini for Workspace will make things even better, as it will be, for example, able to recognize a receipt in Gmail, and allow you to organize it in Drive and Sheets. That’s just one example of a quick jump to another Workspace app. Gemini will also be able to analyze your expenses in Sheets, for example, and so on. There are a lot of possibilities.
Gemini ‘Gems’, a way to get customized AI
Google will offer you the ability to customize Gemini AI, basically. This feature is coming “in the coming months”, and it will enable you to interact with Gemini in a specific way. Google is calling this ‘Gems’. You’ll be able to customize Gemini with personalities and abilities. For example, you’ll be able to make it to be an upbeat gym buddy if you ask it to, or a deep-voiced friend. Those are just some examples.
Android is getting a full dose of Gemini
As you’ve noticed thus far, Gemini AI is the name of the game. Android is getting its fair share. Google will start injecting Gemini straight into Android with Android 15. That will help Android become “more helpful and context-aware”. It will enable apps to be more useful as well. Google also said that later this year, you’ll be able to drag-and-drop generated images into Google Messages and Gmail, and also ask questions about specific YouTube videos and PDFs.
Gemini Nano & Android
Google says that, thanks to Gemini Nano, Android will be able to warn you in the middle of a call if it ends up spotting suspicious activity. Furthermore, Android’s accessibility feature, TalkBack, will also get improved thanks to Gemini Nano. How? Well, you’ll get “clearer and richer” image descriptions.
SynthID for video
Some of you may recall SynthID which Google announced last year. That feature basically adds imperceptible watermarks to AI-generated images and audio. The same will be happening with video content too, as Google is expanding the functionality of SynthID. This will include Google’s ‘Veo’ model, of course.
LearnLM, “fine-tuned for learning”
Google announced ‘LearnLM’, a family of models based on Gemini. The company says that these models are “fine-tuned for learning”, as LearnLM “applies educational research” in order to make Google’s products like Search and YouTube more personal, active, and engaging. Google also mentioned that the new Learning coach Gem is using LearnLM in order to provide ‘step-by-step study guidance’. This feature is going to become available “in the coming months”.
