I attempted 8 of Google’s latest AI merchandise and updates at I/O 2024

The improved lengthy context window may even pull data from a number of paperwork when responding to a single immediate. Within the facet panel in Docs, I requested for assist writing a pattern letter to a possible job candidate — within the immediate I linked to the job description doc and the applicant’s PDF portfolio, each of which have been in my Drive — and immediately acquired a electronic mail draft, which factored in related particulars from each paperwork.

Gemini 1.5 Professional isn’t our solely shiny new mannequin, although: I additionally bought to strive the freshly-announced Imagen 3, our highest-quality text-to-image mannequin but. One of many new skills I used to be enthusiastic about was its potential to generate ornamental textual content and letters, so I put it via its paces. I began by asking for a stylized alphabet — like letters spelled out in jam on toast, or with silver balloons floating within the sky. Imagen 3 generated a full alphabet of letters, which I might then use to kind out my very own (scrumptious) menus.

After my Imagen 3 interlude, I continued with extra Gemini demos. In certainly one of them, I might pull up Gemini’s overlay on an Android cellphone and ask questions on something on the display screen. This actually confirmed how we’re not solely increasing what you may ask Gemini, however we’re additionally making Gemini context conscious, so it could actually anticipate your wants and supply useful recommendations.

The use case right here was a prolonged oven guide. Whether or not it is a demo or actual life, that is not one thing I would be enthusiastic about studying. As an alternative of skimming via the doc, I pulled up Gemini and instantly bought an “Ask this PDF” suggestion. I examined questions like “how do I replace the clock” and shortly bought correct solutions. It labored simply as nicely with YouTube movies. As an alternative of watching a 20-minute exercise video, I requested a fast query about tips on how to modify planks, bought a solution, and was on my manner onto the following demo, the place I examined a brand new dialog mode known as Gemini Live that permits you to discuss with Gemini within the app, no typing required.

Talking with Gemini was a special expertise than the standard chatbot interface: Gemini’s solutions are much more conversational than the paragraphs of texts and bullet-pointed lists you would possibly normally get. In my demo, I realized you may even lower off Gemini in the course of a solution. After asking for a listing of child’s actions for a summer time trip, I used to be in a position to interrupt a listing of recommendations to dive in deeper on what supplies I’d want for tie-dying a shirt.

The Project Astra — or “superior seeing and speaking responsive agent” — demo took issues a step additional to indicate the reducing fringe of the place our conversational AI tasks are heading.