Google Releases AI AGENT BUILDER! 🤖 Worth The Wait?

Hey Matt. When you select "function", you're responsible for writing the code and driving the conversation via API calls. The flow goes a little like this. You submit a message and you check the status and it tells you you need to run the function with these inputs. YOU then submit the output back to vertex via the API, where it uses the function output to generate the next message. You can implement the function wherever you're DRIVING the convo. Makes sense? This is a very similar flow to the assistants API which I have a video about on my channel if you're interested

@19:25 you set your Tool Type to OpenAPI, and the endpoints in the instructions will work. I imagine it's 1 function per tool for modularity. It's more or less exactly the same setup as the Custom GPTs configuration.

I've been eagerly awaiting a local AI solution to sort through our family's extensive collection of decades-old photos. This AI should be capable of identifying and eliminating duplicate or blurry images, learning to recognize faces, and categorizing photos by individual. Moreover, it should have the ability to estimate the age range of each person depicted. Ideally, this solution would allow for easy exportation of organized data so that Stable Diffusion models could learn from it. Additionally, integrating facial recognition capabilities into home security cameras would streamline access control, eliminating the need for physical keys.

You’re so right Matt on that agents in the Mercedes. There’s this weird movement of bells and whistles, and what we can do rather than true functionality that is efficient. Especially when it’s not level five self driving, we still have to drive and less distraction equals more attention to the road and thus is safer driver.

Considering Google ought to be a solid year or so behind OpenAI, despite their best efforts, what kind of monster is OpenAI planning to release that beats all this?

I'll be ready for Google to put that kind of tech to use on youtube. No more trap videos where the title, description, and thumbnail don't match the content. Youtube could auto-down-vote videos that don't match their title, description, and thumbnail. I can imagine that Google will be the king of multi-modality due to YouTube.

Matt on it! Thanks for going over this. Nobody else is really going into it like you.!👍👏

@26:10 they're using the equivalent of Agent (preview) app in the Agent Builder service which as they word it "can answer questions from data, connect with business systems through tools"

thanks for the review . I was short on time today , perfect timing

Hey Matt, with all of these models and apps along with their updates, it would be nice to see a real time list of your favorites. For example you said Gemma wasn't to impressive but it may be now with its new update. Maybe like a place where you can rank the best models or frameworks and update the source as things change. For example crew Ai, autogen, Gemma,, Devin, Devin alternatives, and all the different llms. It be nice to see an up to date list of your rankings in each category. Or does something like this already exist?

Agent > Example > create example The example is a dummy dialogue. You need to actually mockup a weather convo. Each prompt/return is an ‘action’ in the example convo. There are other actions, like tool. Tool is where you put the code

the Mercedes missed opportunity is probably an insurance issue. They'd need to long term test of it's safe because they don't want to get sued if someone expects the ai to read their mind and then goes road rage and blames it on them

Some of Matt’s videos were a bit lightweight in the past, but he really nailed this one. 👍 good analysis, brought in external knowledge and not just AI fanboy content. Nice job. Will be subscribing to this channel now!

i heard the keynote before, and I really appreciate your insights and commentary to make it more "real." There is often a strong corporate biase when we hear these kinds of releases. Great video very informative. Love your stuff

It's like Autogen with frosting on top. I can dig it saving me a lot of design setup time for agent clouds.

10:05 GM is already there. Every modern GM vehicle starting from the Cadillac Lyric, 2024 Acadia & Traverse, 2025 Enclave, all have the Google agent built in. As the agent improves, the software will be updated over the air. GM is even phasing out Apple CarPlay and Android Auto, because the Google voice agent will fulfill all the requirements, including infotainment and HVAC control, vehicle information query, mapping, POI search and navigation, traffic updates, etc. Eventually all GM models will get the Google interface.

#Using ai to select ai, #Using ai to debug code and create insertion points (~cross cutting)....it could then analyze and respond by altering the code to fit conditions (~FPGLA). I'm working on knitting Neo4J and ai-models for an interactive teal time graph dashwindow to use user priveldged based access. It seems that ai could refine alternative models based upon a user's project and use rag to learn preferences...and present them ranked with pros and cons of each.

3:00 that’s great , I hope the model works well though

Great announcement, thanks for sharing! I've been working with several of these offerings over the past few months, developing some great use cases. I was thinking of perhaps showcasing some of them on my channel soon.

Still watching but as always love the content and the way you deliver it, so glad I stuck around as you are my go to to keep up with whats going on