Gemini creeping towards total control of your phone – you might never need to open an Android app again

Gemini looks to be taking a big step towards agentic actions soon

Gemini app icon on Pixel phone
(Image credit: Future / Chris Hall)
Quick Summary

Gemini could soon order food deliveries or call an Uber for you, with code in a beta app pointing to new abilities.

Using a screen automation system, it's thought that this is the next step towards an agentic AI system on your phone.

Gemini might soon be able to help with a range of tasks thanks to a system called "screen automation". Code referring to these action has been found in the Google app, leading to more understanding of the company's plans for the AI system on Android.

Google has always talked about the agentic powers that it wants AI to offer – that is the ability to do things on your behalf. It sees a future where you'll ask Gemini to create an itinerary for a weekend away and once you've approved the suggestions, you'll get the AI to book hotels and make reservations at restaurants.

It's these latter actions that are seen as "agentic", ie., the AI is acting on your behalf, beyond just surfacing information for you.

According to 9to5Google, there are strings of code in the Google app 17.4 beta that refer to "Get tasks done with Gemini". It appears to be codenamed "bonobo", with Google explaining that it can help with "placing orders or booking rides, using screen automation on certain apps".

Using screen automation suggests that this isn't going to happen in the background, it's likely that it's all going to happen in front of your eyes, meaning you can intervene if things aren't heading in the right way.

How could an agentic Gemini help you?

While there's no working example, it's easy to imagine that asking Gemini to book you an Uber home would open the Uber app, fire up the destination, select a ride and tell you when it's arriving.

Equally, asking Gemini to order your usual from a delivery service should be fairly straightforward, as long as Gemini knows your preferences.

The source identifies a couple of additional pieces of information lurking in the app, including the declaration that "Gemini can make mistakes" and that "you're responsible for what it does on your behalf".

This latter phrase is a great disclaimer, so that if Gemini orders a stack of pizzas, you can't then turn around and say that you didn't want them and the AI is to blame. I think we all know that's going to happen a lot in this agentic future.

It's said that you'll be able to intervene and stop the AI at any point, but that does sort of beg the question of why you'd automate this process in the first place? Is it so arduous to open an app (which Gemini will already do for you) and make a few taps?

It's said that "screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on" which will raise privacy questions with some users, while it also says "don’t enter login or payment information into Gemini chats", suggesting that it can't be trusted with sensitive information.

Like all things AI, the big question comes with whether this is a good thing or not. We know that AI is gaining skills all the time, with the ultimate aim to carrying out tasks on your behalf in the future.

But if it can't be trusted to get it right and there's a chance that personal information might not be secure, is this ever going to get off the ground?

Chris Hall

Chris has been writing about consumer tech for over 15 years. Formerly the Editor-in-Chief of Pocket-lint, he's covered just about every product launched, witnessed the birth of Android, the evolution of 5G, and the drive towards electric cars. You name it and Chris has written about it, driven it or reviewed it. Now working as a freelance technology expert, Chris' experience sees him covering all aspects of smartphones, smart homes and anything else connected. Chris has been published in titles as diverse as Computer Active and Autocar, and regularly appears on BBC News, BBC Radio, Sky, Monocle and Times Radio. He was once even on The Apprentice... but we don't talk about that. 

You must confirm your public display name before commenting

Please logout and then login again, you will then be prompted to enter your display name.