[Webinar Replay] What Happens When Real People Try AI Mode? Reviewing UX Data with Farrah Bostic

by Francine Monahan

08.22.2025

When Real People use AI Mode

It’s been about two months since AI Mode officially launched. Sundar PIchai even declared it to be the future of search. So, who’s using it? How often? For which queries? And how is the user experience?

We wanted to know what the early user behavior of AI Mode looked like. We talked to 100 American users and gave them five tasks to perform in a search that gave them access to AI Mode. In this webinar, Farrah Bostic of The Difference Engine brought the data and discussed it with iPullRank’s own Mike King and Garrett Sussman. 

Some of it was surprising. Other parts were not so surprising. Overall, the answer seemed clear: people just weren’t using AI Mode much. But why?

Let’s look at the data and see how the user experience for AI Mode is shaping up.

AI Mode User Behavior Study

Last month, we received data from Similarweb that said over 50% of users tried AI Mode once and then bounced. That was pretty telling. 

AI Mode retention

“It really feels like users are still experimenting and haven’t formed consistent habits yet around which tools they use for what,” Farrah said. 

Users are treating AI assistants as research companions instead of search replacements, and only a small minority rely on AI for all queries. 

“Even if people are using AI Mode, either they’re getting exactly what they need when they’re searching for it, or they just think it’s garbage compared to the other ways that they can search,” Garrett Sussman said. 

Parameters of the AI Mode Study

In this study, 100 people participated in unmoderated interviews, 23 of which had live interviews with Farrah, and were asked to perform five different search exercises. Participants were asked about their use of voice assistants, AI tools, and AI Mode.  

We also wanted to determine how the availability of AI search tools impact people’s search results, their satisfaction with those results, and their preference for AI tools over other channels.

We recruited a nationally representative sample of users from June 24th to 30th, 2025, to undertake tasks that encapsulated a range of search intents:

  • Informational intent 
  • Local intent
  • Commercial intent
Sample demographics

With this study, we wanted to understand how the introduction of AI Mode in Google search changes user behavior depending on the context of different search types.

“One of the key things with these types of qualitative studies is really getting a sense of the narrative and the stories that are told.”

Sample Search Tasks

The five queries we asked participants to look up were:

  1. Best credit card loyalty program
  2. Healthiest breakfast cereals
  3. Top local sports and news headlines
  4. A health clinic in your area
  5. A pair of shoes to wear indoors for less than $75
AI Mode study parameters

How did our participants handle the tasks? We’re going to look at each task and the results. 

1. Best Credit Card Loyalty Programs

  • 52% were shown AI Mode as an option in the unmoderated task; in the live interviews, almost all had AI Mode visible.
  • Only 5% used AI Mode.
  • 38% engaged with AIOs and 55.3% of those unfurled the AIOs.
  • 14% framed the search as a question.
  • 13% began the search with “best”.
  • 6% included the year in their query. 
  • Others refined their search with follow-ups like “no fee” or “for travel”.
Results of Task 1

Although 20% of users did not click on any links in the AI Overviews, participants clicked on articles from familiar websites like Forbes and NerdWallet. Some even clicked on sponsored links from popular banks like Chase and Bank of America. 

Most of the participants were looking for particular words or phrases in their search, like “0% interest” or “no fee”. In general, users liked how everything was laid out in AI Mode but they wanted to easily see the pros and cons of each card, which wasn’t present.

Many said they would just go straight to NerdWallet or other sites for this information instead. They were not going to go through every single credit card available in the long list AI Mode has laid out: 

Credit card AI Mode results

The overlap with AI Overviews also caused some confusion with the users. AI Mode can often feel redundant (and AIOs can even be present within AI Mode). 

“AI Mode is not quite as highly adopted yet as some of the other tools they have at their disposal.”

2. Healthiest Breakfast Cereals

  • 33% were shown AI Mode as an option for this search.
  • 2% used AI Mode in the unmoderated task; no one in the live interviews used AI mode unprompted for this task.
  • 47% engaged with AI Overviews and 76.6% also unfurled them.
  • 20% of those who engaged with AI Overviews clicked on links.
  • 9% phrased their search as a question. 
  • Users also refined their search with specific keywords (such as “best” or “good”). 
  • One participant even used “kid friendly” while others used specific qualities like “with protein” or “organic”.
Results of Task 2

When it comes to clicks, over half of the users clicked on articles, blog posts, or review sites, about a quarter did not click any links, 5% clicked on e-commerce sites to look at prices, ingredients and nutrition labels, and 3% clicked on social media/forum links, especially Reddit.

Participants also clicked on “People Also Ask” and shopping-related widgets like “In Stores Nearby” and “Sponsored” shopping links. Some turned to ChatGPT or Gemini to get more info, but many said that realistically they would just go to a grocery store for food items rather than purchase them online.

Cereal AI Mode

However, when two participants clicked on Cheerios, it created a new Google search, which they didn’t like. 

“Users hate that Google links to other results in Google and hopefully they will stop doing it.”

3. Top Local Sports and News Headlines

  • 38.24% were shown AI Mode as an option.
  • 2% used AI Mode.
  • 18.6% engaged with AI Overviews and 75% unfurled them.
  • 14% of those who unfurled AI Overviews clicked on links.
Results of task 3

In this task, search terms were more varied and often focused on a particular sports theme. Most participants, based on the prompt, included terms that would ensure locality of the results like “near”, or a specific city or county name as part of the search string.

Some also used words that would yield current results like “today” or “breaking”, as well as specified geographic locations, like adding town/city names or zip codes. 

This type of search was more challenging because it was so localized. Most of the users said they get local news in their Facebook feed, the radio, hearing from friends, or other social feeds like Reddit. They also tended to go straight to local news websites. 

In this search, many found the AIOs and AI Mode (as well as ChatGPT) to be inaccurate, less trustworthy, and not up to date, but the participants didn’t expect these sources to be timely or accurate in the first place, which is an issue in itself. 

News AI Mode

4. A Health Clinic in Your Area

  • 29% were shown AI Mode as an option.
  • 2% of those who were shown AI mode used AI Mode.
  • 3% engaged with AI Overviews but <1% unfurled them.
  • Users tended to use words and phrases like “local” or “near me”, as well as using zip codes and city names to narrow the search.
  • Some chose to distinguish between different types of “health clinics” such as “urgent care”, “general practitioner”, “health care providers”, “primary care”.
  • Some had specific use cases in mind, using terms like “Find nearest ENT doctor’s office”, or “women’s health clinic near me”.
  • And some had specific brands of health clinics in mind like City MD, Care Now, or Minute Clinic.
Results of task 4

Hardly anyone used AI Mode or AIOs and barely unfurled the overviews. In this location-based case, users wanted to see a map at the end of this search as there is a higher level of trust in Google Maps results. They also wanted to see hours, reviews, and booking information. 

AI Mode results lacked this specificity and buried the maps on the page, causing frustration in users:

Clinic AI Mode

5. Indoor Shoes for Less Than $75

  • 26.5% were shown AI Mode as an option.
  • 2% of those who saw AI Mode used AI Mode.
  • <1% engaged with AI Overviews and also unfurled them.
  • Participants added some context, typically around gender (men, women), and around usage context (house).
  • Many also added their shoe size to narrow down the results, as they anticipated a shopping page experience with retailers attached to products.
  • Others also sought reviews and would add the word “best” to their search.
Results of task 5

Some users were inspired by the prompt to search for something specific, e.g., “Juicy Couture house slippers”, “Adidas slides”, “Nike shoes”, “New Balance 847 women’s size 12 wide”, and “traditional Crocs under $75 from Walmart”. 

Overall, users preferred visual browsing with product images and filters (size, price, type). Also, most knew exactly what they wanted, reducing the need for AI recommendations. Many even skipped the search entirely, going straight to Amazon, Walmart, or their favorite brands.

AI Mode often showed lists of stores/brands instead of products, which disappointed users. This lack of direct purchase links made AI Mode feel the “least informative.”

Shoes AI Mode

Participants in this task rarely used AI Mode or AIOs. They wanted to go into a shopping browsing experience or straight to a brand’s website as quickly as possible. 

What We Learned About AI Mode

There are some major takeaways from this study:

  • Search happens in a lot of places: Google, Reddit, ChatGPT, YouTube, etc. Wherever there is a search bar, people will search. 
  • Not all searches require a lengthy response. 
  • In location-based and shopping-based searches, there is a desire for a visual experience or a map. If they don’t see one or a carousel of products they can click, they will leave. 
  • When you’re comparing products, many users already have go-to sites for comparisons. 
  • AI Overviews and AI Mode have a branding problem. Many users didn’t know AIOs were different from AI Mode (as an overview could be within AI Mode). Also, AI Mode doesn’t stand out yet and many users didn’t even notice it. 
  • AI Mode’s “Dive Deeper” button was seen as an irritant and no one clicked on it. 
  • Many users preferred ChatGPT for broader results. 
  • Traditional search is good for quick answers, visual browsing, and location-specific searches.
  • Voice assistants are popular with many users. 

Overall, AI Mode isn’t quite there yet. There was early interest when it first launched, but that enthusiasm hasn’t been sustained over time. For local or shopping searches, it doesn’t provide the best user experience, but it still seems to be a decent choice for informational searches. We’ll see if it evolves to a more user-friendly interface in the future. 

As Farrah said: “Context and relevance is so important to understanding what tools people are going to use and how satisfying those tools are going to be.” 

Want to learn more about AI Mode?
//.about_the_author

Francine Monahan

Francine is a Content Marketing Manager at iPullRank with 10 years of digital marketing experience and nearly 20 years of professional writing experience. As a former journalist, she is constantly asking questions, seeking out and following the latest SEO news trends, and trying to learn everything about everything.

//.recent_content

As AI rewrites the rules,

read between the lines.

AI is reshaping search. The Rank Report gives you signal through the noise, so your brand doesn’t just keep up, it leads.