Three “wow’s” in a row were the reaction of one participant in our AI Mode study who was using AI Mode for the very first time.
“It’s like a search within a search engine,” she said.
She enjoyed asking follow-up questions to her query for the healthiest breakfast cereals for children, like which of them included strawberries. Though unsure at first how AI Mode would contribute more than was already provided by AI Overviews, she’s officially been converted and will be using it more in the future.
“I am pleasantly surprised,” she said.
We saw many interesting reactions to AI Mode in our study with Farrah Bostic of The Difference Engine to learn the search habits of average people. She surveyed 100 participants and spoke one-on-one with 23 users specifically about AI Mode and watched them perform searches in real time.
Participants ranged in age from 18 to 75 with a balanced mix of ethnic backgrounds, genders, education levels, and locations across the country.
Each user performed five different queries:
- Find out which credit cards offer the best loyalty programs.
- Find out which kinds of breakfast cereals are healthiest for children.
- Find out the top news or sports headlines in your local area.
- Find a health clinic in your area.
- Find a pair of shoes to wear indoors for under $75.
Today we’re looking at the second search task: finding the healthiest cereals for children. For those in the ecommerce industry, the patterns emerging may give you a taste of what the future of search will look like for your prospective customers.
Overall Data from the AI Mode Study
These are the general findings from task #2 in our survey:
- 33% were shown AI Mode as an option for this search
- 2% used AI Mode in the unmoderated task; no one in the live interviews used AI mode unprompted for this task.
- 33 seconds was the average dwell time in AI Mode
- 47% engaged with AI Overviews
- 76.6% of those who engaged with AI Overviews unfurled them
- 20% of those who engaged with AI Overviews clicked on links within it
- Average number of links clicked: 1.96
- Average time to complete the task: 2 minutes, 7 seconds
- Average score for ease: 4.05
- Average score for satisfaction: 4.05
Here are some typical search terms participants used in this study:
- 9% of participants phrased their search as a question
- Users also refined their search terms with more specific keywords like “best” or “good” (as opposed to healthiest /healthy /most healthy), or “kid-friendly”
- Some also specified qualities like “with protein” or “organic”
Overall, AI Mode was considered useful for summarizing nutritional guidelines and recommending brands for healthy cereals. Users also appreciated being offered a way of thinking about what constitutes a healthy breakfast cereal for kids, noting characteristics like fiber, low/no sugar, and no food dyes.
Users consistently noted that for groceries like cereal, they would typically shop in person at a grocery store rather than purchasing online through search results. However, participants valued AI’s ability to synthesize information from multiple sources (blogs, articles) to provide a comprehensive answer, saving them time.
Three participants in the live interviews felt the most likely scenario for them in a search like this would be to compare options in-person at a grocery store rather than searching online. And almost all participants said their most likely next step would be to purchase cereal in-store rather than online.
Using AI Mode to Research Breakfast Cereals
One participant in the study who typically uses ChatGPT decided to use AI Mode first to search for the healthiest breakfast cereals for children. She believed that AI opens up a wider and broader search, and is more reliable.
“I know it’s going to scan the research to give me the information that I’m looking for versus if I just would have Googled it,” she said.
However, she was disappointed because she had to scroll pretty far down in AI Mode to get to the actual cereal brand names. And even then, it only offered her five results, which she thought wasn’t enough:
Another user agreed and also didn’t like having to scroll far down to reach the actual products. He would’ve preferred the order to be reversed. He did like the information it provided, though, saying “I could engage with this more.”
One user liked the practical tips for choosing healthy cereal, saying she found them helpful.
“This is more interesting to me,” she said.
However, other users saw branded products show up first in their search results, showing the unpredictable nature of AI Mode and how different everyone’s results can be. One user said she was pleased with the number of results suggested, even though there were still only five:
Another participant found it interesting that the cereal links on AI Mode took you to another Google page that had product and shopping information:
One participant wondered if the AI results were less “SEO-influenced” than traditional search results. He believes that you’re often seeing a biased version of the results you want on Google, so he thinks optimization might be impacting the cereals that appear in the results.
“There’s obviously something on the backend that Cheerios is doing to make them pop up first, even if what they’re saying is true,” he said.
AI Overviews vs. AI Mode
Much like the previous task in this study, some users expressed confusion about the difference between AI Overviews and AI Mode results, but many seemed to agree that they found AIOs helpful.
“Whenever I see this AI Overview pop in, I generally look at it,” one user said. And if he has additional questions he will click “dive deeper” and ask AI Mode follow-up questions.
Another user said there was some overlap between AI Overviews and AI Mode, but AI Mode is more detailed. It even divided the cereal options into low sugar and high fiber, and gave more advice on what to pick:
When AI Overviews show up, another user said she always looks at them first. Instead of just getting a list of links to webpages, she now gets explanations and descriptions.
She also likes using AI Mode on her phone through voice commands because it’s “like asking a really smart friend.”
When she searched for cereal on AI Mode, however, she thought some of the articles on the right were a bit too old to trust, but said she would probably click on the most recent one:
Another user agreed that AI helped take some of the work out of searching. Instead of querying and then having to read a bunch of top 10 listicles and making comparisons himself, AI Mode could handle that for him.
However, he said he wouldn’t use AI Mode for quick facts, like researching the population of a country or the age of a celebrity, because Google still has that perfected.
User Search Preferences
A number of participants said they would typically skip this type of query in favor of an in-person visit to the grocery store to compare ingredients. Or they would just do minimal research first before going to the store.
“I would probably just research this and then make a mental note for the next time I’m at the grocery store,” one user said.
Some would even go to their local grocery store’s website or Instacart instead. One user said he goes to Amazon and reads reviews, focusing on the negative reviews because he feels those are the most honest.
One user shared that he would go to Reddit and visit the parenting subreddit and search there.
“I’d probably come to actual parents for that, not necessarily the robot,” he said.
Another interesting search habit was the user who has had his Google set to Advanced Search for years simply because he likes the option of adding words and phrases for filtering.
“It’s just second nature to me,” he said.
AI Mode Considerations for the Ecommerce Industry
There are so many ways that people shop online these days. Some perform deep research before making a purchase, while others just head over to Amazon and click whatever has good reviews. And with the addition of AI Mode, shoppers have a new way to find products, even if the information displayed doesn’t always feel natural or follow the format of how people shop.
Ecommerce teams should consider the growing importance of comprehensive product information that can be easily parsed by AI systems, as users increasingly rely on AI to filter through multiple sources and compare products. However, the study also highlights that despite AI’s ability to provide detailed product insights, the purchase journey for many food products still culminates offline.
While AI Mode shows promise, the reality is that most users still prefer the experience of in-store shopping for everyday items like groceries. Will that change as people become more familiar with these AI Search platforms? It’s difficult to determine. But only a certain type of person would jump through these research hoops to find a healthy breakfast cereal.
The unpredictable nature of AI Mode results also shows the importance of strong SEO fundamentals alongside emerging AI optimization strategies, especially as users begin using AI Mode more often.
For the one participant using AI Mode for the first time in this study, even though using Google is still a habit, she may start using AI Mode from now on.
“The AI Mode tab is definitely intriguing me after today a little bit more,” she said.
