“Honestly, I would click Reddit.”
That’s what one participant in our AI Mode study said when searching for the best credit card loyalty programs. She believed getting someone’s personal, unbiased experience would be more helpful than what you might learn from someone getting paid to promote a particular card.
For someone working in the finance sector, it can be panic-inducing to hear about the constant changes in search with the growing use of LLMs and the release of AI Mode and AI Overviews. How will you continue to reach potential customers? How can you ensure the right people find your company?
We worked with Farrah Bostic of The Difference Engine to learn the search habits of 100 participants, and Farrah spoke one-on-one with 23 users specifically about AI Mode and watched them perform searches in real time.
Participants ranged in age from 18 to 75 with a balanced mix of ethnic backgrounds, genders, education levels, and locations across the country.
Each user walked through five different queries:
- Find out which credit cards offer the best loyalty programs.
- Find out which kinds of breakfast cereals are healthiest for children.
- Find out the top news or sports headlines in your local area.
- Find a health clinic in your area.
- Find a pair of shoes to wear indoors for under $75.
Most of the people who participated in the study had their own preferred ways of searching and a particular search channel to which they remain devoted, even with the introduction of AI Mode. But after using it, many were enthusiastic about what AI Mode offered.
We’ll eventually go through the results of all five of the tasks in greater detail. Right now, we’re going to do a deep dive into task #1: how participants researched credit card loyalty programs. The patterns we see emerging can impact how prospective customers will find certain organizations in the finance industry in AI Mode in the future.
Overall Data from the AI Mode Study
Here are some general findings from this particular task in our study:
- 52% were shown AI Mode as an option in the unmoderated task; in the live interviews, almost all had AI Mode visible (either from the initial google.com search bar, or once they got into search as a tab, or as the dive deeper button from AI Overviews)
- 5% used AI Mode
- 33 seconds was the average dwell time in AI Mode
- 38% engaged with AI Overviews
- 55.3% of those who engaged with AI Overviews unfurled them
- 14.3% of those who unfurled AI Overviews also clicked on the links within it
- Average number of links clicked: 1.09
- Average time to complete the task: 2 minutes and 7 seconds
- Average score for ease: 4.27 out of 5
- Average score for satisfaction: 3.85 out of 5
Typical search terms participants used in this study include:
- 14.6% framed searches as questions (“What,” “Which”)
- 13% began with “best”
- 6.25% included “2025”
- 5% asked for direct comparisons (“compare credit card loyalty programs”)
- 3.5% refined with “points”
- 20% used follow-ups, often naming a card, adding “best,” “no fee,” or “for travel”
When asked how often participants use AI-based services, more than half said daily, and only two users stated they rarely or didn’t use AI at all. The rest said weekly, so it’s clear that AI has worked its way into many people’s routines.
Using AI Mode to Research Credit Cards
In the one-on-one interviews, participants could choose whether to start with AI Mode on their own, and 6 out of 23 did. Those who did not choose to start with AI Mode were subsequently prompted to use it to compare search results.
Here are some of the positive reactions to AI Mode:
- Overall, users seemed to appreciate AI Mode’s ability to summarize complex financial information and present options in a more digestible format, often including pros and cons or key features.
- Participants valued seeing annual fees, cost per year, pros and cons, and categories for earning points laid out directly in the AI Mode results
- They especially liked receiving side-by-side comparisons or having categories broken down (e.g., travel rewards, cash back).
- Some also remarked that they liked how AI Mode tried to break down “how to think about” the category, and that it depends on the customer’s goals/behavior, helping them figure out what kind of loyalty program would be best for them.
- The ability to ask follow-up questions within AI mode (like “side-by-side comparison” or refining criteria) was highly valued by those who discovered it, though it was not consistently available or functional across all users.
Due to the fact that side-by-side comparisons with apples-to-apples categories are what people expect to get when shopping for a credit card, this style was expected from personal finance websites.
As a result, many preferred visiting traditional websites (NerdWallet, Forbes, Motley Fool) for credit card comparisons due to perceived greater accuracy, timeliness, or trust in established financial sources.
There was also some confusion or sense of redundancy when some users first saw AI Overviews and then clicked “dive deeper” with AI Mode. The AI Mode result often does not seem different or more comprehensive than the AI Overview.
AI Mode User Search Patterns and Reactions
When one user was prompted to research credit cards with the best loyalty programs, she immediately said, “That’s an AI question,” and went right into AI Mode because she believed it worked better.
“I think AI has a way of analyzing the information that I’m looking for to give me the answers,” she said.
Another participant who typically uses ChatGPT also had positive feedback about AI Mode.
“I think it does the same thing, if not better, than ChatGPT because it provides the links and the visuals,” she said.
Search Channel Preferences
Many of the study participants had preferences for which search channel on which to start their search. They varied greatly from LLMs to social media.
One user started the search on TikTok, as she liked seeing real-life comparisons and hearing people’s opinions in real time
Another used Grok. He liked the table comparison format that Grok provided, as AI Mode was unable to create one at the time of use (though the table function worked for other users):
Another user said his go-to search function is Gemini. He found the information offered by AI Mode to be too general, but said he would consider asking a follow-up question.
His follow-up query specified “without an annual fee with good bonus categories for cash back.”
“When it comes to research it can really help me kind of dial in on certain things,” he said about AI Mode. “If it’s just browsing every product out there, I might have an easier time using Walmart or Amazon or a specific retailer.”
A participant liked the summaries provided by AI Mode, but she is subscribed to a travel site called going.com and preferred to explore articles on there to compare credit cards. She added that for prompts requiring more “critical thinking”, she would not use AI Mode.
One person who probably wouldn’t have gone to AI Mode initially received a pop-up prompting him to use it, so he did and was pleased with the result. He thought that this type of investigation and comparison would usually take a while “but this is kind of cool that it’s doing it for me.”
He entered a follow-up prompt with “which credit card offers the best loyalty value” and received a shorter list of two credit cards, then asked it to compare the loyalty perks of the two it picked. However, he admitted that he would want to “check its work” a few times and click the cited links manually to make sure everything AI Mode reports is correct.
“Sometimes it’ll scoop information out of context and misinterpret it,” he said.
An interesting method came from the user who said she will simply start typing and then look at what auto populates in the search bar and choose the most relevant query:
She also asked AI Mode to create a side-by-side comparison table and this time it worked:
Another user was also able to get the table function to work in AI Mode, however his first choice for finding credit card loyalty programs would be to ask what bank the people he knows would recommend.
When one user admitted she would typically pose this type of question to ChatGPT, she decided to narrow down the search on AI Mode and asked for the top 3 travel reward credit cards:
Some of the AI Mode results specified what each credit card company reward was best for. One of the users had a positive reaction to this distinction and said it would be helpful in making a decision.
Another participant in the study highlighted the fact that while doing a traditional Google search, he would just type in keywords, but he often talks to LLMs in full, complete sentences with question marks.
“Sort of like another person,” he said.
He then chose to ask AI Mode for links for each credit card so he could go down the list clicking each one to do additional research and comparisons on his own:
Negative Search Reactions
When shown a number of sponsored posts, some users refused to click them, typically because they did not trust them. One user was worried that sponsored links “might send you somewhere else”.
Another claimed that whenever she sees sponsored posts, she never clicks them because she believes the ads will take her information and sell it.
“Anything that says sponsored, I just scroll down until I can find my answer,” she said.
Although she liked the information provided by AI Mode and found it helpful, she believed the results were too similar to AI Overviews:
“It said ‘dive deeper’ but to me it looks like the same thing.”
While one user said he would end up going to Forbes for a query like this, he felt that AI Mode summarized it well.
“It’s very ‘matter of fact’,” he said, with easy-to-read, brief summaries. However, if follow-up queries or deep dives were required, he would still go back to a traditional Google search to refine the query.
AI Mode Considerations for the Financial Industry
The findings from our AI Mode study showed a variety of ways consumers research credit card loyalty programs and how they interact with AI Mode. While users appreciated AI Mode’s ability to synthesize complex financial information into digestible comparisons (particularly side-by-side analyses and clear breakdowns of fees and rewards), some maintained a healthy skepticism about AI-generated results.
The persistent preference for user-generated sources like Reddit, established financial sites, and the need to “check AI’s work” underscores that authenticity and verification remain critical in financial decision-making.
For financial institutions, this presents both an opportunity and an imperative. Search behavior is changing, but consumer needs for trust, accuracy, and personalized guidance remain constant. As AI-driven search becomes more prevalent, organizations must ensure they’re providing clear, authoritative content that AI can reference while maintaining transparent, accessible ways for consumers to verify that information.
Though many users did not consider AI Mode as their top choice for research, it may eventually become the default search channel. For one study participant who has been using Google for almost 30 years, he said force of habit might continue to control his search choices but admitted he would use AI Mode if it’s right in front of him.
But for now, he says: “I’m just not used to that yet.”
