“It didn’t occur to me to use AI Mode.”
That’s what a 74-year-old Orlando man said when prompted to use it to research local news in his area instead of his usual sources of Fox News, ABC News or the Drudge Report (yes, it’s still around and it looks exactly the same).
Nearly all of the participants in this search did not initially consider using AI Mode until they were prompted to because they all had their own favorite sources for local news and sports. Local news stations, ESPN, social media, and Reddit topped the list of preferred news sources.
They were also concerned that AI Mode was not accurate or timely with its information, so there was some skepticism about the credibility of the summaries.
There were many interesting reactions to AI Mode in our study with Farrah Bostic of The Difference Engine to learn the search habits of average people. She surveyed 100 participants, spoke one-on-one with 23 users, and watched them perform searches in real time.
Participants ranged in age from 18 to 75 with a balanced mix of ethnic backgrounds, genders, education levels, and locations across the country.
Each user performed five different queries:
- Find out which credit cards offer the best loyalty programs.
- Find out which kinds of breakfast cereals are healthiest for children.
- Find out the top news or sports headlines in your local area.
- Find a health clinic in your area.
- Find a pair of shoes to wear indoors for under $75.
Today, we’re looking at the third search task we asked participants to perform: researching local news and sports headlines. We’ll see if any interesting search habits pop up so those in the publishing industry can get a better grasp of the future of search for news.
Overall Data from the AI Mode Study
Here are the overall findings from task #3 from our AI Mode study:
- 38.24% were shown AI Mode as an option
- 2% used AI Mode, but none of them used it unprompted
- 18.6% engaged with AI Overviews and 75% unfurled them
- 14% of those who unfurled AI Overviews clicked on the links
- Average number of links clicked: 1.09
- Average time to complete the task: 56 seconds
- Average score for ease: 4.1
- Average score for satisfaction: 4.39
These are the typical search terms the study participants used:
- Terms that would ensure locality of the results, like “near”, or a specific city or county name as part of the search string
- Words that would yield timely results like “today” or “breaking”, indicating they were searching for current, up-to-date information
- The word “stories”, which might indicate either a desire for narrative/articles over stats and headlines
- Refinements to search specifying geographic locations, adding town/city names or zip codes to the search string
Some clicked on the News tab in Google then engaged with the Top Stories there, and while some engaged with the AI Overviews, many were concerned that the information was not timely or current.
Many bypassed searching altogether by going straight to their local TV station websites, or ESPN, Fox Sports, or other sports-specific websites if their interest was more in sports news. Some also preferred to go to social media or Reddit for news. One ChatGPT user found the LLM’s news inaccurate.
AI Mode was mostly seen as generic, less visual, and less trustworthy for current events.
Using AI Mode to Research Local News and Sports
No matter how many of these recordings I watch, I will always see someone do something I’ve never tried before. A 43-year-old participant from Fresno, CA, said she typically selects the Google News page from the menu on the right. I never even noticed that option in there before:
She then clicked the local news feed on the right to see the article headlines:
When prompted to use AI Mode, she thought it was much easier to simply click the News tab in Google rather than type out a prompt because that involved extra steps.
“I didn’t like that, to be honest,” she said.
Though AI Mode gave her different results, she thought they were still good. However, she wished there were more links within the summary (aside from the ones on the sidebar).
Another user had a different way of searching local news that I hadn’t seen before, either: he entered his zip code into the search.
When prompted, he tried AI Mode but decided “it’s more generic to me.”
One participant prefers to visit her local Facebook page for news, but overall tries to avoid the news if possible.
“I don’t watch the news because that’s just too much negative information for me,” she said.
She prefers to look up sports news for her favorite teams, the Phillies and the Flyers. Typically, she would enter “Phillies schedule” or “Flyers news” into Google to find out scores or game dates.
Oddly enough, unprompted, she said: “I would actually like it if the AI tool gave a summary of what’s been going on instead of having to look at all these different headlines.”
Well, she’s in luck, because that’s exactly what AI Mode does. But although she thought its summary was good, she didn’t think there were enough links and didn’t trust the content.
“Where are they getting this information from?” she asked.
She wasn’t the only user looking at their local Facebook group for news, though. A 45-year-old from New Jersey said that’s the first place she’d look for news, but she decided to give AI Mode a try.
She searched specifically for “body in Schuylkill,” which is apparently such a common occurrence that AI Mode showed a list of all recent times they found a body in that particular river, and even included a warning at the bottom emphasizing that these were separate instances/bodies:
Another participant would check her local NBC affiliate first for local news, but would consider checking Google and AI Mode for any follow-up questions. She said she liked how AI Mode gave a summary of the main headlines:
We can’t discount Bing, because someone in this study uses the Bing app to see top news headlines from Microsoft. She also checks social media for breaking news.
“It’s in my algorithm,” she said.
Many users tend to look at social media first for breaking news.
“I usually go on social media apps like Instagram, Twitter, Reddit,” another user said.
She has tried clicking the News tab on Google but said, “It’s never been appealing to me as an option,” explaining that the times she’s used it, the results have “been somewhat random.” After trying AI Mode, she was pleased with the results.
“This AI Mode actually is better than a lot of other news update features I’ve seen recently. Way, way better than the News tab.”
Reactions to AI Mode and its Format
One study participant specifically visits X for news, “just to see what people are talking about,” but tried AI Mode to look up sports news in Minneapolis and liked the results:
“It gives a good summary of each team,” he said, because it lists the key points so you don’t have to hunt for them. He added that he likes using LLMs like ChatGPT, Grok, and Gemini for in-depth analysis, but when it comes to just searching for quick info like this, he’d probably just use the Google search bar.
Another user who was less than keen on reading the news prefers to read video game updates. He tried AI Mode to see if it was up to date with the latest Xbox layoff news (he didn’t expect it to be) and the result surprised him.
“This does seem to have some of the recent news information in it,” he said.
But he didn’t enjoy the layout and the lack of images and graphics in AI Mode:
A 29-year-old participant from Virginia also disliked the layout pretty strongly. When he used AI Mode to find the top news headlines, the block of text he received didn’t feel impactful to him.
“Whenever the information is like this, it makes me not want to read it,” he said. “It’s not how I want to consume this information.”
However, another user actually liked the layout of AI Mode because he was looking for text to read rather than visuals. When he looked up sports headlines in NYC, he said he liked how the summary was divided by the different sports and they weren’t all “jumbled together.”
“It’s definitely showing me a different way of searching right now,” he said.
The AI Mode layout pleased a 29-year-old from Boston, who said: “There’s nothing wrong with my current workflow, but if I were really just looking for, like, a two-minute general rundown like this, I could see myself using it.”
While another user would typically do either a traditional Google search for local news stations or look at Fox News and ABC News, he was open to trying AI Mode to see the results.
He liked how it summarized the headlines for him in a clean format. “This is a much better result for me,” he said.
One study participant is on various work-related listservs and receives news updates every morning. He also gets news from X and LinkedIn, only clicking on articles that look interesting.
However, he noted that these days, articles are often behind paywalls and you need a subscription to read them. So if he wants to look up specific sports news, he tends to go to ChatGPT where it’s more direct and laid out point by point.
When he tried AI Mode, he was impressed:
“This is much better than the typical Google search,” he said. “This is much more helpful.”
He even tried a follow-up query for more specifics:
Skepticism for AI Mode
A 41-year-old from California tried AI Mode to look up stock news and liked the results. He was concerned that stock news in a Google search is not up to date, but he liked how AI Mode displayed dates.
“I actually like this. It gives you a pretty good summary of anything you search about,” he said. “It’s timely and it’s not based off something else years ago, months ago.”
When one user searched for local news in Idaho Falls, he noticed a few articles on the side panel that had dates, but it was hard to know the timeliness of the news in the AI summary.
“They are relevant I guess, I just don’t know when they happened,” he said.
He also didn’t think the top stories were very interesting.
“It says they’re top stories and urgent alerts but is this something I would consider a top story? Probably not,” he said.
Ozzy Osbourne had just passed away at the time of this study, and one participant said when he heard about it on social media he Googled it and saw all the news stations carrying the story, so that proved to him that it wasn’t fake.
AI Mode also knew about it, but since it just happened there wasn’t much to say about it yet. It just took the same info from other articles:
But when the user looked up a video game he liked, there was much more information for him:
AI Mode Considerations for the Publishing Industry
When it comes to local news and sports, AI Mode isn’t winning over the crowd just yet. Most people still fall back on their familiar habits, be it social media, local news sites, or going straight to ESPN for sports news. But what we did see was curiosity and an openness to try AI Mode for news.
Some participants appreciated the quick rundowns and clean summaries, helping them get the news in one place rather than clicking around to various articles. Others found it too wordy, too generic, or too out of date for breaking news stories.
The tension between convenience and trust, and speed and credibility will define how AI Mode grows in the news space. Users aren’t closed off to AI summaries, but they want them to feel timely, transparent, and useful. Until then, AI Mode is more of a supporting act than the main stage in the publishing industry.
Read the previous articles in this series:
