Businesses scramble to get noticed by AI search

For many businesses, their website is a vital shopfront, so losing 140 million visits in a single year would be a big problem.

That’s what happened to HubSpot, and the cause was AI.

The company provides sales, marketing and customer service tools for business-to-business companies.

Like many firms, HubSpot has been hit by a crucial change in the way we search the internet.

“I remember the days when I would search [the web], and there was no good information,” says Kipp Bodnar, chief marketing officer at HubSpot.

“Sometimes there was some stuff, but I had to scroll through 10, 20, 30 links.

“What you have now is access to all the world’s intelligence in an instantaneous way. How people find information and subsequently take action is very, very different.”

For companies like HubSpot, there are several causes for the drop in traffic. Search engines rejigged their algorithms to fight AI-generated slop, making it more important for a website to be seen as credible on a core topic.

Users are increasingly switching from search engines to AI tools. Meanwhile, search engines themselves are including AI overviews at the top of their results, which often means users get their questions answered without having to click through to another website.

“The click-through rate for searches that have AI overviews is about 60% to 70% lower,” says Bodnar.

So, companies are trying to figure out how to be prominent in the answers AI gives.

Answer engine optimisation (AEO), sometimes called generative engine optimisation (GEO), is about helping websites to rank well in AI tools, including AI overviews and tools like ChatGPT.

These are built on an AI technology called large language models (LLMs).

Many companies are using AEO alongside search engine optimisation (SEO), which aims to get websites ranking in search engines.

“We’ve been able to use answer engine optimisation to increase the conversion rate and quality of the people who are coming to us,” says Bodnar.

“I don’t know how you are a competitive business in the future without having a strong competency in this.”

It requires an understanding of how search behaviour is changing.

“Maybe you enter four to six words in a traditional Google search,” Bodnar says. “In an AI search engine, the average length is 40 to 60 words. So, you’re talking about an order of magnitude of specificity change.”

He gives the example of a company that rents motorhomes in New Zealand. Someone might ask AI for a complete holiday plan for a family of five, including an opportunity to see a favourite animal.

To be cited in the answer, the motorhome company might need to publish an article on the most popular animals in New Zealand for children to see. It needs to be written in natural language that matches the questions people might ask.

HubSpot has been restructuring its own content.

The company used to have long articles about its products and how all their features work together. That’s not needed so much now that AI can provide that explanation, Bodnar says.

The new structure uses small chunks of content that the AI can easily extract. If someone asks about the contact management feature, for example, AI tools can easily find that chunk of information.

AI now delivers between 7% and 12% of HubSpot’s website visitors in most months, but Bodnar says it will be an even more important way for customers to discover the brand.

“You’ll see people coming through direct traffic and other sources because they were influenced by those LLM responses,” he says.

“In order to survive, you have to adapt,” says Ann Lowe, head of PR and communications at Spice Kitchen. The company sells spice gift sets.

To support its latest product, Spice Kitchen is building a content cluster about the history of the spice trade. It’s a dedicated subsection of the company’s website that aims to demonstrate authority on the topic.

“We’re wanting to see whether we can hit the AI search bots with that content,” she says. “It won’t be a shop. It will look almost like a training course. This is for people who are doing research, but they get to discover us along the way.”

She’s worked closely with an agency – Lumos Digital.

“Historically, you’ve always optimised the product page so that you are picking people up at the moment they’re ready to buy,” says Nathan Pearson, co-founder of Lumos Digital.

“Now, that focus seems to be shifting towards the research and decision stage and winning them at that point,” he says.

He recommends that companies publish buying guides. “If you’ve got a guide of the best trainers for long-distance running, make sure all the products are listed and have a clear winner. AI loves that.”

Research or media organisations that want to rank in AI can learn from some of Spice Kitchen’s other practices.

Andy Lochtie, co-founder, Lumos Digital, emphasises the importance of expertise, authority and trust indicators.

That would include having lots of links in to your website from other trusted websites, linking out to high-quality websites, and having content policies and author biographies to boost credibility.

Andy Pickup is the digital director at MKM Building Supplies, an independent builders’ merchant that also sells directly to the public.

“We are seeing fewer people come to the site because they’re getting the answers from an AI model,” he says. “They don’t need to visit our website to read a blog on how to fit artificial grass or whatever it might be.”

“If that trend continued, you’d potentially see your site traffic almost dwindle to nothing.”

Pickup recognised the importance of being cited in the AI results. “We need to make sure that, when people are searching for answers around building projects, these AI models are referencing us rather than our competition.”

He hopes that will help to drive footfall in stores, where customers can get help with their projects from the staff.

Although Google is the dominant search engine, ChatGPT is sending more visitors than Google’s built-in AI.

“It’s a seismic shift in user preference of what app [customers] use,” he says. “They’re making a conscious decision to not go into Google, even though it’s got built-in AI, and are actually going into ChatGPT.”

He embarked on what he calls a “defensive strategy”, creating blogs about the best-selling products for the AI tools to reference.

“It was similar to SEO, positioning yourself as an expert in these areas and making sure you’re giving the LLMs everything they need to provide a thorough and conclusive answer,” he says.

“The content evolves from just talking about a product. It’s more about how this product’s going to help you solve a problem.”

Search engines look for keywords, but AI engines need to process the meaning of the page easily. As a result, MKM’s new pages have a summary, bulleted lists to break up information and frequently asked questions (FAQ) lists.

“It’s about making sure your content is very clear and concise and easy to understand,” he says.

Behind the scenes, there is a site map to help AI bots find their way around the website.

While many people will simply read the AI answer, some will click through to the source. In the last year, MKM’s traffic from AI has increased from almost nothing to “a low double-figure percentage”, and it’s still going up.

AI visitors are much more likely to buy than search engine visitors, Pickup says. “My theory is that customers have got the information they need from the LLM answer, which gives them confidence to make a purchase.”