A closer look at the SEMrush AI Toolkit: Game Changer or Gimmick?
TL;DR: The SEMrush AI toolkit looks slick and promises strategic insight based on LLM data — but under the hood, it appears to be a lot of guesswork dressed up in dashboards. While still cheaper than alternatives such as Profound, I don't see it's worth the $99/month - yet.The first thing you have to know is that the SEMrush AI toolkit isn’t positioned as a tool to optimize your visibility in LLMs, but rather as a “strategic insights tool”
Yes, some suggestions aim to improve visibility — but the pitch goes further: Just enter your domain name and get competitive intelligence, brand sentiment, and strategic recommendations, all based on LLM output.
So I put it to the test. After three weeks of hands-on use, demos, conversations with customer support, and client projects, here’s my take:
So how does it work?
The official documentation lacks detail, but after speaking with customer success and support, here’s how I understand the process:
There’s a large data set of queries to draw from. I call them queries and not prompts because they are mostly simple questions not detailed commands. Also, these are NOT real queries that users actually typed into ChatGPT. Partially they come from the SEMrush search module (i.e. keywords) and partially they are AI generated themselves (i.e. probabilistic guesswork). I'll get into the problems with that later...
The only input you give is your domain. SEMrush will not crawl the content of your website. Instead it will select a set of queries that should be analyzed further, based on the topic cluster your domain fits into. So for the example report analyzing for Warby Parker, this could be questions such as "What eyewear retailers offer virtual try-on experiences?"
This set is then fed to the major LLMs: ChatGPT, SearchGPT, Gemini and Perplexity (Claude and Deepseek are announced)
All those queries will produce a new dataset of LLM outputs with the answers.
This set of outputs is synthesized into reports and “strategic recommendations”. Your visibility is compared to your competitors.
These are the basic steps how the strategic insights are generated
The three reports
The tool offers three different reports that can each filtered by platform:
1. Business Landscape: This section shows your market share compared to your competitors (It's called market share, but what they mean is your relative visibility in the LLMs)
2. Brand & Marketing: This is a sentiment analysis, listing your top positive and negative drivers for your brand perception.
3. Audience & Content: An overview of the different topics that your brand is showing up for, categorizing different queries
Let's go through them one by one.
1. Business Landscape Report:
The main insight the Business Landscape Report provides is an indication of your brand's relative visibility across different LLMs.
While this is a nice overview of visibility, it's impossible to properly interpret, since the methodology isn't transparent. How many queries have been analyzed? Which queries? Where are these queries coming from? - Nobody was able or willing to give a satisfying answer here.
This makes it difficult to trust the "strategic recommendations" SEMrush makes based on these data, like:
This recommendation suggests that Warby Parker should offer eye exams in more stores, based solely on ChatGPT outputs and data that SEMrush doesn't share. Switching to results for Perplexity or another platform shows entirely different recommendations. This is expected of course, since the outputs will be different for other LLMs. But you claiming that you should change your business strategy based on these LLM outputs, feels a bit presumptuous.
Running the same report for a client, showed similar recommendations that sound smart at first glance, but don't seem to be based on anything of substance.
2. Brand and Marketing
The core of the Brand and Marketing section is a sentiment analysis, showing positive and negative factors contributing to brand perception.
However, once again the trustworthiness of the data is more than questionable:
When running this report for a client, the market share for ChatGPT and Gemini showed 0%. That makes sense, because the startup's brand isn't part of the core training data set of these LLMs. To really be sure, I asked ChatGPT directly what it knows about the brand, while specifying to NOT search the internet.
It's clear that there is no data available...
... yet, there's an in depth sentiment report in SEMrush for both ChatGPT and Gemini... So where is the data for that coming from?
Upon inquiry SEMrush responded, it's still theoretically possible that some brand mentions occured, even with 0% visibility an it has just been rounded to zero.
But how much confidence would this "almost zero" data give you in the validity of this report?
2. Audience and Content
Lastly, there's the Audience and Content report. Here, we go in depth on the different topics LLMs associate your brand with most frequently as well as the specific intent and questions users ask.
For each topical cluster ("Shopping experience and convenience" in the example below), you can see percentages for research, comparison or purchase - as well as specific examples of queries for each type of user intent.
There's just one problem: The questions aren't real questions users ask, but just AI generated questions, users might ask. But what's the point of that? Somewhat annoyingly, SEMrush insists that these are actual queries, real users typed into ChatGPT. But I can assure you they are not.
If you are running that report for a large brand, you might not even notice it, because all of these questions sound plausible. But when I ran the report for a pre-seed startup with a hand full of branded searches per month, it became glaringly obvious that nobody actually asked these highly specific questions including their brand name.
The positives
Ok, I've been ripping on that tool now for a bit, but of course it's an early version of an entry level toolkit (I've had a look at the enterprise version as well, which is quite a bit more advanced.. and expensive), so maybe my expectations were a bit high.
While the reports are still quite basic, SEMrush is releasing new features constantly. Claude and Deepseek are coming soon as additional LLMs to be analyzed, and while I was writing this a long list of competitors was added. So now you can't just compare your brand with the top 5 most frequently cited competitors, but with any competitor that shows up:
About 50 different competitors available, you might even find some you didn’t know of before. Neat.
About 50 different competitors available, you might find some you didn't even know of. Neat.
This is great, because in our case the most prominent competitors weren't the most relevant ones.
Also, the visibility overview over time can give you a general indication of how you're doing on each platform over time.
The core issue
To understand the general issue with this tool, let's take a closer look at the "strategic insights". Look at this is an example of a strategic recommendation that SEMrush generates after analyzing millions of queries against dozens of competitors:
Ok sounds reasonable, but what data is this based on?
Now, compare this is to a ChatGPT answer when prompting it to give some strategic recommendations for Warby Parker:
Practically the same insights as above
Right, it's pretty much the same. I'd even argue that the ChatGPT recommendation is better because it cites sources, data, and is more specific.
Not every example is like that, but the problem is clear - everything is kept vague and generic:
1. The data and process:
How many queries are there? How are the relevant ones chosen? How are the insights selected? Many things stay very opaque in this report and documentation isn't very illuminating either.
2. The recommendations themselves:
Examples from a client's report:
“Improve your onboarding”
“Monitor feedback”
“Increase market awareness through targeted marketing campaigns”
Most of these recommendations either apply to most businesses, or are just taking it too far. And as the example above shows, can be created with a simple ChatGPT prompt as well.
If I would put these on a slide, sure they would make a good action title. But as soon as someone asks a question like “Ok, but why? What data or deeper insight is this advice based on?” - it falls apart.
A useful recommendation would be: "All of your competitors are listed in major news sources, that are referenced in 60% of queries" - Get published on [source 1,2,3,4] to gain visibility in ChatGPT Search and Perplexity
Verdict & how to make this tool much more valuable:
In it's current form, I can't say this tool is worth $99/month. Not for a startup and not for a big brand either.
SEMrush simply lacks the context to confidently make suggestions like "Expand your business line to capture market share in the premium segment" or "Lower your prices".
What I would gladly pay for: Just give me clean data and let me draw the conclusions myself.
