Ep 5.3: AI Visibility Metrics: Are They All Bull&*$t?

[Genny]: AI visibility tools are everywhere. If the algorithm even gets the slightest whiff of your interest in AI analytics, you can be sure that there’s an aggressive ad for a GEO tool luring you in with promises of complete AI visibility and the blueprint for total AI domination.

Sign up and see how you score on ChatGPT and Perplexity.

Get the insights you need to show up in more answers, reach your customers, and kick your competitors off the board.

After all, our clients are using AI systems to search for solutions. They are having conversations with their favourite AI sidekicks. Understanding what our brand’s current presence looks like on these systems does make complete sense.

But hold on a sec. What exactly is an AI Visibility Score? Because we don’t have access to true first-party model data or decision logic…so where are these numbers supposed to be coming from?

Are they simply vanity metrics designed for marketers being pressured by their cigar-smoking, yacht-hopping CEOs to dominate AI conversations before somebody gets fired?

To figure that one out, let’s go down the list of AI metrics these tools package up and figure out what’s actually useful, what’s actionable, and what’s complete bullshit.

I just want to frontload this episode by saying that these metrics aren’t totally useless. The problem is less about what they measure and more about how they’re being sold.

We’re limited by what we can actually track, so these tools are doing the best they can with the data available. Once you understand the methodology, the sample size, and the limitations, you can start using the data strategically instead of treating it like gospel.

Because there’s no use getting excited about an AI visibility metric that isn’t going to help you close your topic gaps, create the right content, and be part of more conversations.

The metrics we’re talking about today are of course:

• AI Visibility Score, or the weighted estimate of how often and how prominently your brand appears in AI-generated responses compared to your competitors.
• Share of Voice, or how often your brand is mentioned, recommended, or cited in AI compared to your competitors.
• Citation Count, a tally of your mentions in AI.
• Brand Sentiment, or whether or not your brand is represented in a positive or negative way.

And before we get started, I’m just going to address the elephant in the room right now…

Yes, we’ve changed our podcasting format. We’ve actually received millions and millions of listener requests to see my face, so let us know what you think about the format change in the comments. And please be gentle, my ego is a fragile baby bird.

First up, let’s talk about what is perhaps the most contentious, the most talked about metric, the one that everyone wants, but the one that experts are kind of scoffing at:

The AI Visibility Score

Is it bullshit? Yes!

And…no.

There’s a little rumour floating around the water cooler that all these tools slinging AI visibility scores aren’t being totally honest with you.

Because, like I said earlier, where is this score coming from? What is it based on? Go down that rabbit hole and you’ll soon realise that the score isn’t actually based on first-party data but the company’s proprietary methodology for compiling the data.

Those tools can see the prompts they send, but not the hidden system inputs, retrieval layer, or decision logic behind them. So what they’re doing is inferring visibility from prompt testing, mention frequency, citations, and the rerunning of prompts to see the difference.

So, in that context, is it a vanity metric? It kind of depends on how the tool is selling it. If they’re packaging this score as some kind of magical doorway into referrals, conversions, qualified traffic, or pipeline, go ahead and check your eyes because someone might be trying to pull the wool over them.

But Genny, marketers don’t have access to Google’s algorithmic data, and SEO has been the backbone of digital marketing for over two decades. So why should that matter?

Yes, that’s true. SEO isn’t a perfect science, but the metrics we use to calculate your SEO performance, like rankings, traffic, impressions, clicks, and indexed pages, are much more standardized and historically validated.

But not having access to first-party data is only part of the reason your score isn’t the holy number it’s often made out to be.

It’s also because AI answers, as you probably know, are fickle and can change significantly with prompt variations or over time, especially for sloppy queries. Tweak your question or prompt ever-so slightly and you may get a totally different response based on not just your marketing efforts but also, not to oversimplify it, on how the AI decides to respond on a particular day.

It’s a snapshot, not a complete picture of your overall AI visibility.

This feels especially true if the tool excludes some of the most popular AI models out there. That’s because answers can vary wildly across the LLM sphere, so you may get a referral in Chat but be completely missing from the conversation in Claude. It just goes without saying if its sample size is smaller, the data is less representative.

And believe it or not, not everyone’s using Chat in 2026. For reasons we don’t need to go into, some buyers actively avoid using ChatGPT, which, pronounced in a French accent, is bound to get a few giggles from the Francophone community. And if you don’t know what I’m talking about, go ahead and ask Chat to explain it to you.

All that to say, find a tool that does more than give you a Chat score.

To get a better idea of how your brand measures up, you need to scan your visibility frequently and put more faith in trends, not final scores. Which of course, is highly recommended anyway, and something most AI tools offer, but I’m saying this just to dial down the mania around getting this arbitrary score over a certain threshold.

So, to bring it all back: AI visibility scores are proxy metrics. They can help you understand which direction to go in, but they are not the same thing as outcomes like traffic, conversions, or pipeline.

Don’t use it as your north star because you may end up wasting your time, budget, and sanity.

Why this topic gets so many experts riled up is that even though we tell marketers why these scores are misleading, they’re still under a lot of pressure to send clean and readable reports to upper management.

That solid number is a great deliverable, but it also creates not only a set of false expectations, but also a bit of cognitive dissonance: We know the number by itself is a vanity metric, but we still have to present it as a real KPI.

And yet…

The score isn’t entirely useless. In fact, it’s only a vanity metric when you treat it as something precise or business critical. And if the tool you’re considering is selling it like that, maybe treat that as a red flag.

It is certainly not a vanity metric if you balance it out with other data like your share of voice, citation rate, and yes, your SEO data.

And I’m just going to say it.

It sometimes feels like the industry is having to whisper about SEO these days just to avoid the barrage of LinkedIn influencers calling them out for being out-of-touch dinosaurs.

But this entire conversation doesn’t even matter if your SEO sucks, so do not drop that hot potato, and please do not base your entire marketing strategy solely on the advice of people whose job it is to get your attention.

Your AI visibility score may not be a like-for-like representation of real data like your GA4 metrics, but there is a strategic methodology behind it.

For example, GeekyIQ, Geeky Tech’s new AI Brand Intelligence Platform, uses a number of prompt types that cover what we’ve determined give the best representation of your visibility at the time of the scan.

It runs through a long series of prompts that cover everything from Brand Identity, to Competitor Landscape, to Category Ranking, and more. Each prompt type is scored individually and then added up and voila, there’s your shiny number. Like I said, not a like-for-like representation, but not based on nonsense after all.

To summarise, we set out to discover whether or not your AI Visibility Score is a bunch of bullshit, and the answer is slightly more nuanced than yes or no. It’s more about how the score is sold and used and less on the number itself.

The goal isn’t to chase a perfect visibility score. The goal is to become a source AI systems consistently trust, cite, and recommend.

And now I think we’ve kicked this dead horse enough times. On to the next contender for AI metric bullshittery, which is AI Share of Voice, or SoV. Are people using SoV?

Share of Voice

Is AI share of voice a bullshit metric? I think it tries not to be, but it’s another one of those ‘it depends on how you use it’ type of thangs. Oh, isn’t the world wonderfully nuanced?

Bad news: AI share of voice is also not a hard science…

AI Share of Voice isn’t bullshit, but it can be misconstrued or misunderstood. In SEO, your SoV shows you which keywords, topics, and SERP features are driving that visibility.

In GEO, however, your AI SOV is again, based on a sample of prompts—a very large volume of prompts, but certainly not every prompt variation under the blue sky. So, again, in that sense, it’s leaning towards a highly educated guess. Which is certainly better than a kick in the face.

But the other thing is, your SOV alone doesn’t tell how your brand is being represented.

Is 80% share of voice still a great number if Perplexity is incorrectly representing your brand? You need more context in order to understand if that number is doing your brand any good. Don’t turn it into a vanity metric by just looking at the number and making a snap judgement.

And lastly, knowing you have a 40% share of voice isn’t all that helpful if you don’t know how you got there. That doesn’t make it a vanity metric per se, but it needs to be used in context and balanced with other data.

I wouldn’t consider it a pure vanity metric because if you’re scoring really high, you may see more clicks, depending on how often links are included and how users behave; that is of course, if AI isn’t talking smack about you. Like your AI visibility score, your share of voice is best measured as a trend and not as a snapshot.

So, dig a little deeper when shopping around for GEO tools to find out if they’re offering more than just the number. We don’t just need the numbers, we also need the instruction manual. We need the actions behind the insights!

AI Citation Count

Next, let’s look at citation count. This is a nice metric especially if the numbers look impressive. But it creates a false sense of performance because this number by itself doesn’t really tell you all that much.

Just like SEO, it doesn’t indicate how you stack up against your competitors, which is obviously the most important detail. You could be mentioned 1000 times but if it’s only about a single topic or a low volume topic, it’s not actually that impressive, no offense.

Don’t get me wrong, it does show that you’re on the AI scoreboard. But if you want to know whether you’re actually seen as a trusted authority, you also need to look at citation rate, also called link rate, and this measures how often the AI provides a clickable link. This, by the way, is what we call referral traffic, and that’s not a vanity metric.

And by the way, it’s also important to compare your mention rate to your citation rate. If AI mentions your brand but rarely links to you, there’s a good chance your content isn’t being seen as the most useful source to reference.

Now that would give you something useful to look into.

Brand Sentiment

Yes, knowing whether or not someone is shit-talking you would be valuable information to have. But the thing we tend to forget is that most mainstream models have their dials set to neutral or soft language. I mean honestly, when have you ever seen Perplexity truly rip into somebody?

We’re talking about the same model that had to be updated in 2025 because everyone was complaining about how annoyingly sycophantic it was being.

AI generally tries to give a balanced opinion on everything and even when it’s being slightly critical, it’s also sandwiching that criticism with compliments. So, yes it’s nice to know if you show up in a response in which you’re given 5 gold stars, but it’s not really going to help your marketing strategy all that much.

Brand sentiment is a huge selling point for some of these tools, but I think what’s happening is the whole idea of needing to police AI sentiment is slightly overinflated.

So, what’s the bottom line: GEO and AI visibility metrics are useful when they help you make better decisions, not when they just give you another number to report.

So, yes, go forth and find an AI visibility tool because it brings you much closer to understanding how you show up in AI, even if those tools need a side of insight and context to make it make sense, which by the way, is also true of SEO.

The real value comes from understanding what the data means, where the gaps are, and what to do about it. Because like I said, having these numbers is one thing, but knowing how to act on them is something entirely different.

That’s what GeekyIQ is built for, if you’ll excuse my shameless self-promotion. Our strategists and innovation team got together and developed an AI brand intelligence platform that does just a scootch more than give you a pretty dashboard. We also provide prioritised recommendations in clear language so you know exactly what you need to do to close those topic gaps.

And if you’re like every last one of our Geeky Tech clients and don’t have the resources to build your content, don’t worry, because GeekyIQ’s content generator can do that for you. We also make it easy for you to track your progress, run your own visibility audits, and effectively complete your entire AI visibility workflow in a single app.

Okay, consider the shameless self-promotion over.

So that’s all for today folks. Let us know what you think about the new podcast format. Do you think you’ll switch to video or stick with old faithful? Tell us your thoughts. I promise I will take everything you say personally.

And if you’re interested in testing out our newest AI Brand Intelligence Platform, go ahead and click on the link in our show notes or mosey on over to GeekyIQ.com to start your 30 day FREE trial.

Having never signed off on a video podcast before, I assume you’re supposed to wave and smile? Or maybe just stare at the screen long enough to add a vignette over my face in post-production? Or maybe I’m just supposed to walk away? Or maybe….oh, I know….