AI as a substitute for thinking

As you are no doubt aware, AI is everywhere. Or, rather large language models (LLMs) and some other tools which use “artificial intelligence” are everywhere.

I was, from the beginning, a skeptic. That is my nature. I have a slight Luddite tendency but that is heartily reinforced by my dislike of hype - I am always skeptical of hyped things - technology, cultural artifacts - and often react very negatively to those things.

However, over the last year I’ve come to use AI more. Specifically, Claude. I have used Claude (and occasionally Gemini or ChatGPT) for some of the following things:

• Helping me with Linux issues as I’m a noob, the people in forums are assholes, and it’s so much easier to just get Claude to tell me what to do than to learn something

• Writing generic content for websites

• Performing competitive analysis for competitor websites

• Confirming the existence of some piece or research (and other in depth internet searches that might take me quite a while, especially with how bad Google search has gotten)

• Critical feedback on things I’ve written (for work, not for my blog or for this newsletter)

• Design help (as I always say, I cannot design my way out of a paper bag, so Claude’s design abilities are a true life saver).

But I have few illusions. I know LLMs hallucinate. And they get things wrong all the time.

• They make up sources (you must click and read the links if you are using them for research!),

• They miss things (the number of times I’ve used Claude to evaluate a webpage and it’s missed something obvious is…well, a lot of times)

• They give different advice and answers if you ask the same question twice,

• etc.

And I think I’ve found a pretty good example of how flawed they can be: research about people. If you’re a very prominent person, they seem to do a good job. But if you’re not, they get confused or fooled very easily. To wit:

I am not a very prominent person. However, I have been writing on the internet for 20-something years and for much of that time, much of the writing I’ve done is at rileyhaas.com. I have written lots for other sites but I’ve only used so many usernames - often my full name or initials - and so many photos. It should be easy for an LLM to figure out who I am.

In fact, if you ask Google about “riley haas” and use AI mode (i.e. Gemini), it does a pretty good job:

• “Who they are: A Toronto-based digital marketing strategist and trainer.

• What they do: He is a passionate writer who obsesses over music and cinema, having reviewed thousands of films and music albums.

• Where to find them: He runs Riley Haas' Blog and frequently contributes to music commentary platforms like Cover Me. [1, 2]”

I think that’s a pretty great answer given I didn’t say “riley haas writer.” It missed lots of places I’ve written but it has the current sites. And it’s the first one (well, at least it was, for me personally, at around 9:40 AM on July 26.)

But there’s this site called “Are you in the weights?” (Weight is the current jargon for how LLM models are tweaked to give better answers. As with all new jargon, I find it extremely annoying how suddenly everyone is talking about “weights.”)

It takes answers from multiple LLMs to tell the user how prominent a particular person is. It does a good job on really famous people.

And guess who the first answer is for my name?

It says Gemini says:

It says Mistral says:

If you’re a Leafs fan, you know there is no Riley Haas playing for the Leafs. (Wouldn’t that be wild?) I did write a book about the Leafs, though, didn’t I? Interesting.

Searching Hockey DB, which has ECHL stats, for “Riley Haas” leads to a Riley Hawes who played over a decade ago.

My own Gemini search does not yield a hockey player but rather a women’s college soccer player.

Both Are You in the Weights and Gemini agree there is an actor though they completely disagree on how many roles they have. IMDB and Gemini agree she has one role. Are You in the Weights - which should be “more powerful” i.e. smarter than Gemini alone - invents a whole bunch of roles which are not in the actor’s IMDB page.

Then, Are You in the Weights has “5 possible hallucinations” which they say are “probably fabricated.” They are:

1. “Private of Low Profile Individual”:

   1. GPT says "A name that may refer to a private individual; no uniquely identifiable public figure is strongly recognizable from the query alone.”

   2. Claude says “A blogger and writer known for cultural commentary, book reviews, and essays on music and philosophy, often publishing on a personal website.” (Hey, look at that!)

   3. Claude (different model) says: “Possible social media personality or private individual; name is common and lacks distinctive public recognition without additional context.” (My name is common?)

2. “American soccer player”: GLM says: “American soccer player who played for the Colorado Rapids and Seattle Sounders.” (No pro soccer player on Soccer Reference though there is a college player.)

3. “American baseball player”: Llama says: “Riley Haas is an American professional baseball player.” (No professional baseball player on Baseball Reference.)

4. “Gaming YouTuber”: Qwen3 says: “Riley Haas is a YouTuber and content creator known for gaming and lifestyle videos.”

5. “Canadian lawyer and author”: GPT says: “Canadian lawyer and author associated with legal writing, including commentary and work on Canadian law and civil liberties.” (I have written a fair amount about Canadian civil liberties over the years but sure this could be another real person.)

(I would note that there used to be a musician named Riley Haas, don’t know if he still performs. He doesn’t appear here.)

What is going on here? Why is a site using inputs from many LLMs worse than one? Shouldn’t it be the opposite.

The most obvious answer is that Are You in the Weights is badly programmed. (I assume it is programmed by so-called agentic AI.)

But I think it’s deeper than that. I think Gemini gives a better answer because, despite my lack of prominence in the world at large, I am quite prominent in Google search. (Gemini is Google’s LLM.) It is probably trained more on Google search results than some other information.

The other models are maybe less, um, weighted towards search results than Gemini and so are getting confused by other sources that maybe I cannot so easily access.

Moreover, the combination of the models seems to have made the results worse!

Only one of the responses I can see at Are You in the Weights is correct (about me) and it was deemed a hallucination because of actual hallucinations and other people who share my not very common name. That’s bad, right?

For me the takeaway about AI remains what it was last year, even though I use AI (Claude, mostly) for lots of stuff now:

AI is fine for certain tasks but is no substitute for thinking.

I am a person who exists who has written on the internet since 2004 or so, mostly under his own. This is factual, it is not something that needs to be reasoned. Moreover, my life is partially on the internet so easy to verify unlike, say, my parents’ existences.

And yet, the AI models can’t agree that I exist or agree about who I am. Moreover, they appear to have invented multiple slightly more prominent people in order to cover up their inability to just figure out I’m the guy at rileyhaas.com.

I find this fascinating but I also think it’s pretty damning. This isn’t a hard thing to figure out, is it? I mean, it would be for another family member of mine, who is less all over the internet than me. But, even having deleted most of my social media, I am still all over the internet. This should be a layup for any LLM, right?

To be clear, I’m not demanding they put me first or anything like that. I just think they all should be able to identify one Riley Haas as Gemini did. It strikes me that, if they can’t identify me, they likely can’t satisfy a lot of other factual queries.

Be careful out there.