There has been some buzz over a new paper called Durably Reducing Conspiracy Beliefs Through Dialogs With AI. You can read the paper here. In their paper, they found a roughly 20% reduction in conspiracy beliefs, and this reduction was still in effect two months later. So far, so good.

To be fair to the paper’s authors, it has not been peer-reviewed. I’m also not directly refuting their results, but I am questioning their data, which affects the results. In this post, I’d like to highlight a few items that should raise eyebrows as well as give people some ideas for a more critical eye when it comes to data. Far too often, people read the abstract of papers like this and share the post, but we must give a more critical eye to research rather than just taking results for granted.

The Obvious

There are a couple of obvious observations here that I’ll quickly note and move on since it’s not the subject of this post.

• If the outcome is true, then the inverse is probably true. So, if AI can break people’s conspiracy beliefs, it also has the power to implant them.
• This wasn’t the default version of GPT-4. They used GPT-4 Turbo and specifically tailored it to have these dialogs.
• This study was done outside the normal area where people with conspiracy beliefs would encounter such tools.
• In an age of personal AI, people with conspiracy beliefs may have their own AI that they’d use to refute these tools. So, the AI vs AI fight club is probably closer than we think.

Now, let’s look at the bigger problem.

The Bigger Problem, Data

As soon as I saw this study, a huge question surfaced. Where did they find these conspiracy theorists?

Conspiracy theorists are typically suspect of government and academia. On a recent road trip, a truck passed me with a bumper sticker stating, “Not a Ph.D.” This was followed by several other bumper stickers claiming the election was stolen, Q something or other, and a few more that could be easily guessed. However, this person didn’t appear to claim the earth was flat, so I guess that’s a silver lining, but there’s still a catch.

Conspiracy beliefs are like Pringles. It’s hard to have just one. These beliefs make them more paranoid and suspect, especially in studies and academia. I find it hard to believe that these true believers would willingly participate in such a study and that a few rounds with a chatbot would reduce their beliefs.

In the real world, family members and loved ones have implemented similar strategies to no avail. These people have emotional ties and strong bonds, and even that wasn’t enough to change their minds. The only thing that helped was a completely different tactic, not refuting the beliefs but telling people that they were worried about them and that they cared for and loved them.

To change beliefs, one’s mind needs to be open to new possibilities. Conspiracy beliefs are powerful because they allow one to play both the victim and the hero at the same time. I wrote about this aspect back in 2020 with an article called The Cult of Conspiracy Theory. All of this is evidence of pretty strong cognitive distortions that conspiracy theorists hold. So, back to the question. Where did they find these conspiracists?

They used a product called CloudResearch Connect. This is a site that pays people to take surveys.

I read the website but didn’t do a deep dive into this company or examine its methods more closely since this wasn’t important for the points I’m making in this post.

Now that we know the source, a new question arises. Are over a thousand conspiracy theorists waiting to complete surveys on Connect? It’s possible but doubtful. Here is another question to ponder. Is it more likely that people with durable conspiracy beliefs are a part of CloudResearch Connect, or is it more likely that people looking for extra cash would be willing to answer questions with the persona of a conspiracy theorist?

When reduced to this question, the answer is fairly obvious. I also get it. It’s hard to find people to participate in studies when you are researching topics like this. Far too often, this research is done with participants made up of the student body—hardly a representative sample of the real world.

When Data is Dark

The participant data in this paper covers multiple categories of David Hand’s Dark Data. In his book, Dark Data: Why What You Don’t Know Matters, he covers 15 categories of what he calls dark data. It’s an illuminating read, and once you see data in these categories, you won’t be able to unsee them. Hence, despite reading the book four years ago, the categories stuck with me.

Immediately, I spotted at least three main categories of dark data.

• Choosing just some cases
• Self-Selection
• Feedback and Gaming

Choosing Just Some Cases
The researchers choose to count on Connect for their entire sample. The sample provided by Connect may not be representative of the real world. This may cause them to miss the effects of real people with conspiracy beliefs and instead have people posing as conspiracy theorists.

Self-Selection
All the people filling out the surveys volunteered to be part of the study. There may be a stark difference in how people who decide to be part of the study answer the questions from those who do not. In this case, actual conspiracy theorists.

Feedback and Gaming
This happens when the feedback itself influences the values of the data. In this case, getting paid to be part of the study. Participants may be more likely to answer favorably to certain questions, playing the role of a conspiracy theorist to get paid.

In fairness, the paper’s authors were aware of some of these issues and tried to account for automated responses and confidence in their conspiracy beliefs. This can be seen in the Materials and Methods section of the paper. I think, in the end, there just weren’t any true conspiracy theorists on this site. Well, that’s my theory anyway.

Most Surveys Results Are Garbage

Let me give an example of how to think about these scenarios more simply. If I stood in the frozen dessert section of the grocery store and asked every person buying ice cream if they liked ice cream, I’d be in the high 90% range for respondents. This would be different if I asked people at the entryway to the grocery store or on the busy streets of Manhattan. I’d also have to ask if my sample size was representative of the population I was measuring. Did I ask everyone walking into the grocery store or only ten people?

I saw a survey a few years ago that made broad statements about the cybersecurity industry worldwide by surveying less than 100 people. Is it possible for less than 100 to represent an industry with many different focus areas across many different industry verticals, in many different countries, and at various levels of seniority? Of course not, but this type of thing happens all of the time. Survey data quality has reduced dramatically in value over the years. This, combined with poor data collection, equals garbage.

For a quick gut check on surveys.

• Does the sample size seem representative of the population?
• Does it seem likely people would answer questions differently based on the collection?
• Did respondents self-select?

Here’s an example to clarify the self-select question since it may be blurry. Say the International Pizza Lovers Association sent out a survey. As a pizza lover, I may be more likely to respond than someone who is lukewarm or doesn’t like pizza. So, I self-select based on my enthusiasm for the topic, thereby biasing the data collection. This can also be negative. Anyone who’s spoken at conferences and gotten feedback knows that the people most likely to fill out feedback forms are the ones who didn’t like that talk.

Conclusion

I think the manipulation of humans by bots is a fascinating topic that deserves more research. However, for any research results to hold, reliable sampling of the population being studied must be conducted. Otherwise, we may be fooling ourselves. There’s an ongoing replication crisis in academia across various disciplines. In this post, we examined one of the contributing factors to this crisis.