Expert Bullshit Detection

John V Petrocelli. Skeptic. Volume 26, Issue 3, Summer 2021.

“You can fool some of the people all of the time, and you can fool all of the people some of the time, but you cannot fool all of the people all of the time.” — Abraham Lincoln

If Abraham Lincoln was correct, then there should be people who are rarely duped by bullshit. Who are these people? Can they help us understand the moments when critical thinking fails and how to overcome them?

These are the questions I’ve asked in my qualitative research on bullshit and bullshitting. It involves identifying expert bullshit detectors and understanding their styles of thinking.

From my research, I am convinced that there are many specialized, expert bullshit detectors in our midst. Through years of experience, they’ve “seen everything.” They’ve been witness to every bit of bullshit pulled in their respective professions and industries. They have highly specialized knowledge that makes it very difficult for people who don’t know what they’re talking about to get away with bullshit. We have much to learn from them. And so, I went out into the field to discover what they do differently from the average person. Here are two examples from my book—used car dealers and real estate agents—and some principles of bullshit detection that we can all apply to our lives.

Curtis Baker and Used Car Dealers

As an automobile hobbyist and enthusiast, most days Curtis Baker can be found working on an old car. I joined Baker while he refinished a 1995 Porsche 911 Carrera Cabriolet in his 60,000-square-foot warehouse. After discovering the car on CarGurus.com, he flew to Dallas, Texas, and drove it all the way home to Charlotte, North Carolina. I’ve never known anyone like Baker, who can work with tools, weld, talk, and eat spaghetti and meatballs, all at the same time. Baker is the perfect person to ask why car buyers are so often duped by used car dealer bullshit.

“I don’t think people are duped so much by direct bullshit from sellers. People are duped more by the facts sellers don’t share with them. And only someone who doesn’t really want to sell is going to tell you something that would make you avoid the car. The seller of that sweet 2006 Mercedes-Benz SL500 with 50,000 miles isn’t going to tell you that soon enough you’ll need to replace the hydraulic suspension that will cost you anywhere between $3,500 and $5,000. Dealers bullshit by omission.” Baker believes that information omission is orchestrated by both sellers and buyers. The seller is not readily forthcoming, but the buyer participates by not asking the right questions. He believes that if buyers can demonstrate to the seller that they know what they’re talking about by the questions they ask, there is considerably less room for the seller to bullshit.

What are the questions that Baker asks himself and sellers when he’s interested in buying a car? Baker explains, “I ask myself, if I was to buy this car, how could I drive it for free for the next two to three years?” Baker never buys a car he knows he can’t sell at a profit later. He points to a 2010 Toyota FJ Cruiser he bought last year for $16,000. “I will drive it for a couple of years, put miles on it, and still sell it for $20,000 without doing anything to it. Why? Because I’ve done my homework on the Toyota FJ Cruiser. Not only are they not the gas guzzlers you think they are, but after 2014, Toyota stopped making these cars. So every day there are fewer and fewer on the roads. And when I’m ready to sell it, there will be even fewer. And people absolutely love these SUVs, so the demand will remain steady, but the supply will continue to diminish.”

Baker is convinced that every car has a price that would enable you to drive it for free. “You really just need to do your homework so that no seller can bullshit you into buying something you don’t really want. Almost every car model has an online forum you can check out for free. Say you’re still interested in that 2001 BMW X5. Get on the online forum and see what people who have the model are complaining about. You’ll find out real quick which models have crap parts and when common problems are likely to emerge. Doing your homework will also place you in a better position to develop a strong BATNA (Best Alternative to a Negotiated Agreement). And when the dealer says to you, ‘You don’t have to be such an asshole,’ that’s when you know you’re getting a great deal.”

Whoever has a stronger BATNA—a better alternative—wins in negotiation. If you’ve done your homework and know everything there is to know about the make and model and invested some time to find multiple alternatives, then you’re likely to have a strong BATNA. Essentially, having a BATNA means having a viable plan B.

Don’t let dealers bullshit you into believing there is only one car for you. As the buyer, you have multiple dealers and cars to choose from. And with readily available alternatives from CarGurus, Auto-trader, and eBay in the game, you have hundreds if not thousands of cars to choose from. Baker believes that one of the easiest ways to develop a good BATNA is to connect with multiple car sellers and “fall in love with” at least two cars—or as many as you can. Falling in love with that special one and only is fine when it comes to people, but not when it comes to cars and properties. Baker explains, “Falling in love with only one car or one house or one job is almost guaranteed to make you a sucker. Why? Because if you’re so head over heels with a single option, you have no choice but to submit to details of the deal made by the single seller—you have to accept whatever terms they’re offering.”

If you give the illusion that you’ve been swept away by another car at another dealer, then you are building some leverage for your BATNA. You will feel much more comfortable pushing back on the original, higher priced deal when you have this leverage. At a minimum, your homework will teach you something about the market for the car you want and cultivate your confidence when you try to sound like you know what you are talking about. Although there are more sophisticated BATNA schemes, Baker claims there is no chance of driving a car for free without one.

When he has an eye on a car, Baker digs to discover its true value. That is, what are the total costs given the inevitable maintenance, risks, interest to be paid, insurance costs, and depreciation of the model? And what price would he likely get for the car when he’s ready to sell it? So many car buyers focus on the payments for four, six, or even seven years of financing. They focus too much on monthly payments and not enough on the true value and costs of the car. But as Baker explains, the right focus is to ask, “What are the objective facts concerning this car? What does the Carfax report say? If you see a collision on the Carfax report, you don’t want that car, because unless you’re planning on driving the car for the next ten years, you’re going to have trouble selling it with that red flag. Let’s say there was never any damage to the car, it has only 10,000 miles, and is only five years old, but has had seven different owners. The Carfax will tell you that, but you really don’t want it because it could be a hot rock and you don’t want to take a chance at holding it.”

Baker’s most useful recommendation for learning about a car is to ask the seller or dealer to take the car to a nearby, certified auto dealer to have a pre-purchase inspection done at your cost. Usually this runs about $350, but it’s worth it. The dealer will look at everything closely and tell you about any problems. If the car has no problems, great. If the car has some problems, even better. You can use those officially reported problems as leverage with the seller. A pre-purchase inspection might sound excessive if you’re buying from a trusted dealer, but you don’t want to pass up this level of investigating and questioning when purchasing a used car. As Baker claims, “I’ve seen BMW X5S with only 35,000 miles and the transmissions fall right out of them. Because I will tell you, that oil that you see in a small puddle under the car, the dealer will just tell you that they just changed the oil and spilled a bit. Bullshit. Get the pre-purchase inspection and thank me later.”

Before buying a car, Baker also likes to get a feel for whether the seller will consider an absurdly embarrassing offer. He argues that one never knows what kinds of pressures sellers or banks are under. “You might only have a 5% chance of success with the absurdly embarrassing offer, but you’ll get a hit 5 out of every too absurdly embarrassing offers, and that’s equity you just earned. I’m always amazed at how many people don’t do this. And if you’ve done your homework and can actually sound like you know what you’re talking about, you might just remind the dealer why the car might be otherwise difficult to sell. If you bring up the fact that you’ll need to spend at least $1,000 on four new tires when just one of the tires blows on that four-wheel-drive Cadillac Escalade, you can get the price reduced by $1,000 just like that.”

Finally, Baker adds, “I ask myself: Is this an emotional buy I’m considering? Would this be an impulse decision? The one thing I avoid at all costs is to make an emotional or impulse buy. Base your decision on facts and reason and not emotion. The only time I’ve ever lost money on a car was when I bought a car on emotion. I always loved the 1994 Pontiac Trans Am. It had white wheels, and I just had to have it. But I had a hell of a time getting rid of it when it came time to sell. Dealers know fully well that people buy cars for emotional reasons. That’s why they want you to test-drive it. See what it feels like to ride in it with the top down. Feel its horsepower. Smell the leather. Imagine yourself driving off with it today. They really just want you to fall in love with the car so that they have some leverage in your decision. They are more than ready to give you a load of bullshit because, despite the fact that they may refer to themselves as a Customer Sales Consultant, A-Team Representative, Sultan of Sale, Sir Close-a-Lot, Sir Come Sale Away, or Mr. Miracle Worker, they are car dealers and their livelihoods depend on selling as many cars as they can. Don’t forget that.”

Christina Pryce in the World of Real Estate

With over 25 years of experience in the Lake Norman area just north of Charlotte, North Carolina, and as a 16-time winner of the Five Star Real Estate Agent Award, Christina Pryce is as knowledgeable and trustworthy as real estate agents come. She also happens to be an expert bullshit detector. I know this firsthand, as both a buyer and seller. Her no-bullshit approach to real estate has literally saved me thousands of dollars.

One of the things that attracted me to Pryce is that she only holds five to six listings at any one time and doesn’t overwhelm herself with dozens of clients who are looking to buy a home. It comes as no surprise that 100% of her business is obtained by word of mouth referrals—she doesn’t advertise and has no web presence. This approach affords her time to study the market and fine tune her craft. As she notes, “I’m not a real estate appraiser, but I’ve taken an appraisal course three times. Things change over time in the real estate business.” She knows zoning laws and how to find appropriate houses for comparisons, as she explains, to “help” appraisers do their jobs. “People in this area often want to live on Lake Norman. When buyers ask if they could put a boat dock where there currently isn’t one, the vast majority of agents will say, ‘Sure thing, you can put a dock there.’ But most of the time they really don’t know what they are talking about. Lake Norman is a man-made lake built by Duke Energy to service their nuclear power plant—any docks must first be approved by Duke Energy, and their rules change all the time. It is heartbreaking to someone who buys a home on the lake only to discover they will never be approved for their very own dock. I avoid this altogether by doing my homework rather than bullshitting my clients.”

Her “spider sense” is founded on the recognition that bullshit is much more influential than most people realize. Pryce believes that appraising homes can be as subjective as valuing art. I find this somewhat disturbing. In an interesting study, cognitive psychologist Jonathan Fugelsang and his colleagues presented people with over 140 artist-or computer-generated abstract paintings and asked them to rate the profundity of each piece. Some of the paintings were tagged with art curator language known as International Art English (IAE). IAE is a style of communication commonly employed by artists and curators to discuss artwork. Rather than employing clear and concise language, IAE uses morphing verbs and turns adjectives into nouns (for example, potential to potentiality), pairs like terms (for example, internal psychology and external reality), and favors hard to picture spatial metaphors (for example, culmination of many small acts achieves mythic proportions).

In Fugelsang’s studies, abstract paintings included IAE titles such as “The Pathological Interior” or “Undefined Singularity of Pain,” mundane titles such as “Canvas No. 8” or “Color Mixing,” or no title at all. The title had a remarkable effect. Those paintings tagged with IAE titles were rated as more profound than paintings tagged with a mundane title or no title at all.

The very same attempts to influence perception through language can also be found in real estate. Savvy real estate agents never use words that might convey negative things about a home they are listing. Cozy, dollhouse, or cottage might sound cute and whimsical, but usually these words are code for “very small.” Custom or unique might characterize the pride held in a home, but potential buyers tend to think “weird corners,” “tacky colors,” and “wild additions” when they see these words. Modern, vintage, and rustic can mean that a property makes you feel like you are stepping back in time, but buyers hear “ridiculously outdated.” “TLC” or “lovingly maintained” may sound to sellers like the home has so much “potential,” but to buyers it signals that a property needs serious work. The worst is “buyers to verify permits,” which is code for “unpermitted additions or conversions.” Instead, your realtor will use code words like “beautiful,” “turnkey,” “spacious,” “backyard paradise,” “open floor plan,” “redeemed to perfection,” and “suite.” Every realtor knows that when it comes to writing real estate ads, a surefire approach is to “sell the sizzle, not the steak”—and what often emerges is bullshit.

What most sellers don’t understand is that all of the coded language is really for other real estate agents who represent the buyers and understand their expectations. In the business of real estate, coded language must be accurately applied. As Pryce explains, “I don’t pull any punches in how I describe a listing I have for sale. If another agent shows up with interested buyers to see my listings and they don’t see what they’ve come to expect from my description, that only damages my reputation—and reputation among real estate agents in the area is absolutely essential to a surviving and thriving real estate business.”

I was selling my home in the middle of the summer of 2014. Two weeks after listing the home on the market, our air conditioning unit died. It turned out to be a $2,500 repair. Little did I know, Pryce had already registered my home with a seller’s home warranty. The repair cost me $50 to register and $100 for a service fee, saving me $2,350. Pryce explains, “I do this automatically with any sellers I’m working with because no matter how new or old a home is, things can unexpectedly go wrong. People who have a 30-year-old home want to bloviate about how their home is like new and worth at least $20,000 more than it actually is. It isn’t. It’s 30 years old, and worth $20,000 less than you think it is—and no one’s 30-year-old home is like new—something will break before it’s sold.”

One of the most annoying tactics that Pryce encounters is when buyers carry on about their ability to buy a home. “When looking at a house, a potential buyer says, ‘Oh, this house is smaller than the one I’m in now. I could pay cash for this.’ And more often than not, it turns out he can’t, because a week later he’s telling me, ‘Well, interest rates are so low, I think I’ll get a mortgage.’ Just as it is with sellers, it is best to assume that, for whatever reason, buyers will somehow try to bullshit me.”

She also dislikes when other real estate agents bluster about their own credentials. “Reputation is everything in the real estate business. Any home valued at $400,000 or less is going to have multiple offers. Now the home seller decides which buyer they will work with, but agents can inevitably influence this decision. Agents working for the home seller do not want to work with someone who is inexperienced because the more experienced agent will be expected to do ‘everything’—and nobody wants that. So, what you have is agents who are new to the area, don’t really know what they are talking about, and exaggerate their experience and success. I just had another agent tell me he was doing so well in real estate that he was on pace to clear $20 million this year—and it was September. I thought to myself, ‘Well, you must expect to have a killer last quarter then, because I already looked you up on the multiple listing service that shows you only cleared $5 million.’ It is much better to be transparent with people you are dealing with.”

She also demands that her clients be clear-eyed and realistic when it comes to the process. “My very first question is ‘How are you going to buy this home?’ Because the reality is many people are not ready to buy a home—they think they are, but they really are not—and I’m not interested in just taking you, or anyone else, on a grand tour of Lake Norman. Once the ability to buy a home is established, then we can start working together. And next, I want to know what my clients really want. Do they want a home for the next five to ten years? Are they looking to make a profit? What is this potential home buy really all about?”

Even if her clients have decided that they will buy a home, deciding which one can be overwhelming. To help, Pryce suggests her clients do something research scientists are all too familiar with. She asks that they open an Excel spreadsheet. In the first column of the spreadsheet, they have to list the various features of a home they want to buy, including price, location, kitchen, bathrooms, size of master bedroom, flooring, ceilings, landscaping, neighborhood, school district, distance to work, even an X-factor they might not be able to fully describe. Let’s say they have 25 features that are important to them. They then decide which of the features are most and least important to them and list the features in descending order of importance. They assign each feature a weight, ranging from .00 to 1.00, to represent how much influence each feature should have in their final decision—they put that in the second column—and make sure the sum of the weights is equal to 1.00. They then add a column for each home on their short list. They rate each home on each of the features they’ve already identified they want to include in the decision using a one-to-five scale. They then multiply each rating by the feature’s respective weight. Finally, they sum each of the products. The home with the greatest weighted-sum score is declared the winner. Many judgment and decision-making experts refer to this method of decision-making as the weighted additive model, and it happens to be a very effective way of making decisions when multiple alternatives and features are available. It is comforting to know that some real estate agents are not relying solely on the bottom line to nudge their clients in one direction or another. Yet as a social psychologist on the lookout for bullshit, I’ll continue to rely on the Excel spreadsheet.

Lessons Learned on Bullshit Detection

Over my years of research, I have asked hundreds of people I consider to be expert bullshit detectors to explain why so many people are duped by bullshit in their professions and industries. What failures in critical thinking and questioning do they see? Each and every expert bullshit detector has reported to me the very same things:

When people have good information they typically make good decisions. Better information doesn’t always lead to better decision-making, but better decision-making almost always requires better information.
Failed bullshit detection is usually about things people don’t do and the questions they don’t ask. The good news is that we can learn from their approach. Here are some common habits that will help you weed out bullshit and make smarter decisions.

My ideal bullshit detector is Lieutenant Frank Columbo, played by Peter Falk in the 1970s television series Columbo. He was a homicide detective and famous for solving complicated “whodunit” murder mysteries by asking suspects “just one more thing.” This last question would always be the one that cracked the case. What does the Columbo critical thinking mindset look like in practice? We can list the basic habits of critical thinking as the following:

Having a passionate drive for clarity, precision, accuracy, relevance, consistency, logic, completeness, and fairness.
Having sensitivity to the ways in which critical thinking can be skewed by wishful thinking.
Being intellectually honest, acknowledging what they don’t know and recognizing their limitations.
Not pretending to know more than they do and ignoring their limitations.
Listening to opposing points of view with an open mind and welcoming criticisms of their beliefs and assumptions.
Basing beliefs on facts and evidence rather than on personal preference or self-interest.
Being aware of the biases and preconceptions that shape the way the world is perceived.
Thinking independently and not fearing disagreement with a group.
Getting to the heart of an issue or problem without being distracted by details.
Having the intellectual courage to face and assess ideas fairly even when they challenge basic beliefs.
Loving truth and being curious about a wide range of issues.
Persevering when encountering intellectual obstacles or difficulties.

Philosopher Peter Facione and the American Philosophical Association identified five critical thinking skills in the landmark 1990 Delphi Report: interpretation, analysis, evaluation, inference, and self-regulation. Each of these skills is essentially a different way of asking questions.

You are best able to detect bullshit when you are able to accurately interpret the claim. If you can answer the following questions, you can better understand the meaning and significance of a claim:

What does the claim mean? How is it meant to be understood?
Is there anything unclear, ambiguous, or not understood about the claim?
How can the claim be best characterized and classified?

An expert bullshit detector analyzes the arguments that could be made in support of and against a claim. Engaging in analysis involves asking these questions of the claim:

On what basis is the claim being made?
How does the individual know the claim is true?
What assumptions must be made to accept the claim and its conclusions as true?

When critical thinkers assess the logical strength of a claim, they engage in evaluation. They determine if the arguments and evidence for the claim justify the conclusions. Evaluative questions include:

How compelling is the evidence supporting the claim?
How well does the claim follow from a reasonable interpretation of the evidence?
Do the results of relevant investigations speak to the truth of the claim?

Expert bullshit detectors engage in inference, which occurs when the relevant information needed to draw reasonable conclusions is secured and connected to the implications of the claim’s truth. Inference is promoted when you can gain answers to questions like:

What does the evidence imply?
If the claim is true, what are the implications moving forward?
If major assumptions supporting the claim are abandoned, how does the claim’s truth stand?

Self-regulation involves assessing one’s own motivations and biases and asking whether these influence one’s interpretations, analyses, inferences, and evaluations of a claim. Self-regulation works best when engaging in metacognitive thought (thinking about one’s thoughts) by answering questions such as:

How good was my method in evaluating the claim?
Are my conclusions based on evidence and data, or are they based on anecdotal evidence or what I read in the news?
Is there anything I might be missing (or wanting to miss), and are my conclusions about the claim motivated by something other than the truth in any way?

Standards of Comparison, Reference Points, and Benchmarks

Expert bullshit detectors tend to see things differently than the rest of us and know things we don’t. Specifically, expert bullshit detectors recognize the usefulness of different standards of comparison, reference points, and benchmarks from the ones that everyone else uses. For instance, if you are asked to think about the concept of political leaders in general and then asked how to rate Bill Clinton, George W. Bush, and Barack Obama, you might rate them negatively. However, if you are asked to think about specific political leaders, like Adolf Hitler, Benito Mussolini, and Saddam Hussein and then rate Clinton, Bush, and Obama, you might rate them more positively. All judgments depend on standards of comparison and reference points.

Before purchasing a car, rather than focusing on specs and financing, Baker focuses on what will increase the likelihood he can sell it for a profit. When dealing with clients buying homes, rather than having her clients objectively assess their subjective preferences, Pryce encourages them to create weighted additive models. Experts do not focus on what everyone else is looking at. They tend to employ effortful, critical thinking within their areas of expertise, something that does not come naturally. Here’s an example from the annals of skepticism.

Since 2007, the annual number of terrorist incidents worldwide has averaged over 10,000, and injuries and deaths resulting from terrorist incidents have averaged more than 15,000 and 20,000, respectively. Most terrorist attacks have occurred outside the United States in places such as Syria, Iraq, Nigeria, Afghanistan, Pakistan, India, China, and Russia. The unsettling reality is that terrorist attacks can and do occur anywhere in the world. As a result, multiple efforts have been made to better detect this threat.

One such effort was Advanced Detection Equipment (ADE) 651. The ADE 651 was a handheld device designed to detect explosives by the British company Advanced Tactical Security and Communications (ATSC). It had a plastic handgrip with a swiveling antenna mounted on a hinge and it required no battery, as it was powered by static electricity. All that was required was for an operator to walk a few paces to “charge” the device, holding it at a right angle to their body. The antenna swiveled in the user’s hand and “tuned into the frequency” by pointing in the direction of an explosive.

The ADE appeared to be groundbreaking technology and an answer to detecting hidden terrorist explosives. With so much excitement about its potential, Iraq, Afghanistan, and 20 other countries in the Middle East and Asia entered into contracts to supply security forces with the detectors for a price of up to $60,000 each. ATSC profited [pounds sterling]50 million from sales of more than 7,000 units, and the Iraqi government alone is believed to have spent [pounds sterling]52 million on the devices. The ADE 651 was quickly adopted as a bomb sniffing device because it offered a fast solution to a difficult problem.

There was only one problem with the ADE 651—it didn’t do what ATSC’s founder, Jim McCormick, claimed it could do. In fact, the ADE 651 was a piece of junk produced in the 1990s by American car dealer, commercial diver, and treasure hunter Wade Quattle-baum, who originally promoted the device as the Quadro Tracker to find lost golf balls. With mountains of golf balls produced as “proof,” he reasoned that his device could also be marketed as a detector of other important things, like hidden stashes of marijuana, explosives, and weapons.

The Quadro Tracker allegedly worked by dowsing. But McCormick and the Iraqi military leaders who purchased the ADE 651 didn’t care enough about the readily available facts on dowsing. Apparently, no one put the ADE 651 through a simple diagnostic test that would have surely revealed its lousy performance. Major General Jihad al-Jabiri, head of Iraq’s bomb squad, said, “Whether it’s magic or scientific, what I care about is it detects bombs.” If only Jabiri had cared more about the evidence he could have saved his country millions of pounds. McCormick and Iraqi military leaders successfully promoted the belief that the tracker could find explosives.

One group that was concerned about the truth behind the ADE 651 was the FBI. In fact, in 1996, the FBI had already permanently barred dowsing devices from being manufactured or sold as bomb detectors, declaring them to be fakes. The FBI had already tested several devices like the ADE 651, and none had ever performed better than random chance. But the evidence wasn’t enough. By 2009, nearly every police checkpoint and many Iraqi military checkpoints had their own ADE 651.

Because of the military’s reliance on the ADE 651, suicide bombers managed to smuggle two tons of explosives into downtown Baghdad on October 25, 2009, and killed 155 people and destroyed three ministries. Video surveillance proved that the bombers would have had to pass at least one ADE checkpoint. Unfortunately, the lives lost that day were among hundreds of lives lost that could have been prevented had it not been for the investments in bullshit.

When the dust settled, McCormick was convicted of three counts of fraud, sentenced to ten years of imprisonment, and ordered to forfeit cash and assets worth nearly [pounds sterling]8 million. But the cost of Mc-Cormick’s bullshit was much greater. People lost their lives because his bomb detection devices didn’t work.

Life Without Bullshit

The science of detecting bullshit may not change society—but it can have a life-changing impact on you personally. By adopting a critical posture and the power of inquisitive questioning, you awaken the natural scientist and critical thinker inside yourself.

If enough people join the collective stand against bullshit, our world will become a very different place. We won’t have to listen to people talk about things they know nothing about. We won’t be exposed to baseless arguments. We won’t have to rely on incompetent people trying to do important jobs. Rather, we will collectively replace bullshit with evidence-based communication and reasoning, making more rational decisions based on facts, evidence, and reality.