7 Ways to Detect B.S. and Guru Syndrome

• 

Popular science and evidence-based self-help writing—both in books and articles on the internet—has grown astronomically over the past decade. On the one hand, this is great because it means that more people have access to fascinating and potentially helpful, even life-changing, ideas. On the other hand, this is not so great because so much of what passes for popular science and evidence-based self-help writing isn’t really scientific or evidence-based at all.

To help you separate signal from noise and know what to look out for when reading popular science and self-help, we’ve put together the following guide. If you come across any of the below, approach whatever it is you are reading with caution and a healthy dose of skepticism. 

1. Non-stop complexification.

Popular science and self-help should not sound like academic writing, because it’s not. If a writer is using broad, complex-sounding, and ambiguous words (e.g., toxins, detox, bio-engineering) it could be a sign that they may not have much of substance to say. Also, look out for lots of neuroscience terms, especially if the writer isn’t a neuroscientist. Very few people know a lot about the brain; and technically the brain is involved in everything we perceive as reality, so it’s an easy way to dress up baloney and make it sound scientific. Finally, someone who is an expert on something should be able to describe it in fairly simple terms. Complexity is easy to hide one’s lack of knowledge behind while trying to sound smart.

 2. The writer is selling something or trying to develop a specific brand.

Anytime a writer is selling a specific product, diet, service, program, or some other “proprietary approach” you should read with a very skeptical eye, if you read at all. Science is supposed to be dispassionate. Yes, any good writer wants their ideas to spread. But there’s a big difference between wanting to spread one’s ideas and wanting to build a copyrighted brand based on them to the exclusion of other ideas. 

3. Precise and narrow interventions with massive impacts.

Anytime you read about a single intervention with super wide-ranging effects you should be skeptical. Here’s a metaphor: The flu shot works for the flu. The flu shot does not work for every single illness. Saying some new behavioral concept or approach can be perfectly applied to tons of things and make a big difference is usually overreaching. (The one exception being exercise, which really does help with just about everything.)

4. The story is the main driver.

Humans learn through stories, so deploying stories in science writing is a good idea. As a matter of fact, great writing should have great stories. That said, the story should represent the science, not the other way around. Two of the best and top-selling non-fiction writers in the world are master storytellers: Malcolm Gladwell and Michael Lewis. Their books are great. But their books should be read as entertainment and ways to challenge one’s thinking and come up with new ideas. They shouldn’t be read as lessons that will work for many people in many situations. Gladwell himself says as much.

5. There is no works-cited.

This is immediate cause for concern. If you are making an argument with research-backed claims, you need to provide that backing and open yourself up to criticism, especially if your argument gains its authority from being scientific or evidence-based.

6. Attempting to create an artificial “us versus them” mentality.

Unfortunately, conflict sells. But if someone is using conflict and tribalism to sell their idea, then the idea probably isn’t strong enough to stand on its own. You see this all the time in writing about diets, fitness, and behavioral change. If the method is strong and probably true, the writer shouldn’t need to fabricate battles to sell it. 

7. Guru syndrome.

If a writer lacks humility, never addresses how they might be wrong, and reaches far beyond their expertise in providing answers for everything, it’s probably best to stop reading their work. 

As for what to look for when reading popular science, we like to think of a three-legged stool:

1. Is the idea backed by a strong theory? Does the theory make sense in simple terms?
2. Is the idea supported by empirical evidence, and not just a few small studies but lots of studies, and ideally large ones?
3. Does the idea (or variations of it) appear in different contexts throughout history and in modern practice? Are there solid patterns that emerge? 

If you can answer yes to all three of these questions, then the idea you are reading about is probably true with a capital T.

We hope this helps you develop your B.S. detector. We’d love to hear any other methods you use. Reach out to us on twitter @Bstulberg and @Stevemagness. Thanks for reading.

— Brad and Steve