Untangling Success: Does the Cream Rise to the Top?

“The cream rises to the top.”

Our world is littered with aphorisms and sayings that all point to the same message: the quality of our work and our chances of success are deeply intertwined. If we produce good work it’ll catch on, the story goes, and success will soon follow.

Yet connection between quality work and success is a bit more tenuous than we realize.

A few years back,  Duncan Watts and colleagues  at Columbia tested the influence of quality on success in the music world. They took over 14,000 participants and had them listen to a slew of previously unheard songs. With each listen, participants would rate the song from 1 to 5 and be given the option to download it.

Here’s the twist: while one group of participants could see the download numbers, indicating popularity, another group was blind to it. They had to rate the song without any clue of what other people thought. This allowed the researchers to see the impact of social popularity on the ratings and the subsequent impulse to download a song.

What they found is that when downloads were displayed:

1. Popular songs become more popular and unpopular songs became less popular. In other words, inequality accelerated with social proof.

2. Success became less predictable. The correlation between the quality ranking in the independent/without social popularity group and the other group that included the popularity information ceased to correlate: A song that ranked high in the independent group could be very unpopular in the social influence group.

The researchers main conclusion was this: while quality generally raised the floor, ensuring that a song wasn’t the lowest downloaded among the group, beyond that, quality didn’t correlate to success. As the researchers summarized, “In general, the best songs never do very badly, and the worst songs never do extremely well, but almost any other result is possible.”

If we step outside of the research world and into the real one, we see the same effect. One of the clearest examples is when famous authors take on a pen name. After Stephen King became a literary star with the release of The Shining and Salem’s Lot, he adopted the pen name of Richard Bachman.

During the 1970s and 80s, it was almost sacrilegious to put out more than one book a year. Publishers thought it would saturate the market and hurt sales. But King was a writing machine. The solution? King would put out work under Bachman’s name.

Under Bachman, King’s books did well enough. For example, Thinner sold 28,000 copies before King was outed as Bachman. Those sales numbers are strong and consistent with someone who could make a modest living as a writer. But they pale in comparison to a traditional King book, or even the amount of copies it sold once King was unmasked. More recently, we saw the same with another star writer, JK Rowling. Before her pen name, Robert Galbraith, was exposed, her first book sold only about 8,500 copies in a few months.

What’s the takeaway?

Too often, we equate conventional success with quality when in reality there is a lot of luck involved. Sure, you can influence your luck by marketing a book well or trying to get as many reviews as possible to take advantage of the download effect we saw in the music experiment. But in the end, success still requires a fair share of luck.

This research serves as a reminder of why we shouldn’t attach to an outcome. If some of the most prolific and quality writers in the world can only sell a few thousand copies based on the quality of their writing alone, then it’s a message to not let our successes or failures become self-defining.