Programmers who base music choices on research and data feel comfortable and confident in the decisions they make. They reason feedback from real listeners is more reliable than good old fashioned “gut”. But there is a flaw in the logic, especially if the information is gathered through a research sample from online music clubs.

Since online music research debuted in the early 2000s, programmer acceptance has evolved from resistance to acceptance to passionate proponents. But over the years, response rates to online music research has declined. In many cases, dramatically.

Don’t get me wrong. I love online research as a tactic. It works. In 2002, I conducted one of the very first online auditorium tests, recruiting over 20,000…yes, twenty thousand…respondents to engage in a 700 song test.

Today, the technology is even better. So testing songs via online testing should be even more relevant. Right? Not so fast.

Online Music Tests Attract Fewer Respondents Today