Online Music Testing: Good or Bad?

Ask the Research Doctor a question.

Moderators: shawnski, jdenver, rogerwimmer

Forum rules
How do you ask the Research Doctor a question?

Just click on "New Topic" below and then type your question to me. Please put in a Subject and then ask your question in the body.

Your question is submitted annonymously unless you include your name.

Click "Submit" and then check back to find Dr. Wimmer's answer to your question.

If you wish to have a private answer to a question (not posted in the forum), please include your email address. Dr. Wimmer will send your answer directly to you.

Dr. Wimmer maintains an Archive of questions from the forum. Go to and click on "The Research Doctor Archive" link.
Post Reply
User avatar
Posts: 197
Joined: Mon Sep 28, 2009 12:40 pm

Online Music Testing: Good or Bad?

Post by rogerwimmer » Tue Apr 27, 2010 3:30 am

Hi Roger: Online music testing using a station's database as the sample seems to becoming more the norm these days. Besides the expense savings, when I ask people why they use this methodology versus callout, the answer I hear over and over is, "The results look about the same." If playing the right songs is as critical to PPM success as Arbitron stresses, would you put finding the right songs in the hands of online testing using your database? Or is it close enough as a lot of PDs are saying?

Thank you for your time. - Anonymous

Anon: I have answered similar questions about online music testing before, so please go to this page on The Research Doctor Archive and scroll down to the question titled, "Online Research." (Be prepared for a long answer.) In addition, there are a few more questions/answers on the next page of the Archive.

Finally, for information about checking a sample, read my answer to the question "Music Tests & Small Samples" that is on the column below, or click here for the question.

If you read all that material, you'll know that I don't have a major problem with online music tests if the sample can be verified and if procedures are used to eliminate respondents who don't belong in the sample.

However, the major problem you identify is when you said that what you repeatedly hear is that the results from callout (and auditorium tests) " . . . look about the same." I don’t know what that means. The results LOOK the same? "Looking the same" doesn't mean the results are the same. The only way to determine if the results are similar is to conduct a statistical test. Simply looking at the data is not a valid and reliable test. I'll be happy to review a statistical test that compares the music testing procedures, but I haven't seen anything yet. What I hear is the same thing you hear, "The results are about the same." That's meaningless to me.

Side Note: I wonder if the people who say that the results from different music test approaches "look the same" would agree with me if I said that a "Corvette looks the same as a Volkswagen Beetle." (They both have four wheels, a motor, headlights, etc.)

(Want to comment on this question? Click on the POSTREPLY button under the question.)
Roger Wimmer is owner of Wimmer Research and senior author of Mass Media Research: An Introduction, 10th Edition.

Post Reply