After wondering for years which of the big music streaming services truly sounds the best, I decided to take matters into my own hands (I mean, EARS!) and do a SCIENTIFIC audio quality test. This is NOT a subjective test where I or somebody else listened and I judged what they thought might be better or worse. This is COLD HARD DATA, to be precise KiloBits Per Seconds (kbps) of data transfer and MegaBytes (Mb) of downloads. But fear not!!! I will keep the results super easy to understand and you won’t need a degree in anything to know who to pick. I can’t however promise you that you won’t be disappointed when finding out how these streaming companies have been lying to us all along about the sound quality they promise. NO MORE! With actual and factual numbers at hand I will tell you who sounds best and who delivers on their promises and you will be able to make an informed decision knowing that you are supporting the company that supports the BEST SOUNDING audio and that you are not paying full price for half the quality!

By the way, I plan to do a lot more of these tests, so if you are interested in findings such as these or in the music industry in general, make sure you follow me on Twitter at @marcurselli or at facebook.com/marcurselli

Without further ado, here we go…

The Problem

I am still very much torn on whether to support streaming music services at all or not. Although I do of course support the idea of paying for music, the current payment distribution methods employed by these streaming services are NOT based on actual play counts and do NOT serve (or even fairly compensate) independent artists. These artists therefore absolutely cannot rely on income from these services (more on that at the end of this article) and a lot of even famous artists (like Radiohead’s Thom Yorke, David Byrne, John Zorn, AC/DC, Led Zeppelin and many others) understood that and took steps to get their music off of streaming services. Should you want to take such steps you will have to rely on a company such as Muso for example.

This said, in these times when everyone is streaming music from the cloud, if you are a music lover, it is almost impossible not to rely on one of these services for music discovery and because my no.1 concern when it comes to online music streaming is THE QUALITY I decided to set up the THE ULTIMATE AUDIO QUALITY TEST to find out which of these services actually deliver the BEST SOUNDING music. Period.

Back when there was only Spotify and MOG to choose from, I myself chose MOG because they were the only company that even bothered to mentioned streaming quality and they always claimed that they streamed at the highest quality. I have been a MOG user for a few years now. At that same time I had approached Rdio and asked about streaming quality. They claimed they were “streaming music at CD-quality, and only do lower bit rates over 3G” but when I asked for a more technical answer all I got was “I’m sorry, but we’re not sharing this information”. I honestly do not understand why a company that promises a quality service would not provide customers with all the information about the service and the quality they are promising. Usually corporate secrecy is synonymous with wrongdoing and this lack of transparency was a huge turn off for me. So I passed on Rdio.

Fast forward to 2014. MOG has been acquired by Dr. Dre’s Beats Music and is going to shut down on April 14th. The new Beats Music is already available and is getting good reviews because of its curatorial aspect, but I was never a fan of the Beats headphones (although I have not tested all the high-end models) and if that was their idea of good sound quality I was afraid I was going to be disappointed with their streaming quality. I went ahead and tested all the most popular music streaming services available today, which include the ever popular Spotify, the disappearing MOG, the new Beats Music, the underdog Rdio and the relatively new kid on the block Play Music (by Google). I have decided not to include iTunes Radio, Rhapsody or Pandora because they are the kind of apps that don’t allow you to listen to a specific song or artist (you can only create stations based around an artist).

This test will ONLY and specifically be about QUALITY of audio, not about user interface, number of songs or other factors. There are plenty of reviews out there for that and I’ve added links to some of those at the bottom of this page for your convenience.

The Method 

In order to ensure realistic and reproducible results I tried to be as scientific as I could about this.
Like so:

• I streamed the same exact song on all of the services.
• the song I chose to stream is a song of mine from an old 1995 album by my old italian electro-industrial band The M.E.M.O.R.Y. Lab that was re-released on CD in 2009 and the reason I chose something of mine is because I own the master and therefore I know EXACTLY how big the file is that we are streaming (whereas if I were to choose some famous artist’s song I wouldn’t know how big the original file actually is!).
• The song I chose had to be available on all these services and I chose the first song off the record (the song is called “Change (Cambia)”)
• For the test over the LTE Cellular Data Network (although findings should be the same for 4G, 3G or 2G networks). The original WAV file is 66.3Mb and a 320kbps MP3 version of it is 15Mb.
• For the WiFi test I chose a different song because I wanted to make sure that the previous song wasn’t in some way cached by the device or the app and therefore altered the results of the test. The song I chose for the WiFi test is the fourth song off the record and it is called “Mind Rape (Government’s Business)” and the original file was 52.1Mb while the 320kbps MP3 version of that is 11.8Mb.
• Needless to say I never played these songs on these apps on the iPhone before so they were in no way cached on the device or these apps.
• I set ALL 5 of the AT&T iPhone apps of these streaming services to the highest streaming quality settings they have available (see pictures below) and I streamed the songs over the AT&T LTE network and over my home WiFi network without ever-moving the phone around (it was just sitting on my desk) in order not to affect network reception in any way.
• I searched for the artist in the app, then clicked on the album and then (that’s the cool part) I double clicked the home button on the iPhone to go to the Settings > Cellular > Cellular Usage page (as described here) and reset the Cellular Data Usage so that it would show zero kb.
• At this point, for every app, I double-clicked the home button again, went to the app and clicked on the song to start playing it.
• At the end of the song I would switch back to the Cellular Usage page and take a screenshot of the Cellular Usage page to compile my findings.
• For the test over WiFi this method does not work because the Cellular Usage page doesn’t record data usage over WiFi (since, as the name implies, it is not cellular) so I downloaded the free iOS app Data Monitor by Nutec Apps LLC  which monitors WiFi traffic.
• The last test I was interested in performing was the audio quality of the downloads made available for offline listening (a feature that all of these apps provide and call different things like Offline, Sync to App etc and which allows you to listen to music when you have no network coverage and no WiFi). For this test I have downloaded the full 8 track album from which the two above mentioned songs come from. The album is called “Modern Expressing Machines Of Revolutionary Youth” and by looking at the original files I can tell you that the uncompressed WAV files for the CD release were 455.4Mb, the 320kbps MP3 version was 103.3Mb and the 192kbps version was 62Mb. Some interesting findings there, you’ll see.

Beats Music	MOG	Play Music	Rdio	Spotify

The Results

Interpreting these results was not as easy as I expected and at times it is downright shocking!!! Because the numbers are so different it is safe to assume to that these apps all apply their own degree of compression to the music before streaming it. In these images and reports below, higher numbers mean higher quality so it would appear that Google’s Play Music and MOG have the best quality while Beats and Rdio have the worst.

AT&T LTE network in Manhattan
Beats Music 13.2Mb (slightly lower quality than 320kbps MP3 – the loser!)
MOG 24.4Mb (much higher quality than 320kbps MP3)
Play Music 25.1Mb (much higher quality than 320kbps MP3 – the winner!)
Rdio 13.6Mb (slightly lower quality than 320kbps MP3)
Spotify 17.2Mb (higher quality than 320kbps MP3)

Beats Music	MOG	Play Music	Rdio	Spotify

WiFi network in my home in Manhattan
Beats Music 54.70Mb  (highest quality – REAL CD quality – the winner of the award “the only company who didn’t lie to us”!)
MOG 39.18Mb
Play Music 33.19Mb
Rdio 18.59Mb
Spotify 12.15Mb

Beats Music	MOG	Play Music	Rdio	Spotify

Full album Download (over WiFi) for Offline listening
Beats Music 35.8Mb (approximately 96kbps MP3 quality)
MOG 103Mb (equivalent to 320kbps MP3 quality – as advertised – the winner!)
Play Music 64Mb (equivalent to 192kbps MP3 quality)
Rdio 67.6Mb (equivalent to 192kbps MP3 quality)
Spotify 73.3Mb (slightly higher than 192kbps MP3 quality but still lower than 320kbps – the runner-up and de facto winner since MOG will be disappearing soon)

Beats Music	MOG	Play Music	Rdio	Spotify

The Conclusion

Google’s Play Music service is the winner when it comes to streaming music quality over cellular network.
Beats Music is the clear winner when it comes to streaming music quality over WiFi. My hat goes off to them for being the ONLY ones who really deliver the promised CD-quality music, since basically they stream through an uncompressed WAV file!
MOG is the winner when it comes to high quality downloads over WiFi but since MOG will be disappearing April 15th, the runner up and de facto winner is Spotify, in this category.

It is important to notice that MOST OF THESE COMPANIES HAVE BEEN LYING TO OUR FACES THIS WHOLE TIME! Most of them claim to be giving you CD quality music, but guess what, the only one who does so is Beats Music and ONLY when you are on WiFi, which still leaves all of us listening to MP3 quality music, and NOT the promised CD quality music. Over Cellular Data the only ones that streams at 320kbps (which is the next best thing to true and pure CD quality) are Play Music by Google and MOG. And surprisingly NOT EVEN when you proceed to download a full album for offline listening you get true CD quality (except for MOG, for what that’s worth).

At this point it is important to consider what use you will be making of the app. If you are going to be doing a lot of streaming (like driving around listening to new music etc) then you should choose the service that offers the highest quality streaming over cellular, but if you are the kind of person who pre-downloads all your music to listen to it offline in planes or underground subway trains you should go for the app that offers the highest quality streaming over WiFi (since you’ll most likely be doing the downloading at home/work over WiFi).

I am happy to have supported MOG from the beginning until now however, surprisingly (or maybe not) it seems that the acquisition on the part of Beats has degraded the quality of their streaming over cellular. Therefore I will not be transferring my monthly subscription over to Beats when on April 15th MOG shuts down completely. Now that I have to choose a new service I will start with giving Google‘s Play Music a try because it seems obvious from the numbers that they are streaming music at the highest quality (if you take MOG out of the equation you will notice that Play Music has the most consistently high numbers on average).
There are also in my opinion a few other added advantages to choosing a colossus such as Google:

• they will never get bought by somebody else, as it just happened with MOG
• if anyone has the economic might to acquire more content and make it more widely available, it is Google!
• if their “Don’t Be Evil” corporate motto can give us any hope, then my hope is that they will find a way to fairly and equally compensate ALL content creators, and not just the ones who are signed by the 5 majors.

If you don’t really care about audio quality (as sadly most don’t) or you are one of those people who say that they do but then they listen to music with the $5 white ear buds included with the iPhone or iPod by Apple, you might be more interested in the Facebook integration offered by Spotify or the intriguing curatorial aspect (and true CD quality streams over WiFi) offered by Beats Music, but Google achieved the highest average score for quality on this test and my TOP priority is QUALITY. Therefore I will give Google‘s Play Music a spin, mostly because I use music streaming services as a discovery tool. As most New Yorkers, I don’t own a car so I won’t be streaming music on the go very often. More often than not I end up downloading full albums overnight at home over WiFi, then I listen to them on the NYC underground subway trains on my way to the recording studio or on planes on my way to a gig and then I delete those records after one or two quick listenings to make room for new ones. I do this because I try to keep up with what is new out there and keep my finger on the pulse, but, when I do listen to music, I want to be able to listen to the most pure, unadulterated and uncompromising quality I can get. After all, there is a reason I spent almost $300 dollars on my in-ear phones, but that’s another story…

Links to Articles with Feature Comparisons of these Services

http://www.imore.com/on-demand-streaming-music-services-compared-spotify-rdio-beats-music-slacker-google-play-music

http://venturebeat.com/2011/09/17/top-5-streaming-music-services-spotify-mog-rdio

 http://gizmodo.com/5750415/the-best-streaming-music-service

http://www.forbes.com/sites/amadoudiallo/2013/09/24/itunes-streaming-radio-versus-pandora-and-spotify

 http://www.cultofmac.com/265655/winner-best-music-demand-streaming-service 

Links to Articles about Why the Streaming Music Service model is Broken

http://source.nationwidedisc.com/spotify-pandora-and-streaming-music-should-you-post-your-music

http://business.time.com/2013/12/03/heres-how-much-money-top-musicians-are-making-on-spotify

 http://www.theguardian.com/technology/2013/aug/19/zoe-keating-spotify-streaming-royalties

http://readwrite.com/2013/12/06/streaming-music-competition-pandora-rdio-spotify

http://www.digitalmusicnews.com/permalink/2014/02/21/favoritepays

http://www.digitalmusicnews.com/permalink/2013/12/13/quicksummarystreaming