After wondering for years which of the big music streaming services truly sounds the best, I decided to take matters into my own hands (I mean, EARS!) and do a SCIENTIFIC audio quality test. This is NOT a subjective test where I or somebody else listened and I judged what they thought might be better or worse. This is COLD HARD DATA, to be precise KiloBits Per Seconds (kbps) of data transfer and MegaBytes (Mb) of downloads. But fear not!!! I will keep the results super easy to understand and you won’t need a degree in anything to know who to pick. I can’t however promise you that you won’t be disappointed when finding out how these streaming companies have been lying to us all along about the sound quality they promise. NO MORE! With actual and factual numbers at hand I will tell you who sounds best and who delivers on their promises and you will be able to make an informed decision knowing that you are supporting the company that supports the BEST SOUNDING audio and that you are not paying full price for half the quality!
By the way, I plan to do a lot more of these tests, so if you are interested in findings such as these or in the music industry in general, make sure you follow me on Twitter at @marcurselli or at facebook.com/marcurselli
Without further ado, here we go…
The Problem
I am still very much torn on whether to support streaming music services at all or not. Although I do of course support the idea of paying for music, the current payment distribution methods employed by these streaming services are NOT based on actual play counts and do NOT serve (or even fairly compensate) independent artists. These artists therefore absolutely cannot rely on income from these services (more on that at the end of this article) and a lot of even famous artists (like Radiohead’s Thom Yorke, David Byrne, John Zorn, AC/DC, Led Zeppelin and many others) understood that and took steps to get their music off of streaming services. Should you want to take such steps you will have to rely on a company such as Muso for example.
This said, in these times when everyone is streaming music from the cloud, if you are a music lover, it is almost impossible not to rely on one of these services for music discovery and because my no.1 concern when it comes to online music streaming is THE QUALITY I decided to set up the THE ULTIMATE AUDIO QUALITY TEST to find out which of these services actually deliver the BEST SOUNDING music. Period.
Back when there was only Spotify and MOG to choose from, I myself chose MOG because they were the only company that even bothered to mentioned streaming quality and they always claimed that they streamed at the highest quality. I have been a MOG user for a few years now. At that same time I had approached Rdio and asked about streaming quality. They claimed they were “streaming music at CD-quality, and only do lower bit rates over 3G” but when I asked for a more technical answer all I got was “I’m sorry, but we’re not sharing this information”. I honestly do not understand why a company that promises a quality service would not provide customers with all the information about the service and the quality they are promising. Usually corporate secrecy is synonymous with wrongdoing and this lack of transparency was a huge turn off for me. So I passed on Rdio.
Fast forward to 2014. MOG has been acquired by Dr. Dre’s Beats Music and is going to shut down on April 14th. The new Beats Music is already available and is getting good reviews because of its curatorial aspect, but I was never a fan of the Beats headphones (although I have not tested all the high-end models) and if that was their idea of good sound quality I was afraid I was going to be disappointed with their streaming quality. I went ahead and tested all the most popular music streaming services available today, which include the ever popular Spotify, the disappearing MOG, the new Beats Music, the underdog Rdio and the relatively new kid on the block Play Music (by Google). I have decided not to include iTunes Radio, Rhapsody or Pandora because they are the kind of apps that don’t allow you to listen to a specific song or artist (you can only create stations based around an artist).
This test will ONLY and specifically be about QUALITY of audio, not about user interface, number of songs or other factors. There are plenty of reviews out there for that and I’ve added links to some of those at the bottom of this page for your convenience.
The Method 
In order to ensure realistic and reproducible results I tried to be as scientific as I could about this.
Like so:
- I streamed the same exact song on all of the services.
- the song I chose to stream is a song of mine from an old 1995 album by my old italian electro-industrial band The M.E.M.O.R.Y. Lab that was re-released on CD in 2009 and the reason I chose something of mine is because I own the master and therefore I know EXACTLY how big the file is that we are streaming (whereas if I were to choose some famous artist’s song I wouldn’t know how big the original file actually is!).
- The song I chose had to be available on all these services and I chose the first song off the record (the song is called “Change (Cambia)”)
- For the test over the LTE Cellular Data Network (although findings should be the same for 4G, 3G or 2G networks). The original WAV file is 66.3Mb and a 320kbps MP3 version of it is 15Mb.
- For the WiFi test I chose a different song because I wanted to make sure that the previous song wasn’t in some way cached by the device or the app and therefore altered the results of the test. The song I chose for the WiFi test is the fourth song off the record and it is called “Mind Rape (Government’s Business)” and the original file was 52.1Mb while the 320kbps MP3 version of that is 11.8Mb.
- Needless to say I never played these songs on these apps on the iPhone before so they were in no way cached on the device or these apps.
- I set ALL 5 of the AT&T iPhone apps of these streaming services to the highest streaming quality settings they have available (see pictures below) and I streamed the songs over the AT&T LTE network and over my home WiFi network without ever-moving the phone around (it was just sitting on my desk) in order not to affect network reception in any way.
- I searched for the artist in the app, then clicked on the album and then (that’s the cool part) I double clicked the home button on the iPhone to go to the Settings > Cellular > Cellular Usage page (as described here) and reset the Cellular Data Usage so that it would show zero kb.
- At this point, for every app, I double-clicked the home button again, went to the app and clicked on the song to start playing it.
- At the end of the song I would switch back to the Cellular Usage page and take a screenshot of the Cellular Usage page to compile my findings.
- For the test over WiFi this method does not work because the Cellular Usage page doesn’t record data usage over WiFi (since, as the name implies, it is not cellular) so I downloaded the free iOS app Data Monitor by Nutec Apps LLC Â which monitors WiFi traffic.
- The last test I was interested in performing was the audio quality of the downloads made available for offline listening (a feature that all of these apps provide and call different things like Offline, Sync to App etc and which allows you to listen to music when you have no network coverage and no WiFi). For this test I have downloaded the full 8 track album from which the two above mentioned songs come from. The album is called “Modern Expressing Machines Of Revolutionary Youth” and by looking at the original files I can tell you that the uncompressed WAV files for the CD release were 455.4Mb, the 320kbps MP3 version was 103.3Mb and the 192kbps version was 62Mb. Some interesting findings there, you’ll see.
| Beats Music | MOG | Play Music | Rdio | Spotify |
 |  |  |  |  |
 |  |  |  |  |
The Results
Interpreting these results was not as easy as I expected and at times it is downright shocking!!! Because the numbers are so different it is safe to assume to that these apps all apply their own degree of compression to the music before streaming it. In these images and reports below, higher numbers mean higher quality so it would appear that Google’s Play Music and MOG have the best quality while Beats and Rdio have the worst.
AT&T LTE network in Manhattan
Beats Music 13.2Mb (slightly lower quality than 320kbps MP3 – the loser!)
MOG 24.4Mb (much higher quality than 320kbps MP3)
Play Music 25.1Mb (much higher quality than 320kbps MP3 – the winner!)
Rdio 13.6Mb (slightly lower quality than 320kbps MP3)
Spotify 17.2Mb (higher quality than 320kbps MP3)
| Beats Music | MOG | Play Music | Rdio | Spotify |
 |  |  |  |  |
WiFi network in my home in Manhattan
Beats Music 54.70Mb  (highest quality – REAL CD quality – the winner of the award “the only company who didn’t lie to us”!)
MOGÂ 39.18Mb
Play Music 33.19Mb
Rdio 18.59Mb
Spotify 12.15Mb
| Beats Music | MOG | Play Music | Rdio | Spotify |
 |  |  |  |  |
Full album Download (over WiFi) for Offline listening
Beats Music 35.8Mb (approximately 96kbps MP3 quality)
MOGÂ 103Mb (equivalent to 320kbps MP3 quality – as advertised – the winner!)
Play Music 64Mb (equivalent to 192kbps MP3 quality)
Rdio 67.6Mb (equivalent to 192kbps MP3 quality)
Spotify 73.3Mb (slightly higher than 192kbps MP3 quality but still lower than 320kbps – the runner-up and de facto winner since MOG will be disappearing soon)
| Beats Music | MOG | Play Music | Rdio | Spotify |
 |  |  |  |  |
The Conclusion
Google’s Play Music service is the winner when it comes to streaming music quality over cellular network.
Beats Music is the clear winner when it comes to streaming music quality over WiFi. My hat goes off to them for being the ONLY ones who really deliver the promised CD-quality music, since basically they stream through an uncompressed WAV file!
MOG is the winner when it comes to high quality downloads over WiFi but since MOG will be disappearing April 15th, the runner up and de facto winner is Spotify, in this category.
It is important to notice that MOST OF THESE COMPANIES HAVE BEEN LYING TO OUR FACES THIS WHOLE TIME! Most of them claim to be giving you CD quality music, but guess what, the only one who does so is Beats Music and ONLY when you are on WiFi, which still leaves all of us listening to MP3 quality music, and NOT the promised CD quality music. Over Cellular Data the only ones that streams at 320kbps (which is the next best thing to true and pure CD quality) are Play Music by Google and MOG. And surprisingly NOT EVEN when you proceed to download a full album for offline listening you get true CD quality (except for MOG, for what that’s worth).
At this point it is important to consider what use you will be making of the app. If you are going to be doing a lot of streaming (like driving around listening to new music etc) then you should choose the service that offers the highest quality streaming over cellular, but if you are the kind of person who pre-downloads all your music to listen to it offline in planes or underground subway trains you should go for the app that offers the highest quality streaming over WiFi (since you’ll most likely be doing the downloading at home/work over WiFi).
I am happy to have supported MOG from the beginning until now however, surprisingly (or maybe not) it seems that the acquisition on the part of Beats has degraded the quality of their streaming over cellular. Therefore I will not be transferring my monthly subscription over to Beats when on April 15th MOG shuts down completely. Now that I have to choose a new service I will start with giving Google‘s Play Music a try because it seems obvious from the numbers that they are streaming music at the highest quality (if you take MOG out of the equation you will notice that Play Music has the most consistently high numbers on average).
There are also in my opinion a few other added advantages to choosing a colossus such as Google:
- they will never get bought by somebody else, as it just happened with MOG
- if anyone has the economic might to acquire more content and make it more widely available, it is Google!
- if their “Don’t Be Evil” corporate motto can give us any hope, then my hope is that they will find a way to fairly and equally compensate ALL content creators, and not just the ones who are signed by the 5 majors.
If you don’t really care about audio quality (as sadly most don’t) or you are one of those people who say that they do but then they listen to music with the $5 white ear buds included with the iPhone or iPod by Apple, you might be more interested in the Facebook integration offered by Spotify or the intriguing curatorial aspect (and true CD quality streams over WiFi) offered by Beats Music, but Google achieved the highest average score for quality on this test and my TOP priority is QUALITY. Therefore I will give Google‘s Play Music a spin, mostly because I use music streaming services as a discovery tool. As most New Yorkers, I don’t own a car so I won’t be streaming music on the go very often. More often than not I end up downloading full albums overnight at home over WiFi, then I listen to them on the NYC underground subway trains on my way to the recording studio or on planes on my way to a gig and then I delete those records after one or two quick listenings to make room for new ones. I do this because I try to keep up with what is new out there and keep my finger on the pulse, but, when I do listen to music, I want to be able to listen to the most pure, unadulterated and uncompromising quality I can get. After all, there is a reason I spent almost $300 dollars on my in-ear phones, but that’s another story…
Links to Articles with Feature Comparisons of these Services
http://venturebeat.com/2011/09/17/top-5-streaming-music-services-spotify-mog-rdio
 http://gizmodo.com/5750415/the-best-streaming-music-service
 http://www.cultofmac.com/265655/winner-best-music-demand-streaming-serviceÂ
Links to Articles about Why the Streaming Music Service model is Broken
http://source.nationwidedisc.com/spotify-pandora-and-streaming-music-should-you-post-your-music
http://business.time.com/2013/12/03/heres-how-much-money-top-musicians-are-making-on-spotify
 http://www.theguardian.com/technology/2013/aug/19/zoe-keating-spotify-streaming-royalties
http://readwrite.com/2013/12/06/streaming-music-competition-pandora-rdio-spotify
http://www.digitalmusicnews.com/permalink/2014/02/21/favoritepays
http://www.digitalmusicnews.com/permalink/2013/12/13/quicksummarystreaming
Xbox Music?
Tim
Hello Tim, thanks for bringing Xbox Music to my attention. I was not aware of it but it seems you can only create radio stations (like Pandora) and not stream what you want, right?
Also it seems Spotify acquired Xbox so it will probably disappear shortly, or at least I wouldn’t be surprised if it just became Spotify.
Hi Marc
Actually it is a full streaming service (play what you want) and is owned by Microsoft not Spotify so I would be interested in the comparison.
Tim
I see.
As soon as I have time I will have to test Xbox and Rhapsody and add it to the article…
The acquisition I was talking about is this: http://www.ign.com/articles/2014/03/06/spotify-acquires-tech-behind-xbox-music-beats-music?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+ign/news+(IGN+All+News)
Hi Marc,
Nice piece of research and thanks for sharing it. I see that Neil Young will be releasing PronoMusic at SXSW this month. Seems like a big hill to climb, as I understand it they will SELL only hi bandwidth music that is proprietary to the Prono player, itself about 300 bucks. I wonder, as I am sure you have what the quality has to do with anything when it comes to play back. I have a decent stereo set up at home and in my modest studio, very acurate play back, but when you go out and about so much of the music you labor over is played back on ear buds and wireless speakers that sound like crap. This is the first I’ve seen of a test regarding quality, seems that cost and ease of use have won that war. As far as payment and fairness to artists, it keeps me from using theses services, I can’t stand the notion of no pay for play, but there is a whole generation and a half that don’t know what it is like to pay for songs, I guess I was born before that was the norm. Nice work, thanks.
FYI, MOG and Beats Music use the same content provider (MediaNet), and both offer 320kbps streaming when available.
Tim,
Great information, Thank You.
As far as Google, “Do No Evil,” it’s a cool slogan and that’s all.
They have led a conspiracy in California tech companies to suppress salaries of tech workers. The “No Poaching” of workers lawsuit is currently in court, so one could say they are not proven guilty. But Facebook admitted Google approached them to participate Facebook to their credit said no.
Chris
I do not accept your conclusions. You tested the “size” of encoded audio sinals instead of the “quality” of the output of ths sound signal. Those are two different variables. If you really want to analyze the quality of sound, you have to get a way to compare the origial audio signal produced by your cd player vs the signal produced by the services you tested.
If you need to make a “scientific test” to notice a difference in quality of those services then I think it is fair to say that their audio quality is pretty much the same.
Dear Eugenio,
unfortunately “audio quality” is NOT a measurable thing, it’s mostly a subjective opinion. I decided to test the “size” of encoded audio because that is mathematically-speaking quantifiable and I believe we all agree that the bigger the file the less compressed is the file the more of the original quality is preserved, that’s pretty much the science behind MP3 compression!
Unfortunately not all of these services use MP3 as their codec so your comparison is totally invalid.
The assumption made that “more data = better quality” only has a chance of being valid if the same codec is being used by each service. Even then, different encoding settings in the codec could affect the efficiency of the coding process while still giving results that are qualitatively “the same” upon decoding. Most modern codecs (ie, not basic mp3) use some form of perceptual encoding that seek to replicate the source at some perceived quality; no lossless encoder, by definition, is going to decode to a bit-perfect replica of the source material but rather strives to achieve a reasonable approximation that is perceptually identical.
As Eugenio stated, a better test to attempt to quantify the quality of each streaming service would be to capture the decoded audio and compare the output in both the time and frequency domains to the original source. The variability introduced by the DAC/cable/ADC connection from phone to PC should not affect the results that much. You are in a unique position having the original master of material available on each service and I’d be very interested in seeing the performance (particularly in frequency domain) of each service vs your master.
Best,
Gabe
Hi. I appreciate the test you just did here but I’m sorry to say, like many others pointed out, certain things make the results invalid. I have to say the one most important thing why they aren’t valid doesn’t even have to do with their using different codecs.
Beats Music advertises 320 Kbps as their highest bitrate so there’s no way you are streaming real CD quality.
Here’s what happens: You have a fast internet connection and when you are streaming your song it’s already caching other songs so it won’t need to load when you actually play them (and it is something dependent on the internet speed, that’s why it caches more on your home connection and that’s why you thought it was all for the same song). Realize that when you select an album on Beats Music and play the first song, the second one won’t even need to load, it’ll play immediately because when it finishes loading the song you selected it starts loading the next song.
You can try to verify it yourself by doing the test again with that same song that played with REAL CD quality, you will notice that it won’t take much data to load the next songs (don’t be surprised if you see no change in data).
I hope I explained it correctly.
Agree with previous comments. The audio codec is extremely important (eg 96kbps ogg vorbis is acceptable, 96kbps mp3 is terrible). I stick with Spotify, despite the fact that their software is awful, because they use ogg vorbis and are transparent about their bit rates (normal = 96, high = 160, extreme = 320, all approximate as most codecs after MP3 such as ogg have used some form of variable bit rate). The high setting sounds great and saves my mobile data usage, and I use extreme when on WiFi (for piece of mind, subjectively they seem about the same). Google for some reason uses mp3, an extremely outdated format, and seemed to change bitrate on the fly, which sounded noticeably bad to me, so I gave up on it after the trial.
Also, remember “CD quality” is marketing garbage and the only real requirement is that it have a 44.1 KHz setting (although high pass and low pass filters narrow the actual frequency response). The rest is subjective, which is tough as some people have horrible ears (I had someone tell me XM sounded better than a CD once. It blew my mind). On the flip side, somewhere above 128kbps (or >192 kbps with mp3), these files become “transparent” meaning it is impossible to perceive the difference between the CD and the compressed file. This should be considered CD quality, not just files that are the same size or bigger.
I agree with your results completly as tested by my own ears, I will also miss MOG and probably go with Spotify, but right after sound quality is which service has the most of the music I want to discover.
Watch this video. I think it clarifies a lot of concepts about digital audio. http://xiph.org/video/vid2.shtml
Marc, while tend to agree with most of this article from an opinion standpoint, you will have to accept that the testing method here is extremely flawed. With your methods, you won’t be able to discern:
1) If you are calculating ONLY one song file. Some services pre-cache text tracks. Or refresh other tracks that have nothing to do with your playlist.
2) true bitrate/codec information. (This is extremely important and must not be assumed)
I know this is the best anyone could do with the tools and methods provided, but wait… I’ll have to change that. Because you could have easily done this better…
The files for each song ARE cached and/or stored on the device. There are well-documented tools out there to view them and their sizes (no rooting or jailbreak required). You can even easily carve out the file and see the drm, codec, and bitrate of the file. THIS would be a true test.
In any case, I do commend you for taking all this time in the first place. I talk all this crap but I admit I don’t have the time myself to do any of these tests. So you hold the record for best (if not flawed) analysis so far ;)
You will definitely need to include Rhapsody in your next review. They have been around the longest.
Hi Marc,
Thank you for taking the time and going to such efforts to perform these tests. I personally found the results really interesting and appreciate what you’ve done. I’m just a regular non-geek and was having trouble wading through all the info about the various music streaming services, trying to decide which one to go for. I felt your report really helped me with that decision so wanted to say thank you!
you are most welcome!
Thanks Marc for engaging in measured testing. I’d like to share a first experience using simply my ears to (subjectively) compare two streaming services and check whether other audiophiles had similar experiences.
As a long time subscriber to Mog, I recently subscribed to both Beats and Spotify as a trial for Mog replacement. I listen streamed music mostly on a good quality stereo system connected to a Sonos DAC and have a pretty good ear for music (I’ve been playing guitar for years).
After a few days of trial of both solutions trying to rebuild my mog playlists on a PC, I have been leaning towards Spotify based exclusively on usability, assuming the quality would be the same as MOG with all services streaming at 320kbps:
– Sonos has a better integration to Spotify (e.g., ability to save searched music to a Spotify playlist from the Sonos controller) than to Beats (and former Mog);
– I am also having a really really hard time to try to like the Beats user interface on mobile devices and their web player is just a plain a total disaster…
However, when playing my Spotify playlists on the Sonos/high quality stereo components, I immediately noticed a slight degradation of audio quality compared to my years of playing Mog. I discarded this reaction as being too subjective for an overall “feel” on a particular day and time. However, in the process of moving tracks from Beats to Spotify from within the Sonos controller, I ended up switching the streaming from one service to the other for the same track in a couple of occasions. I was not trying to do any type of comparison but the difference in quality was noticeable enough to make me go back and play the track over and over on both services to check and confirm. The audio on Beats was clearer with more separation between instruments and the bass were rounder. In comparison, the audio on Spotify seemed a bit muddy. These are strong words and the difference is actually very subtle. It takes good audio components to hear the difference – I could not hear it playing the track from my Mac DAC connected to a medium quality stereo system. And by comparison, there is a greater difference in quality when comparing a streamed online service like MOG with FLAC music stored locally on my home network and streamed to the same Sonos/stereo component via the Sonos wireless system. I also had discarded the first occasion I noticed a difference as also being too subjective, but it bothered me. Now that I noticed this a second time for a different track (yesterday), I decided to investigate further, which made me search the web and find this thread.
I checked Spotify to see whether the track I was playing yesterday was indeed available as a 320kB track and it was, ruling out a simple explanation from playing 160kB versus 320kB.
I will now engage in more comparison and blind testing, still using my ears. And if I can confirm a better audio quality with Beats, I’ll drop Spotify, despite the horrible Beats UI. (But at least, on the Sonos controller, the UI is decent for the service.)
I cannot wait for Qobuz to be available in the US and try it out. This could be the ultimate streaming/download service for audiophile..
If any of you had similar experiences comparing Beats and Spotify, please update this thread. I’ll also update this post if further comparison of the service with blind testing ends up being inconclusive.
Thank you so much for your comment and insights mogy.
I would LOVE to hear your opinions and test results. I tried to stay away from subjective testing and wanted to keep it mathematical and scientific, even though some people pointed out that might method might have flaws because not the same compression algorithms are used by all the companies.
Please DO POST your results when you have them. I am VERY interested.
I didn’t know about Qobuz but it seems to be available right now already in fact I just made an account, downloaded the app and I am listening to a beautiful Martha Wainwright that I mixed (which was the only thing that came up under my name on Qobuz, which means the catalogue is not as comprehensive)… but it DOES SOUND GREAT and although their web interface is corky, it does seem to speak english and so does the app!
Check it out here:
http://www.qobuz.com/ie-en/offers/music-streaming-subscription
Unfortunately I find this article very disinformative.
At the least, you should and approach the matter from a different angle.
MP3 and lossy compression in general is based on psychoacustic models of hearing which are constantly under R&D.
The whole premise is to get the best audio quality at the lowest datarate (or size).
You can’t forget that streaming is complex and there are additional scenarios to solve, where you must compromise quality to an extent in order to maintain audibility and avoid break ups.
It’s worth mentioning that nowadays streaming audio is not just a “programmer’s problem” anymore.
AES is involved and especially great people like Bob Ludwig who is and has been a top mastering engineer for decades.
Modern encoders’ datarates are proving to give good quality.
But a real big step will be passing from 16 to 24 bit audio also as a standard format.
Appreciate your time for this in depth review. But is it true that the cache from streaming through cellular network will alter the download of the music through wifi? Or is this just what you were assuming. Please let me know as this is important for my decision
This was sponsored by Google for sure. I have compared several downloads from google to origianls and to spotify and Google quality is horrible. I don’t care if it says its 320 the quality is sub par.Listen for yourself.
Hi Marc
Thanks for the interesting article. There are problems with your statistics though. As you are measuring the data transferred through the network interface there is no way of telling what is transferring the data. There could be background processes downloading, and you don’t know how the data that is being retrieved by the streaming applications is being used, eg. caching. This would be most evident over WiFi and is probably the explanation for the anomalies seen for Beats Music. It is possible that BM has detected that you are playing songs from the same album and so pre-caches the rest of the album, explaining why a single song streamed over WiFi was larger than the whole album downloaded.
Due to these issues, the method of determining the size of the song downloaded is giving false results and cannot be relied upon. You can check the bit rate of any downloaded tracks by checking the properties in a file manager or playing through a music application that tells you the bit rate. Differences in codec will also have an impact on size and quality.
Beats Music will not be streaming uncompressed CD data. It would cost them too much in bandwidth for no return. Some music download sites offer lossless compression (eg. FLAC) such as Beatport, but there is a large premium for them.
There is also the problem that MP3 does not support bit rates in excess of 320kbps so any suggestions that some services are streaming MP3 tracks at a greater bit rate are false.
Cheers
Jim
Keep in mind though that 320 kpbs is the maximum for compressed lossy compression. A higher format is always uncompressed, non-lossy. But for the human ear 320 kpbs is fantastic.
@Marco not necessarily true
consider this: a CD is 44100*2(stereo) = 88200 samples * 16bits = 1411200 bits per second
Now when you go down to 320.000 there is a significant amount of data that must be discarded.
While I don’t argue with 320kps being good, my point is about *how* you get to discard the excess data.
That is, the psychoacoustic modeling algorithm does make a huge difference, sometimes it matters more than pure datarate especially when considering that over 256kbps encoders tend to be pretty much at the max quality achievable with them.
Yes, correct. But I am not arguing with that. I am only saying that 320 kbps is the maximum for lossless audio. If you want higher audio you need to go non-lossless.
Well done, i agree about the size. When i use Google Play music to stream music with My home theater, sound are just great, but no like a CD, but at computer with Logitech 2.1 or when i stream from my phone via bluetooth, spotfy sounds better and cleanner.
Well, all the naysayers who didn’t should first bow down and say a big THANK YOU for, most of all, raising the issue of “Is XXX streaming service REALLY streaming at the bitrates they say?”
It is an excellent question which shamefully I had not thought about.
For THAT question, it is quite valid to look at the file size!* Now, are there other things happening to distort the file size? Good question.
I don’t know how to check this, but if someone does it would be cool to post at the page that led me here:
http://forums.stevehoffman.tv/threads/bitrate-of-spotify-beats-rhapsody-truth-or-lies.369739/
Thanks Marc!
*By the way, testing I did a few years back led me to the conclusion that once the bitrate hits 192k and above, all the codecs start to sound pretty good. And at low rates like 96k, they all sound odd. So I think I am more interested in “Am I really downloading a 320k file? Or are you lying to me?” than in whether the codec is MP3 or Ogg Vorbis or whatever. I generally listen to downloaded files, so the streaming situation-where the codec would make more difference at a lower bitrate-is less important to me.
Hey, this is a great article. The effort is superb and the value is incalculable. Here’s why, there’s two ways to approach this, one subjective which is fine and the objective data which is clearly very helpful.
I’ve been with MOG which was a simple interface and the sound was terrific. Sound quality matters to many of us.
Spotify has the most polished interface but I discovered out and about the quality wasn’t as good as I expected of its ratings. So I’ve tried others and they all have strengths and different advantages but if sound quality is of primary importance, this article is a great resource.
Do find that Beats Music over wifi is impressive and so is Google Play Music which was a surprise. GPM allows you to upload your library and all of it if you exceed 20,000 songs for a reasonable price. That with the sound quality is tough to beat, pardon the pun.
Thanks for a great, great effort/article here!
I’ve switched back and forth over the years. Like many, I was most happy with MOG. With my own headphone tests (via a DAC through an amp) MOG and Play sounded identical, Beats not quite as good.
In truth, I miss Rhapsody, mostly because they actually took the time to organize their library sensibly and occasionally update things with reviews. All of these streaming services suck as far as the editing side goes (even though Beats plays up its “curation”).
As others have mentioned, I really appreciate your willingness to ask the question.
Looking back, though, there were obviously some things you mistakenly did not take into account. Beyond the codec issues mentioned, how were you confident in the measures of transferred data? Both iOS and the Data Monitor app track total data in/out of the device during the given time frame. With all of the background updating and syncing that most apps do these days, it might be presumptuous to assume that all of that data was related to your testing.
Anyway, it was an interesting read. Thanks!
I’m sorry, but this test isn’t scientific, as you state.
None of these services use a compression method that is greater than 320kbps MP3. They all use pre-caching in some form, that is, loading the next song or three so there’s no pause between tracks. If you have a track that is about 15MB at 320kbps MP3, you’re not going to get more data than that unless the app was pre-caching (which they all do). Your total data doesn’t account for album art, pre-loading the next advertisements, etc. You honestly have no idea what is inside that chunk of data.
Also, you don’t account for different compression methods, and you falsely state (in another post) that audio quality is subjective. Both of those are mistakes on your part. At 320kbps, most formats (MP3, OGG, AAC, etc.) sound essentially the same. At lower bitrates they can sound substantially different (especially MP3 vs. LAME MP3, which was an attempt to bring lower bitrate MP3 closer to lower bitrate AAC). On mobile, the majority of services try to stream at 128kbps or lower. The ones that choose AAC over MP3 have a significant advantage here. Pandora, for example, uses 32-48kbps over mobile, 64kbps on the “high quality” setting. Because they chose AAC, the “HQ” setting sounds on par with 128kpbs MP3.
You’re not streaming above 320kbps over the cellular network. With this “comparison” you showed your lack of understanding in terms of music data compression and how music streaming applications on mobile fundamentally work.
Jay Kresge everything you say is true. All this apps use pre-caching, when you’re listening a song the next track is stored already in the cache. When Google Play Music was introduced a year ago I was having long cuts (my connection is slow 4mb) when a song finished because the pre caching was not doing the job well, in Spotify the music playback was continuous all the time in the same network. Now Google Play Music has an efficient pre caching too.
As of Aug. 28th, Rhapsody now offers three quality levels, for both downloading and streaming: 64, 192, and 320.
“I have decided not to include iTunes Radio, Rhapsody or Pandora because they are the kind of apps that don’t allow you to listen to a specific song or artist (you can only create stations based around an artist).”
Two out of three ain’t bad. Rhapsody, unlike the other two, is a full on-demand streaming service, NOT a “radio” deal.
Hey bro thanks for you info and analysis. I recently did a similar comparison test to the one Mogy made. I have been using Spotify to stream music for a couple years and had been very happy with it’s music catalog. Last month, I got another free trial for Google Play Music All Access. The first time I had tried the service I was not happy with the music selection so I had stayed with Spotify. I figured since I was given a free trial I’d give it another shot. Having the ability to upload music to the Google Play Music cloud is a great perk. I began searching for songs on Google Play Music that I had on my Spotify playlists. Once the songs began playing I was astounded. The music just sounded better. I couldn’t believe my ears so I started playing the same track on Spotify and my suspicions were validated. The track played through Google Play Music was clearer and better defined. When comparing the two I found tracks on Spotify sounded muddy. I then compared other songs, different music genres and artists. I found that most of the tracks on Google Play Music sounded better than on Spotify. During that first test I used some Audio Technica Ath-m40x headphones. Then I tested this through my vehicles stereo and was surprised my wife even noticed the difference.
Anyways, appreciate your article and just wanted to share my experience. Wonder if anyone else has made these types of comparisons.
Complimenti per l’analisi, uso Google Play Music da un po’ di tempo ed ascolto abitualmente in streaming la mia raccolta su impianto audio con ampli Marantz, frontale Chario e sub IL Basso 850 via scheda audio xonar dx e spd/if. Le mie orecchie hanno sempre considerato il risultato piuttosto buono e mi pare che questo dati oggettivi lo confermino. Interessante il prodotto Beats Music
FYI – For those of you who have a Sonos system and who are in the US, the Deezer streaming service is now available on Sonos in the US, with uncompressed FLAC streaming at 1.5kbps. I have been enjoying it for one week now and compared to Spotify, Beats, former MOG, and the like, this is truly amazing (on a good stereo system connected to Sonos). I was waiting for Qobuz for FLAC streaming but Deezer is now here in North America. Outside of Sonos, there is no difference as Deezer streams in compressed/320 kbps format on your PC or mobile device.
A dumb network but Sky FM/Radio tunes stream in 320kb. Granted it only let’s you pick a genre then off it goes without you being able to tweek it. EXACTLY like a radio station. I tested them for a month and give them a solid B+ as their buffering raye sometimes was annoying. However ZERO runaround when it came time to cancel. I emailed them and next day my premium service was canceled. No retention specialist and all that crap. Cancel& done. Kudos for them. I’ll probably be back because of that. I’m spinning Google play in trial for now so we’ll see.
A good test in my opinion would be to have the CD ripped and loaded in Foobar(the audio player). Inside of Foobar invert the the sound wave so when you play it along with itself non inverted you should just hear nothing. Load up the same song in your streaming player and if need feed it into your PC with some 1/8′ to RCA cables. You could Fast forward on the Foobar track and then hit play right when the Streamed track catches up. If all it well the PC sound card should out put nothing but the differences in the tracks, you could use another program to monitor the audio actually coming out of the PC rather than just listening to it with headphones. If you don’t hear anything at all, then the streaming services audio is good enough
The streaming over WiFi and Cellular networks tests seem fallible. Unless you can accurately depict the test environment and conditions of the test device.
Cellular and WiFi data monitoring has a margin of error if you have apps installed that often run in the background and consume a portion of data. If at different times of the day these apps run randomly it could adversely affect the results of the monitored data traffic depending on which service you tested at any given time. Unless you vehemently turned off all data on all apps – this would ruin the integrity of this test? I know some apps of mine that will take up a couple KB of data in a few seconds at a time even when I leave it alone.
Also leaving the phone in one spot does not ensure accurate cell signal throughout testing periods. Various external factors come into play when it comes to measuring cellular interference and reception. Having a phone in one spot does not guarantee it has a fixed signal rate. I just wanted to point these out to you and not necessarily debunk the tests. I do believe these tests were as accurate as can be and you can never have a perfect test environment. I myself use Google Play to stream music.
Thanks for the hard work!
Also downloading my purchased music from Google Play – I can verify that the mp3’s are in fact 320kbps.
Hi,
Have you tried TIDAL?
Thank You for your excellent article! Hope I get these correctly (it’s way past my bedtime):
I convert my CD’s (PCM 16bit; 44,100Hz;1,411Kbps Stereo)
to the highest quality OGG file available (with my software): SETTINGS:
Compression ratios (0.1q the highest but I use the smallest 1.0q),
Kbps/sample (500Kbps) and a clock sample rate of 48KHz (i.e. 12,16,24,32,48, etc.)
Here’s an example of a CATeFILE’d Album Track/Cut name:
RATHER BE;ft_Jess_Glynne;2014^NewEyes;C04T0349B121-118;OHpop-elect;V8EN;Q10K500Z48.OGG (again thank you for your very helpful article)
While this is all very interesting from a DATA USAGE perspective, in reality it is not taking into account CODECS and DATA Compression techniques used by the services. The music QUALITY may be higher (or lower) than the raw DATA rates indicate. Unless all compression techniques for DATA compression (even setting aside audio codecs) are analysed, we can not know the actual quality of any service.
Google, as an example, have got some very good data compression techniques for Data in general to reduce bandwidth usage but keep the same amount of information as such moving.
While this gives a very ball park indication of actual data, what we truly need is to analyse the actual music files themself, whether from the cached file that is downloaded and played via cellular, or the offline downloaded file.
The other thing to take into account is people want something that sounds good to them, while minimising data usage while mobile as Cellular Data can be quite expensive in some areas.
Over all, not a bad review of data usage, but unfortunately it really falls short in quality comparisons.
that’s just great. I am wondering though when you upload your music to the google cloud what format and bitrate do they store ?
You forgot the fact that Spotify uses the OGG format witch uses less data and has higher quality than mp3. So I think Google music and Spotify might have very similar audio quality regardless of the size of the file.
What a very informative article, I loved how you followed a scientific process in determining which services actually tell the truth and which obfuscate the truth. Bravo!