crinacle’s Headphone Ranking List
If it’s not on the list, it doesn’t exist.
Table of Contents
This has been “behind the scenes” for a while but with the release of the graph database (and getting more listens in for some headphones), I feel confident to finally publish the latest stable iteration of the controversial Headphone Ranking List.
Now this update is essentially a complete overhaul of the ranking list given that I’ve basically tripled the number of entries on the list, so a changelog of what’s added (and what ranks have changed since the previous iteration of the list) would be largely pointless. That said, there are a number of non-entry-related changes to the list that I’d like to get into now.
Changelog
- Fit and cup type column has been added.
- These include but are not limited to: open circumaural, closed circumaural, open supra-aural, closed supra-aural, clip-on, and earspeaker.
- The technical score now gives more weighting to soundstage and imaging capabilities.
- This is exclusive to the Headphone Ranking List and does not apply to the IEM Ranking List.
- Most of the “Comments” sections of older entries have been redone to be of higher quality.
- The Value Rating system has been modified, particularly for the ★★★ ratings.
★★★: “Worth the Blind”
This makes it such that there is a more natural progression from 1-star to 2-star to 3, as opposed to the old system where a 3-star rating is largely irrelevant to the value it possesses (“Revolutionary”).
Note that the tagline of “Worth the Blind” is not an endorsement nor guarantee that one would love a headphone, but rather that a certain headphone possesses qualities that would justify some form of a blind purchase assuming that the reader has that level of disposable income.
Again, a 3-star rating is not a guarantee of personal enjoyment nor is it a recommendation, but rather placing the bar up so high that it becomes a coveted position via exclusivity.
Before I proceed any further, some disclaimers and explanations for those new to the website. For the rest, just skip to the last section.
Disclaimers
AKA: "Avoiding the Strawmen"
- Selection bias is going to be high in the ranking list for now, similar to how it was during the early days of my IEM ranking list. I’ve only sought out headphones that piqued my interest or has significant hype behind them, which tends to be the kilobuck stuff.
Much like for my IEM reviews, I will go with a policy of “out-of-the-box” when determining a headphone’s performance. This means the following limits will be placed:
- All headphones are tested stock with no third-party modifications.
- EQ is not a consideration.
- Exclusion: the EQ solution is a hardware one, and comes in the retail packaging of the headphone itself.
- This means that while things like the Audeze Cipher cable may fall into this exclusion clause, the software-based Audeze Reveal Plugin does not.
- Exclusion: the EQ solution is a hardware one, and comes in the retail packaging of the headphone itself.
- All headphones are tested with stock pads. If the pads don’t come in the retail packaging of the headphone itself, it will not be considered.
- Exclusion: selectable pads on checkout, i.e. ZMF and their pads.
- All headphones will be tested with the same source (iFi Micro Black Label).
- Exclusion: headphones that cannot be powered off headphone amplifiers (e.g. Stax headphones, RAAL SR1a).
- The cost of the additional audio component(s) will be factored in the final valuation of the headphone, based on the “generally accepted minimum”.
- Exclusion: headphones that cannot be powered off headphone amplifiers (e.g. Stax headphones, RAAL SR1a).
I know some would be foaming at the mouth at the last point so I’ll address it further. The world of desktop amplifiers and DACs is a crazy one, with each discrete component sometimes exceeding the cost of the headphone itself. Since the world of headphones is much more intertwined with the world of sources, I know it’s basically impossible to keep everyone happy with whatever testing methodology I choose, even if I were to somehow seek out the ideal sources for every individual headphone.
Therefore, my testing methodology prioritises consistency over optimal performance. The concept of “optimal performance testing” is a slippery slope, since one would have to determine if the most optimal setup (oftentimes the most expensive as well) is a “requirement” when it comes to running a certain headphone. And if there is a consensus that there is, then comes the question of if the headphone should be valued at the cost of the headphone on its own, the cost of only “the headphone plus the most necessary components”, or even at the cost of the entire setup.
So rather than go through that nightmare, I might as well keep my testing process simple and consistent. At any case, what I’m reviewing are headphones, not sources. I understand there are certain caveats that come with keeping the source gear constant for every review (i.e. the concept of “scalability” if you believe that), but this is the methodology that should keep the most happy and step on the least toes.
Weighting
& The Specialist's Clause
For the most part, the overall rank of a headphone is weighted in favour of the lower of the two grades. However I’ve devised a system that accounts for outliers with large deltas, though still weighting heavily in the lower grade’s favour.
The system is essentially a 3-to-1 weighting scale, where only a whole grade (3 sub-grades) difference will result in a single sub-grade increase from the lower grade. An example is shown below:
[Tone grade]/[Tech grade] = [Overall grade]
C/C+ = C (1 sub-grade difference, no change)
C/B- = C (2 sub-grade difference, no change)
C/B = C+ (3 sub-grade/1 whole grade difference, 1 sub-grade increase to lower-of-the-two)
B/C = C+
C/A- = C+ (5 sub-grade difference, still 1 sub-grade increase)
C/A = B- (6 sub-grade/2 whole grade difference, 2 sub-grade increase to lower-of-the-two)
This is also known as the Specialist’s Clause.
However if either tone or technicalities are graded as E or F, the Specialist’s Clause is void. That means that if, for instance, a headphone’s tone score is graded at E, its overall rank will be set at E even if it manages to score an S in technicalities.
Why not use an averaging system?
The problem with a direct averaging system (e.g. C-/C+ = C) is that the performance rating on the ranking list is on a “subjective log scale”. That is to say, the difference between C and C+ is much “higher” than, for instance, A and A+. Thus, I don’t think it’s the right call to, for example, put a C-/C+ graded headphone in the same league as a C/C.
These changes are done in order to penalise deficiencies in audio performance to a higher degree, making the higher ranks a lot more valuable and increasing the standards to which I’d judge a headphone. At any case, if you don’t agree with my weighting you’d still be able to sort the ranking list table by either tone grade or technical grade.
FAQ
Q: Why is (insert headphone here) ranked so low? I thought it was really technically capable/well-tuned and doesn’t deserve that rank.
A: According to the grading process I’ve outlined for IEF, technical ability OR tonality are only half of the equation. I understand that some may value technicalities and ignore tonality, while another may value tonality and not care for technicalities. In those cases, the ranking list allows you to sort by tone grade or technical grade.
However, the overall grade is determined by a weighted average based on the “lower of the two” (explained above) and that’s just how I do things around here. If you disagree, publish your own list and get your own hate mail.
Q: Your sources aren’t good enough for (insert headphone here). Please use (insert amplifier here) for (insert headphone here) next time.
A: If you are willing to add on the cost of the amplifier to the MSRP column (and also changing its value rating in the process), then I can consider it.
Else, I don’t like the assumption that everyone has your amplifier or source or whatever. People have their budgets and so it’s my job to rank the headphone at their cost, not “the combo”.
Q: Would you rank modded headphones?
A: If they are commercially available, yes. If they are one-offs or require one to send in a previously-purchased stock headphone for modification, then no.
Q: (insert headphone here) has a street price of X and it never sells at MSRP. Can you adjust your column entry and value rating to reflect that?
A: If it never sells at MSRP, then tell the company to change their MSRP. I’m just playing according to their own rules.
Q: I don’t like your ranking list and I hate you.
A: It’s okay buddy, I hate me too.
Next up: updating the IEM Ranking List.
Support me on Patreon to get access to tentative ranks, the exclusive “Clubhouse” Discord server and/or access to the Premium Graph Comparison Tool! With current efforts to measure more headphones, those in the exclusive Patreon Discord server get to see those measurements first before anybody else.
My usual thanks to all my current supporters and shoutouts to my big money boys:
“McMadface”
“Galactus”
Will
Man Ho
Denis
Alexander
26 thoughts on “The Spicy Update: 170 Headphones Ranked!”
Wow I have the Sennheiser Amperior, never thought it would have been so low in the list. Tyll from Innerfidelity lied to me – audiophile entry level my ass! HAHAHAHA (nervous laugh)
Sometime it`s also a matter of taste….. don`t give a shit. If it`s good for you it`s just fine
Typo: myraid of small quirks ultimately screws its tonal balance.
-> myriad
How did the LCD-X go from “the best-tuned of the Audezes” to “the worst-tuned of the Audezes”?
I tried two LCD-Xes and both of them are different.
Understood, thanks.
Hi Crin,
Why do the 880 and 600 have the same technical rank? Is it just because of stage and imaging? I have both (Even the 600Ohm version of the 880) and while I do enjoy both I always felt like the 600 has way better image clarity and is way better at bringing out the texture of certain instruments (if that makes any sense to you).
I’d really be happy if you’d answer me.
Best regards Anton
Worst list I’ve ever seen, by far. Hopefully everyone takes his garbage with a grain of salt.
It’s okay buddy, I hate me too.
Can’t agree more 🙂
PogU
You can’t even make your own list 🙈
Crinacle, I enjoyed a lot looking at your rankings, measurements and appreciations!
Attention : few details need to be corrected (e.g. Stellia is a closed not an open headphone).
I am impressed by your honest and competent work! While I don’t agree at 100% (who would given subjective things like ranking “technicalities”), the thrust of your tremendous work is very healthy and much needed, given the corruption taking place in the professional and semi-professional milieu, I mean people hiding severe deficiencies of expensive stuff (because they receive it for free and sell it back afterwards, or are paid indirectly though adds or even directly. I see you have adds on your site, yet you have the courage to be honest), hiding even fundamental failures such as very large deviation too much from neutrality, i.e. re. the most basic requirement defining high fidelity. Your list will be super helpful to those who are misled by the numerous fake and incompetent (usually with exclusively positive, dishonest verbiage for the item being reviewed) reviews on the internet and audio magazines.
Keep up your great job! It will be great to see your list becoming better and better!
I’m really glad crinacle fills the empty spot tyll left when he decided to retire.
Now that you are venturing into the headphone territory I feel that you might offend a lot of old folks with your opinion.
Best of luck
Indeed, you are very right, Patrick!
What was fantastic with Tyll was the great (and of course competent) videos he could produce. For other aspects I find Crinable even much better than Tyll:
– better measurement system
– super-useful graph matching tool
– super-large measurement database
– so much younger ears (how old would Crinacle be? I guess certainly below 30, while Tyll was reaching 60)
– corruption-free system for getting items to be reviewed, independent from manufacturers, like at the other excellent site, Audio Science Review (ASR).
So we lost Tyll, (whose replacement was so utterly incompetent in so many respects that it must have taken him only about one year to have Tyll’s innerfidelity site simply shut down!). But we gained much more with two fantastic, super competent and honest sites going over into headphones, Crinacle’s In-Ear Fidelity and Amir’s ASR :-).
You think you can upload measurements for the Sennheiser HD 407 and HD 569? I own the 407 and need to find a good EQ for this crap, but nobody has uploaded measurements, and the 569 is probably my next buy. 407 might be hard to find though, it’s a developing market model.
Interesting list. Would have to agree with some, not with others, as with all subjective matters.
You forgot to include the most popular, best selling headphones that comprise 50% of all headphone sales in the world. The cheaply made, “cannot distinguish from child’s toy headphones” but its the best because Apple makes it and comes in attention grabbing colors, Beats by Dr. Dre “Studio” Headphones. Where would you place that on this list? Because, I would really like to show some Beats owners this list and they need that to see how well they spent $300.
I’d be afraid the beats rank a lot higher than anyone might expect. (full disclosure – never heard them myself)
Great ranking!
One obvious fault to be corrected please: Abyss Diana Phi is not a “warm” sounding headphone, which your measured frequency response curve clearly shows.
Thanks,
Bernie
Oh… Hi, Sanders!
I’m surprised how high the R70x ended up in the list. I have both the HD6XX and the R70x and found that the HD6XX actually resolves better despite its smaller soundstage.
I do agree though, that the R70x’s tonality is more neutral to my ears.
Just a bit weird to see the R70x, HD800/S and the L300/500/700 STAX on the same tier.
MSR7b DLC coated drivers is one of my favorite headphones of all time. Unfortuante that it ranks so low here.
compared to the old model it is more resolving in the treble, yet slightly warmer. Also very composed on complex music.
It is also very source revealing despite the slightly warmer sound that previously.
It does have some treble bite to it in some areas, but I wouldnt call it thin at all.
the 250-300hz notch that AT does from time to time is absolutely CORRECT for a CLOSED headphone to sound neutral in my opinion. Depending on the housing.
Did you test the 32 or 250 Ohms version of the Beyerdynamic T70?
Considering the T70 is 250ohms and the T70p (portable) is the 32 ohms and he only states “T70” I think we can safely assume he tested the 250 ohms. For all intents and purposes I believe they sound pretty much the same.