The Big Breakdown (Ranking List Update)

Table of Contents

The New System

Tone & Technical Grading

Going forward, the judging criteria for the ranking list will be further distinguished into two distinct metrics: tonality and technicalities. I’ve written extensively about my definitions on tonality and technicalities in both my Graphs 101 guide and my technicalities article, so if you need more information you can simply refer to those.

But as quick refresher (and I really mean quick, don’t argue with me about what I’m about to say since they’re condensed summaries of 5,000 word articles) tonality is basically tuning and frequency response, while technicalities is an umbrella term referring to unmeasurable aspects such as resolution, transients and imaging.

So, why the change?

From the beginning this system was already being used for my ranking list, just subconsciously and in a more “arbitrary” way. Specifying and breaking down the main criteria of my grading system helps me be more transparent about my inner processes, as well as to help me be more consistent with my rankings.

I’ll probably not break down my rankings further than this since the problem of weighting individual components gets worse the more components I specify. I think there should at least be a certain degree of “abstractness” to the rankings since, at the end of the day, this is a subjective list of personal opinions.

On that note, I think there is a sort of “ironic dichotomy” with regards to tone and technicalities. Tonality may be the most easily measurable aspect of audio but the interpretation of the measurements is probably one of the most subjective aspects of the hobby. The concept of “technicalities” implies a certain level of objectivity where more/less of something is always better (resolution, positional accuracy, speed etc.) and yet its immeasurability implies complete subjectivity.

I don’t know. Call me a filthy centrist, but I think a marriage of the subjective and objective (the measurable and the immeasurable) is the best way of approaching the audio hobby.

Weighting

& the Specialist's Clause

For the most part, the overall rank of an IEM is weighted in favour of the lower of the two grades. However I’ve devised a system that accounts for outliers with large deltas, though still weighting heavily in the lower grade’s favour. 

The system is essentially a 3-to-1 weighting scale, where only a whole grade (3 sub-grades) difference will result in a single sub-grade increase from the lower grade. An example is shown below:

[Tone grade]/[Tech grade] = [Overall grade]
C/C+ = C (1 sub-grade difference, no change)
C/B- = C (2 sub-grade difference, no change)
C/B = C+ (3 sub-grade/1 whole grade difference, 1 sub-grade increase to lower-of-the-two)
B/C = C+
C/A- = C+ (5 sub-grade difference, still 1 sub-grade increase)
C/A = B- (6 sub-grade/2 whole grade difference, 2 sub-grade increase to lower-of-the-two)

This is also known as the Specialist’s Clause.

However if either tone or technicalities are graded as E or F, the Specialist’s Clause is void. That means that if, for instance, an IEM’s tone score is graded at E, its overall rank will be set at E even if it manages to score an S in technicalities.

Why not use an averaging system?

The problem with a direct averaging system (e.g. C-/C+ = C) is that the performance rating on the ranking list is on a “subjective log scale”. That is to say, the difference between C and C+ is much “higher” than, for instance, A and A+. Thus, I don’t think it’s the right call to, for example, put a C-/C+ graded IEM in the same league as a C/C.

These changes are done in order to penalise deficiencies in audio performance to a higher degree, making the higher ranks a lot more valuable and increasing the standards to which I’d judge an IEM (or headphone).At any case, if you don’t agree with my weighting you’d still be able to sort the ranking list table by either tone grade or technical grade.

A full up-to-date (as of 19/04/20) list of IEMs that have invoked the Specialist’s Clause is shown below.

Model nameTone gradeTechnical gradeGrade difference (in subgrades)Overall rank increase (in subgrades)Adjusted rank
64 Audio tia FourteB-S72B+
64 Audio tia Fourte NoirB-A41B
A&K T8iEBA+41B+
AAW NightingaleDB62C-
Astrotec Delphinus 5D-C41D
Audeze iSine 20 (No Cipher)DS-82C-
Audeze LCDi4 (No Cipher)DS+103C
Beyerdynamic XelentoBA+41B+
Campfire Solaris (SE)BA31B+
Dita DreamBS62A-
Earsonics EM10C+B+31B-
Empire Ears Zeus-XIVC+S82B
Fearless RolandD+B51C-
Hidition VientoS+A+31S-
HYLA CE-5B+A+31A-
HYLA TE-5BB+A+31A-
HYLA TE-5TB-A+51B
Kumitate KL-REF Type-SCB+41C+
Lear LCM-BD4.2DB-51D+
Lypertek TeviAC+51B-
Noble KhanBA+41B+
Rhapsodio ZombieDB-51D+
Shure KSE1200/KSE1500BS+72A-
Sony MDR-EX1000A-S-31A
Sony XBA-N3AB31B+
Spiral Ears SE5UDB62C-
Stealthsonics U9C+B+31B-
StereoPravada SB7A-S41A
Ultimate Ears UE18+ Pro Gen 2SA31A+
Unique Melody ME1D-B72D+
Westone W80C+B+31B-

List of Rank Changes

Model nameOld rankNew rank
1More StylishD+D
1More Triple DriverD+D
64 Audio N8AB+
64 Audio tia FourteA+B+
64 Audio tia Fourte NoirA-B
A&K T8iEA+B+
AAW NightingaleDC-
Audeze iSine 10/20 (No Cipher)C+C-
Audeze iSine 20 (Cipher V1)AA+
Audeze LCDi4 (No Cipher)BC
Acoustune HS1503CC-
Ambient Acoustics AM7D+D
Beyerdynamic XelentoA+B+
Campfire AtlasB+B
Campfire DoradoB-C+
Campfire IOD-E
Campfire JupiterBC+
Campfire Polaris V1BC+
Campfire Solaris (SE)AB+
Campfire VegaB-C+
Custom Art FIBAE 3BC+
Custom Art FIBAE BlackBB-
Dita Dream XLSB+B
Earsonics EM10BB-
Elysian ArtemisB-C+
Elysian Artemis V2A-B+
Elysian Cora (2019) BB-
Elysian TerminatorAB+
Empire Ears PhantomB+B
Empire Ears ValkyrieB+B
Empire Ears WraithB-C+
Empire Ears Zeus-XIVA+B
Fearless S6ProA-B+
FiiO FH7B-C-
FitEar MH334A-B+
FitEar ParterreB+B
FitEar Private 333BB-
FitEar TG334A-B+
FiR M2D+C-
Future Sonics MG5HXC+C
Geek Wold GK3D-E
Hidition NT6AA+
Hidition VientoSS-
Hidition WaltzA-A
Hifiman RE2000BB-
HUM Pristine ReferenceB+B
HYLA CE-5AA-
HYLA TE-5BAA-
HYLA TE-5TB+B
Intime SoraCC+
JH Audio LaylaBC+
Jomo FlamencoS-A+
Jomo SambaS-A+
Kumitate KL-REF Type-SBC+
Lear LCM-BD4.2CD+
Lime Ears Aether RBC+
Meze Rai PentaBB-
NCM BellaAB
Noble KhanAB+
Noble M3CD+
Nuforce HEM4ED-
Nuforce HEM6ED-
Oriolus FinschiC+C
Rhapsodio ChipperDD-
Rhapsodio ZombieC-D+
Shanling ME100CC-
Seahf AWK-009DD-
Shozy x AAW HibikiED-
Sony MDR-7550/MDR-EX800AA-
Sony MDR-EX1000A+A
Sony IER-M7A-A
Sony IER-M9A+S-
Sony Justear XJE-MH2 “Club”AA-
Soranik IonAB+
Stealthsonics U9BB-
StereoPravada SB7A+A
Shure KSE1200S-A-
Shure SE215DD+
Tin HiFi P1CC-
Ultimate Ears LiveCC-
Ultimate Ears UE18+ Pro Gen 3A-B
Unique Melody MacBethBB-
Unique Melody Maestro V2AB+
Unique Melody ME1DD+
Warbler PreludeCD+
Westone W80B+B-

Rank Distribution

This is the new distribution of ranks after the update. You can see that it’s somewhat Gaussian but with a weird up-down pattern throughout. That’s because they’re meant to be Gaussian within the whole grades themselves (e.g. C+/C/C-), with the “centre grade” having the greatest proportion.

Now it looks more traditionally Gaussian (or at least, skewed Gaussian) when we combine the +/- grades. C is what I’d consider an average “not good but not bad” grade, so it makes sense that it’s the rank with the greatest proportion of IEMs.

While there is a skew towards the higher ranks, I think it can be reasonably explained with selection bias since most of the IEMs that I’d go out of my way to try are typically those marketed to the audiophile crowd, hence the higher average performance. If I were to start ranking every dollar store and Best Buy consumer earphone out there, I’m sure the data will quickly skew in the opposite direction. 

Apart from selection bias, I think it also stands to reason that it’s very rare for a company to greenlight a truly bad IEM out to the market, so that’s why we’re not seeing a lot of D/E/F-tier stuff. F-tier is pretty much the “S-tier” of bad IEMs, and as you can see is typically reserved for complimentary airplane buds-type performance. It’s a high bar to clear, arguably higher than even the lauded S-ranks.

Note Weight

The “note weight” section is still incomplete, but I guess I’ll take the time to explain what it is and why it could help in your purchase decision making.

The simplest definition for note weight is that it’s how “thick” or “thin” the IEM/headphone makes your music sound. On the ranking list this is rated on a scale of -5 to +5, where -5 would be the thinnest and +5 would be the thickest.

The main thing that affects note weight is the balance between lower harmonics (which includes the fundamental) and upper harmonics. A focus on lower harmonics makes the sound rich and deep, while a focus on higher harmonics makes the sound clear and airy. For instance, a heavy note weight would make an instrument  like a piano sound more “grand”, while a lighter note weight would make it sound more like an upright.

Different note weights tend to play better with different instruments depending on the musical range of said instrument, for instance a heavier note weight would play “better” with bass, baritone and tenor instruments, while lighter note weights tend to fare better with alto, soprano and percussions. 

That said, there is no objectively best note weight value since anything that deviates from neutrality is considered “colouration”, and colouration isn’t necessarily a bad thing if you prefer it. Though that said, extremely heavy notes tend to be rather muddy, while severely underweighted notes can sound shrill and lacking in body. Knowing your preferences in note weight is the key in finding the right IEM or headphone for your needs and preferences.

QoL Changes

  • Mobile view of the ranking list has been changed to a side-scrollable interface.
    • This is to address the complaints of the “click to expand” interface being too cumbersome to use, and to allow sorting of other columns on mobile.
  • A small changelog section has been added.
  • Rank sorting has been fixed. Now ranks (overall, tone and technical) will display in the correct ascending/descending order when selected.
  • The page jump issue on Chrome Android should be fixed. If not, let me know again.

Support me on Patreon to get access to tentative ranks, the exclusive “Clubhouse” Discord server and/or access to the Premium Graph Comparison Tool! My usual thanks to all my current supporters and shoutouts to my big money boys:

“McMadface”
“Desertscrub”
“Galactus”
Will
Kurt
Man Ho
Denis
Nicholas
Alexander
Andrew
Ted
chisquare
Rene
“Brbi2kCRO”

7 thoughts on “The Big Breakdown (Ranking List Update)”

  1. That’s a interesting change right here.. It should shake things up and make for a better ranking distribution overall.

Leave a Reply

Your email address will not be published.

First impressions

Audeze LCD-i3: Unboxing

INTRODUCTION Audeze is a company that needs no introduction in the audiophile scene, but here’s some exposition for word count’s

Read More »