Table of Contents
The New System
Tone & Technical Grading
Going forward, the judging criteria for the ranking list will be further distinguished into two distinct metrics: tonality and technicalities. I’ve written extensively about my definitions on tonality and technicalities in both my Graphs 101 guide and my technicalities article, so if you need more information you can simply refer to those.
But as quick refresher (and I really mean quick, don’t argue with me about what I’m about to say since they’re condensed summaries of 5,000 word articles) tonality is basically tuning and frequency response, while technicalities is an umbrella term referring to unmeasurable aspects such as resolution, transients and imaging.
So, why the change?
From the beginning this system was already being used for my ranking list, just subconsciously and in a more “arbitrary” way. Specifying and breaking down the main criteria of my grading system helps me be more transparent about my inner processes, as well as to help me be more consistent with my rankings.
I’ll probably not break down my rankings further than this since the problem of weighting individual components gets worse the more components I specify. I think there should at least be a certain degree of “abstractness” to the rankings since, at the end of the day, this is a subjective list of personal opinions.
On that note, I think there is a sort of “ironic dichotomy” with regards to tone and technicalities. Tonality may be the most easily measurable aspect of audio but the interpretation of the measurements is probably one of the most subjective aspects of the hobby. The concept of “technicalities” implies a certain level of objectivity where more/less of something is always better (resolution, positional accuracy, speed etc.) and yet its immeasurability implies complete subjectivity.
I don’t know. Call me a filthy centrist, but I think a marriage of the subjective and objective (the measurable and the immeasurable) is the best way of approaching the audio hobby.
& the Specialist's Clause
For the most part, the overall rank of an IEM is weighted in favour of the lower of the two grades. However I’ve devised a system that accounts for outliers with large deltas, though still weighting heavily in the lower grade’s favour.
The system is essentially a 3-to-1 weighting scale, where only a whole grade (3 sub-grades) difference will result in a single sub-grade increase from the lower grade. An example is shown below:
[Tone grade]/[Tech grade] = [Overall grade]
C/C+ = C (1 sub-grade difference, no change)
C/B- = C (2 sub-grade difference, no change)
C/B = C+ (3 sub-grade/1 whole grade difference, 1 sub-grade increase to lower-of-the-two)
B/C = C+
C/A- = C+ (5 sub-grade difference, still 1 sub-grade increase)
C/A = B- (6 sub-grade/2 whole grade difference, 2 sub-grade increase to lower-of-the-two)
This is also known as the Specialist’s Clause.
However if either tone or technicalities are graded as E or F, the Specialist’s Clause is void. That means that if, for instance, an IEM’s tone score is graded at E, its overall rank will be set at E even if it manages to score an S in technicalities.
Why not use an averaging system?
The problem with a direct averaging system (e.g. C-/C+ = C) is that the performance rating on the ranking list is on a “subjective log scale”. That is to say, the difference between C and C+ is much “higher” than, for instance, A and A+. Thus, I don’t think it’s the right call to, for example, put a C-/C+ graded IEM in the same league as a C/C.
These changes are done in order to penalise deficiencies in audio performance to a higher degree, making the higher ranks a lot more valuable and increasing the standards to which I’d judge an IEM (or headphone).At any case, if you don’t agree with my weighting you’d still be able to sort the ranking list table by either tone grade or technical grade.
A full up-to-date (as of 19/04/20) list of IEMs that have invoked the Specialist’s Clause is shown below.
|Model name||Tone grade||Technical grade||Grade difference (in subgrades)||Overall rank increase (in subgrades)||Adjusted rank|
|64 Audio tia Fourte||B-||S||7||2||B+|
|64 Audio tia Fourte Noir||B-||A||4||1||B|
|Astrotec Delphinus 5||D-||C||4||1||D|
|Audeze iSine 20 (No Cipher)||D||S-||8||2||C-|
|Audeze LCDi4 (No Cipher)||D||S+||10||3||C|
|Campfire Solaris (SE)||B||A||3||1||B+|
|Empire Ears Zeus-XIV||C+||S||8||2||B|
|Kumitate KL-REF Type-S||C||B+||4||1||C+|
|Spiral Ears SE5U||D||B||6||2||C-|
|Ultimate Ears UE18+ Pro Gen 2||S||A||3||1||A+|
|Unique Melody ME1||D-||B||7||2||D+|
List of Rank Changes
|Model name||Old rank||New rank|
|1More Triple Driver||D+||D|
|64 Audio N8||A||B+|
|64 Audio tia Fourte||A+||B+|
|64 Audio tia Fourte Noir||A-||B|
|Audeze iSine 10/20 (No Cipher)||C+||C-|
|Audeze iSine 20 (Cipher V1)||A||A+|
|Audeze LCDi4 (No Cipher)||B||C|
|Ambient Acoustics AM7||D+||D|
|Campfire Polaris V1||B||C+|
|Campfire Solaris (SE)||A||B+|
|Custom Art FIBAE 3||B||C+|
|Custom Art FIBAE Black||B||B-|
|Dita Dream XLS||B+||B|
|Elysian Artemis V2||A-||B+|
|Elysian Cora (2019)||B||B-|
|Empire Ears Phantom||B+||B|
|Empire Ears Valkyrie||B+||B|
|Empire Ears Wraith||B-||C+|
|Empire Ears Zeus-XIV||A+||B|
|FitEar Private 333||B||B-|
|Future Sonics MG5HX||C+||C|
|Geek Wold GK3||D-||E|
|HUM Pristine Reference||B+||B|
|JH Audio Layla||B||C+|
|Kumitate KL-REF Type-S||B||C+|
|Lime Ears Aether R||B||C+|
|Meze Rai Penta||B||B-|
|Shozy x AAW Hibiki||E||D-|
|Sony Justear XJE-MH2 “Club”||A||A-|
|Tin HiFi P1||C||C-|
|Ultimate Ears Live||C||C-|
|Ultimate Ears UE18+ Pro Gen 3||A-||B|
|Unique Melody MacBeth||B||B-|
|Unique Melody Maestro V2||A||B+|
|Unique Melody ME1||D||D+|
This is the new distribution of ranks after the update. You can see that it’s somewhat Gaussian but with a weird up-down pattern throughout. That’s because they’re meant to be Gaussian within the whole grades themselves (e.g. C+/C/C-), with the “centre grade” having the greatest proportion.
Now it looks more traditionally Gaussian (or at least, skewed Gaussian) when we combine the +/- grades. C is what I’d consider an average “not good but not bad” grade, so it makes sense that it’s the rank with the greatest proportion of IEMs.
While there is a skew towards the higher ranks, I think it can be reasonably explained with selection bias since most of the IEMs that I’d go out of my way to try are typically those marketed to the audiophile crowd, hence the higher average performance. If I were to start ranking every dollar store and Best Buy consumer earphone out there, I’m sure the data will quickly skew in the opposite direction.
Apart from selection bias, I think it also stands to reason that it’s very rare for a company to greenlight a truly bad IEM out to the market, so that’s why we’re not seeing a lot of D/E/F-tier stuff. F-tier is pretty much the “S-tier” of bad IEMs, and as you can see is typically reserved for complimentary airplane buds-type performance. It’s a high bar to clear, arguably higher than even the lauded S-ranks.
The “note weight” section is still incomplete, but I guess I’ll take the time to explain what it is and why it could help in your purchase decision making.
The simplest definition for note weight is that it’s how “thick” or “thin” the IEM/headphone makes your music sound. On the ranking list this is rated on a scale of -5 to +5, where -5 would be the thinnest and +5 would be the thickest.
The main thing that affects note weight is the balance between lower harmonics (which includes the fundamental) and upper harmonics. A focus on lower harmonics makes the sound rich and deep, while a focus on higher harmonics makes the sound clear and airy. For instance, a heavy note weight would make an instrument like a piano sound more “grand”, while a lighter note weight would make it sound more like an upright.
Different note weights tend to play better with different instruments depending on the musical range of said instrument, for instance a heavier note weight would play “better” with bass, baritone and tenor instruments, while lighter note weights tend to fare better with alto, soprano and percussions.
That said, there is no objectively best note weight value since anything that deviates from neutrality is considered “colouration”, and colouration isn’t necessarily a bad thing if you prefer it. Though that said, extremely heavy notes tend to be rather muddy, while severely underweighted notes can sound shrill and lacking in body. Knowing your preferences in note weight is the key in finding the right IEM or headphone for your needs and preferences.
- Mobile view of the ranking list has been changed to a side-scrollable interface.
- This is to address the complaints of the “click to expand” interface being too cumbersome to use, and to allow sorting of other columns on mobile.
- A small changelog section has been added.
- Rank sorting has been fixed. Now ranks (overall, tone and technical) will display in the correct ascending/descending order when selected.
- The page jump issue on Chrome Android should be fixed. If not, let me know again.
Support me on Patreon to get access to tentative ranks, the exclusive “Clubhouse” Discord server and/or access to the Premium Graph Comparison Tool! My usual thanks to all my current supporters and shoutouts to my big money boys:
7 thoughts on “The Big Breakdown (Ranking List Update)”
That’s a interesting change right here.. It should shake things up and make for a better ranking distribution overall.