MonkeyFAQs – Headset Review Method Version 1.0
So often, a headset review includes a score but little on how the score is obtained. This provides minimal insight to you to help make decisions.
Numerous attempts were made before we settled on how to provide a meaningful review score for our readers. This method is version 1.0 and will continually be updated to improve. Transparency is paramount in providing trust.
Therefore, we bring you our scoring method. It is a mix of qualitative and quantitative scoring.
Scoring Category Matrix
Headset Review Point System
Our point system is 10 points awarded for each category totaling 100 possible points for the matrix. Some classes do not apply, and the potential points are reduced by 10 for each category. We also award fractions typically in the ½ amount. The following analysis shows the breakdown of the points.
- Other Reviews – Points 1-3
- Personal Testing – Points 4-6
- Comparables – Points 7-9
- Best in Class – Point 10
The first 9 points are awarded by default for each category. Therefore, each product starts with a score of 9. We deduct points from our results in each category for the first three methods. The level of infraction determines the point deduction.
- Minor -1 – Problems that do not prohibit the operation but are not liked by users.
- Medium -2 – If the issue, or issues, significantly detract from the operation of the headset and are warranting more dislike, returns, or possible low score reviews.
- Severe problems (-3). – If the issue is rampant and appears like a significant detractor, the last point is removed. Problems could include defects or generate reviews at the low end of the spectrum.
The last point is reserved for the best in class or exceeds all expectations.
Description and Example
The following is a discussion and example of our review process for the Quality category.
Other Reviews – Points 1-3
This part of our review is completed before, during, and after we purchase the headsets. There is a host of great sites we visit to read about headsets. This includes the manufacturer’s site.
For the ICOMTOFIT V4.1 review, we looked at three different websites, including collectors. Collectors are sites that scrub other sites and compile reviews. We did not find any issues that would warrant a deduction. We spend approximately 2 hours reading 50+ plus reviews. Therefore, this criteria received a total of three points.
Personal Testing – Points 4-6
The first part of reading other reviews gives guidance to our own testing to verify the online research. Since we only test one, or a few headsets, it is possible to not experience defects from such a small sample size.
Our personal testing of the ICOMTOFIT V4.1 included approximately 8 hours of use, including office, car, and outside testing. We dropped it on the concrete and let it roll around the car a bit. Our conclusion matched the online reviews, and we did not deduct a point.
Comparables – Points 7-9
We group headsets into a peer group and compare them to one another. This peer group determination is mostly by price point. The typical expectations are considered for the build quality. We look at:
- The connections between parts
- The feel of the buttons
- The materials
Our peer group for the ICOMTOFIT V4.1 is under $50 Bluetooth in-ear and ear-hook. The quality appears to be as good, if not better, than many in the peer group. Nothing stood out that would warrant a deduction.
Best in Class – Point 10
This point is the Montecristo, the Mona Lisa, the end-all-be-all quality for a headset. Awarded in each category only to the best. This point is not given lightly. We would rather have all 9’s until we absolutely find the 10. This is based on our thinking that a 10 out of 10 should have meaning.
The ICOMTOFIT V4.1 is close to getting the 10th point on quality due to the lack of negative feedback and the other two tests. At the moment, since we are still testing, we have not awarded the last point.