Why “pass rates” are useless and why we don’t publish them

You have probably seen exam prep companies make claims such as “90%+ pass rate!” or “Our pass rate is twice the average.” Statements like these are almost certainly too good to be true because just like any other statistics, pass rates can be misleading. So, what do pass rates really mean to you when choosing an exam prep company? Unfortunately, not very much.

We are not aware of any exam that has official pass rates based on what materials candidates used. An exam’s sponsoring organization may release statistics by exam part, by region, by quarter, etc. but these statistics don’t take into account the materials used by candidates. Therefore, any exam prep company advertising a pass rate is calculating it themselves, and chances are they don’t tell you exactly how they calculate their pass rate.

The official overall pass rates for the CMA, CIA, or CPA exams generally range from 40% to 50%. There is no doubt that excellent materials and training can provide results above average, but pass rates in excess of 80% are not really believable. Consider the following:

  1. Assuming an average pass rate of 40%, if one company has an 80% pass rate, then statistically either one of their competitors has a 0% pass rate, or every other company on the market has a pass rate below average. (One other possible explanation is that the company making the 80% claim just does not have many students. If that is the case, then there isn’t enough data for a meaningful pass rate.)
  2. Every company that advertises a pass rate claims a pass rate well above the average. If every company had pass rates well above average, then the average would be a lot higher! Because the examining bodies do not release pass statistics by exam prep provider, the only way a company can determine their pass rate statistic is by surveying their students themselves. Not every student is going to respond to the survey, and usually only those who either loved or hated the materials will respond. Or, a candidate that is so frustrated with their materials may not tell the material provider of their result so instead of being included as a fail, they are simply not included at all.
  3. Finally, pass rates do not include how they are calculated, and how they are calculated matters a lot. Here is a very simple example. Let’s assume that Acme Company offers exam prep materials and that they have 1 student. That student studies, takes the exam, and fails. Clearly Acme’s pass rate is 0%. Next, the same student re-studies with Acme, and passes. Does Acme now have a 50% pass rate, or a 100% pass rate?

Pass rates become even more complicated when a student studies with multiple companies. For example, if that same student from the above example purchased materials from ABC Company after failing with Acme and passes, how does Acme calculate their pass rate? They could still claim a 100% pass rate because that student studied with them and did, eventually, pass the exam, but only after supplementing with additional materials.

Here at HOCK, we do not advertise a pass rate because we have no way to calculate our pass rate in a straightforward, accurate, and clear manner. Any number that we could calculate would be a best guess, which is simply not useful.

If you can’t use pass rates to evaluate a company’s materials, what can you use?

  1. Read what other candidates say about the materials, not only testimonials published by the company but what is said on social media.
  2. Read the sample materials offered by the company to see if they match your learning style. Are the materials clear, understandable, and well-organized? Are they updated regularly?
  3. Read the guarantees offered by the company. Do they offer guarantees that provide ongoing support and updates for as long as you will need to pass the exams? Are their guarantees simple and easy to understand?
  4. Call or email the company and ask questions about their materials and support. When you are studying on your own, you need to know that they are behind you 100% and will be able to help you every step of the way.
  5. Who are the people that will help you during your studies? Are they full-time experts committed to helping you pass?

At HOCK, we believe that our materials speak for themselves, which is why we offer free samples of our all of our materials without requiring you to register or create an account. We also encourage you to read what students say about HOCK, not just on our site, but on social media as well.

Having said all of that about pass rates, there is one number that we can give you that is both accurate and meaningful. HOCK is the only company in the CMA market that has a guarantee that pays for you to re-take the exam if you do not pass on your first attempt, which we call the “You Pass Or We Pay” Guarantee. In almost two years of offering the “You Pass Or We Pay” Guarantee, we have never had to pay for someone to retake the exam. This tells us that if you follow the HOCK study plan, you will pass the exam.

We look forward to showing you our CMA, CIA, and CPA materials and helping you achieve your certification not with misleading claims, but with the most comprehensive materials available. If you have any questions please do not hesitate to contact us – we are here to help!

Brian Hock, CMA, CIA

1 Comment

  1. Ashraf on October 25, 2015 at 2:08 pm

    I agree on what you have articulated , thanks Dr. Brain.
    It was very useful information and true.

New Year's Sale Extended through February 28th! Save 20% on all CMA, CPA, and CIA exam prep.Buy Now and Save!