by Amin Saiar, PhD
and William Ellis, MS, RPh, ICE-CCP
Each year, the Board of Pharmacy Specialties (BPS) publishes the pass rates for each certification examination. A common misconception that arises with the announcement of the certification examination pass rates is that they are conflated with the level of difficulty of an examination. In truth, a pass rate simply reflects the overall performance of a cohort of examinees, which changes year to year as new candidates sit for an examination. Let’s consider some key ideas about the BPS examination process that may help pharmacy stakeholders interpret reported pass rates.
Pass rates are a sample-dependent measure. Each year, a unique cohort of examinees attempts BPS certification and recertification examinations, and each year the pass rate fluctuates. While all examinees meet eligibility criteria to sit for the examination, the specific characteristics of both the examination and examinee differ from year to year, which impacts the pass rate. Each cohort of examinees may come from the same population of pharmacists, but they are not the same individuals from year to year. Each year, we experience and observe changes in pharmacy practice, education, training, and myriad other factors affecting the readiness of certification candidates. BPS monitors for large changes over time in pass rate (and pass rates tend to be relatively stable for BPS certification examinations), but incremental changes from year over year are expected.
Like any statistic, the ability to interpret results is limited by sample size. Interpretations made on examinations with larger sample sizes can be made with increased confidence, while interpretations made on examinations with smaller sample sizes should be made with caution. There is no magic number, but statisticians and psychometricians will often look for 100 or more data points on descriptive measures. Each BPS certification examination differs in this regard, with the smallest examination at around 15 examinees per year and the largest at around 3000 examinees per year.
Pass rates are reported separately for the certification and recertification examinations, and further subdivided by first-attempt and all examinees. Within each specialty certification program, the certification and recertification examinations are separate assessments built from the same examination content outline. The recertification pass rate is based on a different cohort of examinees and a different examination than the certification examination. The reason first-time testing attempts are reported separately from all examinees is that repeat attempts can skew the overall pass rate. When looking at first attempt only, all examinee results are weighted the same in determining the pass rate.
Pass rates are a descriptive, not prescriptive, measure. We are often asked at BPS: What should the pass rate be? Instead, the question should be: Is the examination accurately identifying those with the appropriate knowledge and skills to practice safely and effectively in that pharmacy specialty area? Pass rates very close to 100% or 0% may suggest that the examination and the examinees are mismatched, but there isn’t evidence to say that 60% pass rate is more appropriate for a given examination than 70% or 50% or some other number. So long as we can say with confidence that the 60% who did achieve a passing outcome have adequately demonstrated the requisite knowledge and skills, then 60% (or whatever pass rate percentage is found) is the “correct” pass rate.
Passing standards, or cut scores, for BPS examinations are criterion-referenced. The passing standards for BPS examinations are measured by points on a scale. An individual examinee’s pass/fail outcome is never influenced by his or her peers’ performance. Instead, the passing standard for each BPS certification examination is compared to the minimum acceptable level of examination performance reflective of safe and effective pharmacy practice, as determined by subject matter experts in that specialty area. This passing standard is held across alternate forms of the examination to ensure a consistent and fair standard is maintained across all examinees.
BPS policies and procedures, including the reporting of pass rates, are in alignment with certification accreditation standards and certification industry best practices. BPS is accredited to the ISO-IEC 17024 Standards and the National Commission for Certifying Agencies Standards. (For more information on accreditation of BPS and its certification programs: https://www.bpsweb.org/about-bps/what-we-do/) BPS uses evidence-based psychometric processes to promote reliability, validity, fairness, and defensibility of the examinations, and applies such standards across all stages of examination design, development, and evaluation. Nothing is more important to BPS than to provide an independent assessment of pharmacists’ knowledge, skills, and experience in a way that is consistent, accurate, and fair.
We believe that these six points provide insight on interpretation of pass rates of BPS specialty examinations. One of the primary purposes of BPS is to enhance public protection by developing effective certification programs for pharmacy specialty practice. As such, setting meaningful standards for safe and effective pharmacy practice and maintaining the consistency of those standards across all examinees is of paramount importance. Though pass rates may indeed differ from year to year, a consistent and meaningful standard helps ensure that all examinees are given a fair opportunity to demonstrate the knowledge and skills necessary to engage in a pharmacy specialty area in a way that is safe, effective, and promotes optimized patient outcomes.
