Column

By combining IRT and equalization, you can achieve a fair test no matter which test version you take. By operating the test over a long period of time, it is possible to create a collection of equalized test questions (item bank), and to conduct a highly fair test that is resistant to leakage of test questions.

IRT has a concept of test measurement accuracy, which is similar to the reliability of CTT. Measurement accuracy is an index that shows how much an individual's ability value varies on average when they take the same test repeatedly. IRT allows you to know detailed measurement accuracy for Candidate 's ability level. Even if it is equalized, if there is a large difference in difficulty between editions, there may be variations in the accuracy of this measurement. If we can create multiple test versions with equal measurement accuracy within a certain range, we can say that the test is more equal. Measurement accuracy in IRT is quantified by a curve on a number line called the amount of test information, so by setting appropriate standards and selecting questions from the item bank so that the curve is close, a fairer test version can be created. can be created.

In addition, a single exam often has multiple units or groups of exam questions called question areas. For a fair exam, it is important to create a test version that not only has equal measurement accuracy, but also takes into consideration the balance of the number of questions for each unit. If only the measurement accuracy of the test and the number of questions per unit are satisfied, there is a risk that the same test questions will always be asked, and in the end most of the test questions will be solved by the test takers. Therefore, in addition to measurement accuracy and the number of questions per unit, it is also important to properly control how often the same question is used (exposure rate) when conducting the exam. Prometric proposes an experimental method called Linear On the Fly Testing (LOFT). In LOFT, the test question to be answered is randomly selected from the item bank the moment the test taker starts answering. At this time, an optimized algorithm is used to balance measurement accuracy, number of questions per unit, and exposure rate.

In recent years, a method called adaptive testing has become mainstream in CBT, which optimizes the questions asked depending on the test taker's response situation. Adaptive testing is expected to produce highly accurate tests in a short amount of time, but because it requires a huge item bank, test development costs tend to be high. LOFT can be used even with relatively small item banks and is highly fair, so it can be said to be a test method with excellent cost performance.

At Prometric, we use highly reliable techniques such as question quality assessment, IRT, equalization, and LOFT to improve the quality of CBT exams and provide test analysis services that are fair to test Candidate and optimal for test administration organizations. I am.