catsim is a Python package for computerized adaptive testing (CAT) simulations. It provides multiple methods for:
- test initialization (selecting the initial ability of the examinees)
- item selection
- ability estimation
- test stopping
These methods can either be used in a standalone fashion [1] to power other software or be used with catsim to simulate the application of computerized adaptive tests [2], given a sample of examinees, represented by their ability levels, and an item bank, represented by their parameters according to some logistic Item Response Theory model.
Computerized adaptive tests are educational evaluations, usually taken by examinees in a computer or some other digital means, in which the examinee's ability is evaluated after the response of each item. The new ability is then used to select a new item, closer to the examinee's real ability. This method of test application has several advantages compared to the traditional paper-and-pencil method or even linear tests applied electronically, since high-ability examinees are not required to answer all the easy items in a test, answering only the items that actually give some information regarding his or hers true knowledge of the subject at matter. A similar, but inverse effect happens for those examinees of low ability level.
More information is available in the docs and over at Wikipedia.
Install it using pip install catsim.
NEW: there is now a Colab Notebook teaching the basics of catsim!
- Have an item matrix;
- Have a sample of examinee proficiencies, or a number of examinees to be generated;
- Create an initializer, an item selector, a ability estimator and a stopping criterion;
- Pass them to a simulator and start the simulation.
- Access the simulator's properties to get specifics of the results;
- Plot your results.
from catsim.initialization import RandomInitializer
from catsim.selection import MaxInfoSelector
from catsim.estimation import NumericalSearchEstimator
from catsim.stopping import MaxItemStopper
from catsim.simulation import Simulator
from catsim.cat import generate_item_bank
initializer = RandomInitializer()
selector = MaxInfoSelector()
estimator = NumericalSearchEstimator()
stopper = MaxItemStopper(20)
Simulator(generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)All dependencies are listed on setup.py and should be installed
automatically.
To run the tests, you'll need to install the testing requirements
pip install catsim[testing].
To generate the documentation, install the necessary dependencies with pip install catsim[docs].
To ensure code is valid and formatted before submission, install the necessary development dependencies with pip install catsim[dev].
catsim is compatible and tested against Python 3.5, 3.6, 3.7, 3.8 and 3.9.
- Official source code repo: https://github.com/douglasrizzo/catsim
- HTML documentation (stable release): https://douglasrizzo.com.br/catsim
- Issue tracker: https://github.com/douglasrizzo/catsim/issues
You can cite the package using the following bibtex entry:
@article{catsim,
author = {Meneghetti, Douglas De Rizzo and Aquino Junior, Plinio Thomaz},
title = {Application and simulation of computerized adaptive tests through the package catsim},
year = 2018,
month = jul,
archiveprefix = {arXiv},
eprint = {1707.03012},
eprinttype = {arxiv},
journal = {arXiv:1707.03012 [stat]},
primaryclass = {stat}
}catsim does not implement item parameter estimation. I have had great joy outsourcing that functionality to the mirt R package along the years. However, since many users request packages with item parameter estimation capabilities in the Python ecosystem, here are a few links. While I have not used them personally, specialized packages like these are hard to come by, so I hope these are helpful.