After a lot of discussions around the notions around adaptiveness, I’m still confused, but certainly at a higher level.
Therefore I would like to try to try to precise what, for me, after all these discussions, the notion about “adaptive …” means and see if this gets us to a point where we could all agree on.
For me the first precision to make is to see that on some point we are speaking of adaptive we have to make a difference between adaptive testing and adaptive learning.
In an adaptive testing situation, the tool successively selects questions for the purpose of maximizing the precision of the exam based on what is known about the students from previous questions. From the student’s perspective, the difficulty of the exam seems to tailor itself to their level of ability. For example, if an examinee performs well on an item of intermediate difficulty, they will then be presented with a more difficult question. Or, if they performed poorly, they would be presented with a simpler question. Compared to static multiple choice tests that nearly everyone has experienced, with a fixed set of items administered to all examinees, computer-adaptive tests require fewer test items to arrive at equally accurate scores. (Of course, there is nothing about the adaptive testing methodology that requires the items to be multiple-choice; but just as most exams are multiple-choice, most adaptive testing exams also use this format.)
The basic computer-adaptive testing method are algorithms with the following steps:
- The pool of available items is searched for the optimal item, based on the current estimate of the examinee’s ability
- The ability estimate is updated, based upon all prior answers
- These steps are repeated until a termination criterion is met
- Nothing is known about the student prior to the administration of the first item, so the algorithm is generally started by selecting an item of medium, or medium-easy, difficulty as the first item of the sequence.
Minimum requirements for an effective adaptive testing are :
- A large item bank piloted on at least 500 students
- 1,000 students per year
- Specialized IRT calibration and simulation software.
- Staff with a PhD in psychometrics or an equivalent level of experience.
- Items (questions) that can be scored objectively correct/incorrect in real time
- Item banking system and CAT delivery platform
There are 2 adaptive learning approaches:
The “facilitator-driven” approach
One that is “facilitator-driven”, which refers to products that provide instructors with actionable student and cohort profiles — essentially dashboards.
- This approach is content-driven, which means that the dashboard output links a specific course’s content inventory “within a system of standards or learning sequences.”
- Facilitator-driven systems provide instructors with information they can act on;
- In the facilitator-driven model, the regularly feedback of a teacher is an integral part of the concept.
- A facilitator driven system provides teachers and curriculum designers the ability to make granular-level adaptation of the curriculum based on performance profiles and analytics of student’s performance.
The “assessment-driven” approach
In this approach, the system provides close-to-real-time (sometimes called “dynamic”) adjustments of the instructional content.
- Assessment-driven systems make their own adjustments.
- In order to provide these adjustments, assessment-driven systems must be correlated dynamically with assets, items and learning objects to standards, outcomes or other frameworks.
- The assessment-driven model allows students to move the course individually or in a group, without instructor interaction.
- This approach can have limited or no facilitator interaction where the integrated technology offers learners near real-time, seamless adjustments to instructional content.
- With an assessment-driven approach, there is a heavy reliance on the technology’s ability to monitor, track and analyze data to customize the content for the learner.
The two approaches are not mutually exclusive, and both might be found in a single product or system offering.