Moodle’s latest version offers simplified and improved accessibility to analytical data from the quiz module. The latest updates to the Quiz Statistics Report in Moodle 3.6 specifically focus on the response statistics of randomized questions. Instructors can now view response statistics for quiz questions deployed from the pool of randomized questions easily, regardless of the total number of randomized questions within the quiz.
Before Moodle 3.6…
In prior versions of Moodle when 9 or fewer questions appear across all attempts within a quiz, the statistics page would display the list of chosen questions beneath the corresponding set of question placeholders in addition to any static questions (true or false, multichoice, essay question types, etc).
When 9 or more questions have been used in attempts, Moodle shows placeholders then three rows of statistics for the pool of randomized questions, limiting the analytics of only a select number of specific questions, rather than viewing all analytics of all questions of which attempts were made.
See below the display of a quiz containing more than 9 randomized questions and the three selected questions displayed below the table:
The links on each question lead the user to the response analysis for that question. For example, if we click on the randomized question in position #13, the link will direct you to that question’s statistics but does not reveal the specific question used.
However, by clicking on question #14 you’re then shown all questions contained in the category and their analyses. Essentially, this created much confusion and difficulty in simply viewing randomized question analytics directly from the statistics reports page. While being unintuitive, the steps of accessing this data have mostly gone undocumented as well.
Quiz Reporting Upgrades in Moodle 3.6
Now in Moodle 3.6, the statistics report page displays an additional row containing a range of data. This row links to a full list of all analyses of randomized questions associated with its corresponding category. There are no longer direct links to each randomized question in order to consolidate a single, accessible source of all statistics.
Clicking on View Details will then direct you to an overview of the structural analysis for all questions, from where you can then drill down further into the analytics of each question. This will be revisited in the overview of the report.
For those who are unfamiliar with this report, you can locate the Quiz Statistics report by entering the Quiz activity and through the Quiz Administration menu, (accessible via gear icon at the top right corner) first selecting Results, then clicking on Statistics.
This report provides a statistical analysis of the quiz and the questions within it. The top section of this report, Quiz information, gives a summary of the whole quiz such as:
- Quiz name
- Course name
- Number of complete graded first attempts
- Total number of complete graded attempts
- Average grade of first attempts, all attempts, and last attempts
- Average grade of highest graded attempts
- Median grade (for the highest graded attempt)
Quiz Structure Analysis
The following section, Quiz Structure Analysis, displays all questions in a table. It contains links to edit individual questions or dig deeper into the detailed analysis of a specific question by either clicking on the Question name (for static question types) or by using the View details link (specific to randomized questions and their respective categories).
The View Details link will be present at the very bottom of the corresponding category table.
Viewing details of a randomized question provides access to analytics of all questions chosen within the larger selection of randomized questions.
Clicking on the question name will then further breakdown into three tables: Question information, Question statistics, and Analysis of responses.
Question information details the following information:
- Question name
- Question type
Question statistics displays the following data fields:
- Attempts – How many students attempted this question
- Random guess score – The score the student would get by guessing random
- Intended and Effective weight – Intended weight is simply what you set up when editing the quiz. If question 1 is worth 3 marks out of a total of 10 for the quiz, the intended weight is 30%. The effective weight is an attempt to estimate, from the results, how much of the actual variation was due to this question. So, ideally, the effective weights should be close to the intended weights
- Discrimination index – This is the correlation between the score for this question and the score for the whole quiz. From this data, you might determine students who score highly on this question score highly on the whole quiz. Higher numbers are better.
- Discriminative efficiency – Another measure that is like Discrimination index.
Analysis of responses lists all possible responses to the question. This includes percentile of users given partial credit for their submitted answer, count of users that have selected that response, and the frequency in which it’s been selected. The count could perhaps be higher at some points depending upon whether the question allows for more than one answer to be selected (check all that apply) or if multiple attempts have been enabled for the quiz.
The final section of this report is a bar graph illustrating the percentage of correct answers submitted, labeled Facility index, and the Discriminative efficiency index.
By utilizing the improved Moodle quiz statistics and reporting in Moodle 3.6, educators can better track, analyze, and utilize learning data in order to increase test results and ensure student success.
Explore more in Moodle 3.6
View the What’s New in Moodle 3.6 webinar below to explore our Support Team’s favorite Moodle 3.6 features. Or, request a demonstration to see it for yourself!