Requirements

Overview

Redesign, simplify, and make consistent how instructors see response statistics for quiz questions deployed from a randomized pool. Currently, the ability to view details in quizzes with more than nine random questions is undocumented and unintuitive. This project aims to make the layout and navigation of the statistics page consistent and clear regardless of the number of random questions. Fixing this problem would have a positive impact on the pedagogical effectiveness and overall attractiveness of Moodle.

Summary

  • Project size: medium
  • Audience: primary schools, universities, workplaces
  • Target users: teachers, administrators

Goals

The goal of this project is to improve the usability of Moodle’s quiz tool by making response statistics from randomized questions easier to access and consistent, regardless of the number of random questions that appear in a quiz.

Overview - Randomized question pools are an important assessment tool and a strong anti-cheating strategy, especially for online courses.  In Moodle, when ≤ 9 random questions appear across all attempts in a quiz, the standard quiz statistics page shows the list of chosen questions at the end of a corresponding set of question placeholders, amidst any non-random (i.e. "static") questions in the quiz. The links on each question name take the user to the response analysis for the question.

Fig 1: Current display of question stats in quizzes with ≤ 9 random questions displayed

the layout of the quiz statistics page when nine random questions are shown in a quiz

However, in a quiz where all the attempts result in > 9 distinct questions being shown as part of random questions, Moodle shows placeholders and then three rows of statistics for the entire block of random questions. Importantly, Moodle does not present the instructor with an obvious way to see which questions were shown or to view the kind of analyses that are more easily available in quizzes with fewer than nine total random questions displayed.

Fig 2: Current display of question stats in quizzes with > 9 random questions displayed

the layout of the quiz statistics page when more than nine random questions are shown in a quiz

Notably, in this type of quiz, there is no obvious way to see the complete list of questions and/or view the analysis page for each. Moreover, in the image above, the link for "Random (Quiz 2)" in slot #13 goes to a general analysis of how students performed on slot #13 while the link for "Random (Quiz 2)" in slot #14, unexpectedly, goes to the listing of all the random questions displayed as part of that block, with obvious links to each analysis. In other words, the same link text ("Random (Quiz 2)") goes to two different places with no indication as to why that might be.

The tracker at https://tracker.moodle.org/browse/MDL-48092 has an illustration of the difficulty instructors face trying to learn more about how their students fared on specific questions.

Use cases

Instructor creates a quiz containing 50 random questions drawn from a pool of 75 questions.  After the quiz closes, the instructor would like to see how students fared on specific questions.  The redesigned statistics page being proposed here would have a link on it that is clearly labeled to take the instructor to the regular “Structural analysis” page listing all the randomized questions used in the quiz.  At this point, the process would work as it currently does: the instructor clicks the link for a specific question to see the relevant statistics.


Links to existing tracker issues, forum discussions, contrib plugins

This proposal would close existing tracker MDL-48092.  Forum discussion related to this issue is here: https://moodle.org/mod/forum/discuss.php?d=316792.


Requirements

This project would require the quiz statistics page to be redesigned to treat all quizzes equally, regardless of the number of displayed random questions. In addition to an improved UI, the benefits of this are a simpler backend coding as there would no longer be a need to create two different layouts.  The likelihood of a quiz displaying ≤ 9 random questions is low, and so that instance should be subsumed into the layout of the more typical ≥ 9 instance, which will simply become the only layout for all quizzes with random questions.

Below is a screenshot of a suggested redesign.  There are three UI aspects being proposed:

1. Disable the display of individual random question texts for any quiz, regardless of the number of random questions it contains, with the possible exception of the three questions that are displayed in the overall block statistics area.

2. Remove the links from the random question names in the placeholder area. They are confusing and do not result in meaningful information.

3. Insert a new row between the final placeholder and the block statistics with an obvious link to view the questions that were shown as part of that block. The existing "Structural analysis" page is sufficient as the destination for this new link.

Fig 3: Proposed layout of quiz statistics page with improvements for accessing random question responses statistics.


Proposed redesign of the quiz statistics page

Conclusion

Thank you for reading this far! While it might seem that the issues raised in this proposal are purely technical in nature, they actually affect a significant pedagogical practice. The ability for instructors to adequately assess their students through the use of randomized pools of questions is important. Fixing something as basic as the quiz statistics page will go a long way toward helping them with that task.

Further reading

None

Last modified: Friday, 1 September 2017, 12:19 AM UTC