Cognition Knowledge Base

Questions Analysis Report

The Test Question Analysis Report allows you to review the frequency of answers to test questions, to see which questions are most commonly missed and what users thought the correct answers were. This could help you identify gaps in knowledge or perhaps questions that are misleading or confusing.

Running the report

To run this report, first navigate to the Question Analysis page found under the ‘Reports' menu. Next, to the right of 'Test’ click Select.

A pop-up will come up where you can search for a test by code, name, or keyword.

Once you have a test selected, you can optionally further narrow your results:

  • Only include test attempts from users in certain Divisions, Regions, or Locations.

  • In some cases, a Test might be re-used in multiple modules. If you don't wish to include results from certain modules, you can exclude them by unchecking the boxes at the right.

  • Only include test attempts from a certain date range.

At the bottom, you can also choose to include certain questions that are omitted by default.

Only you’ve made all your selections, click Generate Report!

Reading the output report

In the results, you’ll see an overview at the top, and each question listed below.

For each question, you can see:

  • How many times a question was displayed to users

  • How many times each distractor was selected

  • A percentage value for how often a distractor was selected. (The correct answer appears in bold.)

Note that when someone makes a change to a test question, such as changing the text for the question or one of the distractors, the old version of the question is archived and the statistics for the new updated question are then tracked separately.

You may be able to use these statistics to identify stronger and weaker areas of employee training, and make changes to the training curriculum to better cover any gaps you are seeing. Or in some cases, it could be that a question just needs to be re-phrased or re-framed to make the questions clearer or more fair.

Got feedback? Have a question? Let us know!

Some Cognition features may require additional licensing. If your organization is interested in a feature seen here, please contact Reflection Software for more information.