A feedback system to increase quality of professional assessments

Last post 10:03 am September 28, 2015
by Gregor Lorenz
1 reply
Author
Messages
09:12 pm July 16, 2015

One of Forum Rules of Engagement is “No posting Scrum.org assessment questions and answers on the Scrum.org forums.”

Is it possible to implement a feedback system in professional assessments system?

Just like the “Bookmark” function, test taker can check a “Comment” check box if the question is ambiguous or unclear to her/him.

When the test is done, give 5 or 10 minutes to test taker for adding comments or feedback to each checked questions.

The professional assessments are not merely tests of knowledge, but assessments of understanding. But, sometimes, coming up with good questions can be harder than coming up with good answers. A feedback system always increase transparence and quality.

10:03 am September 28, 2015

I totally agree. Feedback helps to improve - it's at the heart of scrum.

In another post, I recommended to add a "sure wrong" ansering option (not just radioi button). This can both be used as a reminder when a question is bookmarked and also demonstrates knowledge. A couple of years ago, I implemented a certification process at a customer. The team concluded that the ability to eliminate "wrongs" is also worth something. If only "wrongs" were eliminated, the answer was still good for half of the points. The rational behind is clear and it also keeps the examinees from taking a wild guess from the remaining options (whcih would result in 0 points if the quess is wrong, or in a lucky point if the guess is right).

In the cert process I mentioned, we also had some very strict rules how to formulate questions. One easy to apply rule is to prohibit negations e.g. "Which is FALSE? or "Which of the following is NOT a ...". This only makes the examinee unnecessarily think in circles and bears the risk that the FALSE or NOT are not recognized in a testing situation - especially if not capitalized or boldfaced. This gets even worse if the answering options to a negated question are true and false.

Talking transparency - it would be also great to know what the ratio of essay/m-chocices is in level II exams and see some examples that give a clue about the level of detail that is required.