2008-09-10

Concerning Tests, Testing, and the Tested

All tests have underlying presumptions: 1) that the question is unambiguous, not open to interpretation, 2) that the result expected by the test designer is the best, or optimum, answer, and 3) therefore nothing is to be learned from an analysis of wrong answers across the group of testees.

In some ways, rather like a spell-checker, where an unexpected word is not analyzed to see if it is indeed the appropriate word based upon the rules of word-creation, such as 'testee' in place of 'test taker', for 'one who is tested'; more compact, and a perfectly valid pseudo-Latinate formation.

The above being an example of conflict between tester and testee, one who tests and one who is tested, with the one in a position to arbitrarily declare the other in error.

Tests designed to identify those who 'think outside the box' are doomed to failure; anyone who truly thinks 'outside the box' will think 'outside the box' they are attempting to place them in, unless the first selection criteria is 'identify those who answer other than we think best', and then start additional testing/analysis to determine which test 'wrong' because they really were in error, and which found something different in the question from what the test designers thought was there.

Standardized tests only work with standardized individuals. I dare anyone to prove me wrong on this!

I had some psychological/analytical testing done a few years back, and an entire section was thrown out because standard analysis stated that only a deliberate attempt to deceive could result in the 'contradictory' answers in that section. No analysis would be done as to why the answers were contradictory, it was self-evident that they were. No shit, I knew that as I answered them, but the questions related to a condition that existed for over twenty years, yet the underlying presumption of the questions was that all answers were for the same time instant. Thus, that I don't drink alcoholic beverages was in conflict with having a drinking problem and with friends noticing signs of alcoholism, because that can't be the case, right? Not for the same point in time, but the questions were scattered about the section specifically to prevent someone from tying them together while answering, to catch someone who is lying, in denial, and instead catch the person is being totally honest, because 1) it has been over ten years now since I have had any alcohol, 2) previously I drank rather heavily, and 3) friends who were themselves recovering alcoholics had recognized the symptoms in me. Without the context of time frame the answers are incompatible, and the test did not specify a time frame.

Unlike a test in an academic setting, where channels exist for challenging the results of a test, which I have used, successfully, there is no venue for such a challenge in these types of tests. And these tests have much greater long-term impact; it is now on my medical record that I attempted to deceive in matters which relate to my disability claim; instead of gaining insights into what's going on with me in regard to cognition, etc., I'm branded a liar.

All because of a poorly designed test. One could argue the test isn't poorly designed, but rather that the follow-up analysis/testing of those giving anomalous results is, in a word, lacking; I'd accept that except that I'm the one on the receiving end, and pretty words don't make it feel any better.

Their reasoning is akin to this: 1) Square pegs don't fit into round holes, 2) you don't fit into a round hole, 3) you are a square peg; no thought to triangular pegs, round pegs of greater girth, whatever other options there might be, just 'if a≠b, and b≠c, then a=c' which is arrant nonsense in a non-binary world. [Look, a qualifier!]

Post this Puppy!

No comments: