Assessment Tests and Their Reliability

Among the fundamental requirements for all psychometric tests are validity, objectivity and standardisation.

To be objective, when taking these tests, every observer or judge seeing a performance should arrive at precisely the same report. This then, reflects the extent to which personal errors (where two different people observing the same behaviour/responses are likely to record different scores as they see things from their own personal bias) have been avoided.

Assessment Tests and their Results

When testing pre-school children, for instance, rating scales might be filled out by teachers and parents. Different people filling out a rating scale for the same child can, however, give different ratings. This is not always a reflection of poor test reliability but rather reflects the different perspectives that different people have. Another source of error is "setting variance," where the parent might see the child at home while the teacher sees the child at school. The child is likely to behave differently in different settings.

Errors, Criticisms and Potential Problems with Psychometric Tests

There are two other possible errors which should be challenged by criteria measuring the reliability of psychometric tests and these are: variable errors and constant errors.

Variable errors arise from accidents and errors due to say, noise, heat or a pencil breaking during a timed test. Test-reliability takes into account the relative freedom from errors of this sort.

Constant errors come under the umbrella of test-validity which checks whether the test measures what it is supposed to measure and is therefore not influenced by constant error.

Other criticisms of Psychometric and Assessment tests include:

• Users not reading the test manual properly;
• Users misinterpreting the results;
• Poor application of tools, i.e. inadequate job analysis;
• Bias in evaluation of test takers i.e. gender bias or cultural bias;
• Unsatisfactory definition of concept to be measured.

Construction of Psychometric Tests and Test Reliability

Another factor which can affect reliability is the way items are written. If there is a range of difficulty among the items, then it may be considered fair to place the easier items near the beginning of the test or at least not to group all the difficult ones together.

Tests which ask for only one answer do not always allow for creative or divergent thinking. If the subject does not give the correct answer, it is marked as wrong even though there may be valid and intelligent reasons as to why he or she has chosen an alternative. Also, true-false items aren't as reliable as say, five-answer multiple choice items as this lessens the chance of response-set and scoring through guesswork.

Faking in Psychometric Tests

Faking is a term used when a person being tested deliberately attempts to alter the results perhaps by cheating or marking more than one answer in the boxes and because of this, a number of devices have been developed to help discourage or detect faking. Low internal consistency in a test would suggest that the test is poorly constructed, or that the items measure qualities other than the ones intended in the design of the test.

Bias or Discrimination in Psychometric Tests

In Psychometric tests individuals should not be discriminated against either directly or indirectly on the grounds of race or gender. Direct discrimination may occur when people being tested who fit a particular category are instantly differentiated against.

Indirect Discrimination occurs when tests or procedures used to assist in say, employment opportunities, will favour one group as opposed to another without this having anything to do with their capability to do the job for which they are being tested.

How Reliable are Aptitude Tests?

An increasing number of companies throughout the world using psychometric tests as an aid to recruitment and these tests. These tests do have built-in lie-detectors and are designed to maximise reliability. Even so it must also be considered that in some tests people can be coached and trained to do better at them. Those who administer the tests should use them alongside interviews and personal observations rather than view the results in isolation.

Sources:

• Annastasi A. Psychological Testing. Sixth Edition Macmillan Publishing Co., London. 1988.
  
• Cronback, L. Essentials of Psychological Testing.
  Harper International Edition, New York and London. 1970.
• Edenborough R. Using Psychometrics. Kogan Page Ltd, London. 1974.
• 
  Hayes, N. Foundations of Psychology. Routledge, London. 1994.
  
• Helmstadter G. Principles of Psychological Measurement. Methuen & Co. London. 1996.