A Caveat on Tests and Scores

For many years I've included a Caveat on Tests and Scores in my psychological evaluation reports, before I report the test results. Here's the current version, taken from a report in process.

Before the Test Results: A Caveat About Tests and Scores

Psychological tests are useful tools as long as we remember their limitations. They are sampling procedures whose goal is to elicit examples of behavior which represent how the person being tested functions on those abilities which the tests seek to measure. The people who make the tests try to select items which are representative of those abilities, and to name those abilities accurately. Tests are interpreted by comparing individual results to how a broad sample of comparable people have responded to the same sets of items. There are thus several steps in the process at which test results can be misleading: test items might not be representative of the abilities being assessed, the abilities being assessed might not be accurately named, and the normative group on which the test was statistically standardized might not be representative of the population to which the individual’s results are being compared. In simple language, tests might be more or less well constructed, accurately named and meaningfully interpreted. All tests are approximate and all are works in progress. Each has its own history of evolution through theory and practice, and will continue to be modified and superseded.

“Intelligence tests,” for example, would be more realistically named, “Samples of Some Aspects of Intelligence.” Measuring human ability is not like measuring a plank of wood, even though we call them both “measurement” and express both as numbers. No one has ever seen “intelligence,” or the difference between one point and another on an I.Q. scale; intelligence is always inferred from samples of observed behavior.

Unlike the measurement of a plank of wood, the scores of psychological and psychoeducational tests nearly always show some variability. The scores of a person who takes the same test at different times, or equivalent forms of the same test, will probably not be quite the same. Similarly, a person who takes two different tests which are supposed to measure the same thing will nearly always receive somewhat different scores.

Test results that are presented as approximate grade or age levels can be affected by many factors, such as the availability of the child to instruction at the time the subject or skill being measured was learned, and the person’s functional abilities, then and now, in attention, receptive language, and working memory. Test results also may not reflect the potential benefit of repetition, study, or remediation, so they may not reflect overall ability. Considerable controversy exists about whether it’s even possible to have nationally accepted age and grade equivalent standards. In addition, a nationally balanced normative group, if such could ever be constituted, may not be representative of the level of academic achievement expected in, or typical of, a particular locality, so students might score higher or lower depending on the particular education they’d received.

Thus, it is usually the overall pattern of scores which are important, how they relate to one another and what they mean about how the person’s performance relates to that of his or her peers, and not individual scores.

It is also important to remember that scores might not always mean what we assume they do. Careful attention has to be given to the reasons why a child makes errors on some items. For example, a child may make an error on an item involving phonological processing because of a true phonemic processing deficit, because of a syntactic mistake rather than a phonemic mistake per se, because of lack of comprehension of what the item was asking of him or her, because of a lapse of attention, or because of some combination of factors.

Furthermore, the nature of intelligence and personality are still being researched, reconstrued, and debated, and the tests which we use to assess them are, of necessity, imperfect. Abilities such as practical intelligence, social intelligence, problem solving, creativity, and what might be called “self intelligence,” meaning ability to manage one’s emotions and behavior and improve through reflection on one’s experiences, are largely ignored by existing tests, but are vitally important for success in life. Wisdom, a vitally important characteristic, is virtually unmeasurable by any extant test.

In summary, psychological tests are useful to help develop an impression of how a particular person is functioning in some ways at a particular time in his or her life, as long as we keep their limitations firmly in mind.

My Friend Talks of Suicide

“Laura” writes:

Dear Dr. Einhorn:

I am a 13 year old girl with a friend that talks of suicide. She
refuses to tell her parents, and I live too far away from her to talk
to them. I know she needs help, but she will not talk to her parents.
Will you please see if there is anything you can do?
I realize you are very busy, but if there's anything at all, please
respond to this email.
Thank you so very much,
Laura D.

Dr. Einhorn replied:

Well, you could call her parents, Laura. Or you could talk to your parents and ask one of them to call her parents.

Laura replied:

Yes, however they don’t believe she is serious, and my parents have never met her before, as I met her at summer camp.

Dr. Einhorn discusses:

Laura’s concern for her friend and feeling of helplessness come through clearly in her question, and this is often the position that friends, family, and even professionals are in when caring for adolescents, or adults, who are, to any degree, suicidal. There is lots of help that can be brought to the suicidal person--including therapy, medication, outpatient and inpatient programs, and support from friends and family--but often the person herself, or, as Laura indicates is the case here, her family, don’t take the risk seriously enough. And it is a serious risk: suicide is rated as the third largest cause of death for adolescents.

Of course, we can only talk in generalities here, because we have no information about the people involved. We don’t know, for example, whether Laura’s friend is talking about suicide as a way of expressing her emotional overload, maybe blowing off steam, without really intending to do anything (in which case she still seems to be in need of help), or whether she is seriously at risk for self-harm. So, the question becomes, not what Laura can do for her specific friend, but what someone in Laura’s position might consider doing to follow through on her concern.

Laura feels like there’s nothing that she can do and she’d like me to step in and do something here. But this is a situation in which I have no “standing,” to borrow a legal term; I don’t know any of the people involved, and they don’t know me. Laura is the person who knows the girl who talks of suicide, so, like it or not, she is the person faced with the choice about what to do. She’s reached out to me, hoping to pass this “ball” to someone who can carry it better than she can, but the truth is that no one can carry it better than she can at the moment. That’s one of the problems and challenges of knowing someone who is suicidal. If you do something to try to help them, your attempt to help might be rejected as inappropriate, interfering, or silly. If you don’t, and the person does make a suicidal attempt, or, actually commits suicide, you might regret not at least having spoken up in some way to try to help. There is no easy way to deal with a situation like this.

I can think of a couple of options that Laura might consider, and readers might think of others. She could advise her friend to talk to a school counselor, who would then be in a position to evaluate the seriousness of her talk and call her parents to advise them to get treatment for their daughter. This might be the easiest path for all concerned if the girl would talk to the counselor. If she won’t, and Laura is still concerned, she can still talk to her parents and ask them to call the girl’s parents, even though they don’t know each other, to express parent-to-parent concern. That might help the friend's parents take a second look at their daughter's state of mind and risk for self-harm. Sometimes students will anonymously advise counselors in schools that a fellow student is making suicidal statements, but I don’t know that Laura would even know what school to send an anonymous letter to. And of course that could always boomerang on her, if it prompts intervention that the girl or her parents resent.

One issue that this raises is what the responsibilities and limits of friendship are. That’s a question that each one of us has to answer for ourselves, in different relationships, sometimes over and over again. It’s a real challenge, and no fun at all, to have to deal with this question in this relationship at this time, but that is the place where Laura’s life has taken her, even at her young age.

It’s a shame that adolescent suicide is such a problem in our society, and it’s worth our time to ask why that should be; what pressures are adolescents responding to? Answering that is beyond the scope of this Q and A, but I’ll appreciate any thoughts that readers care to contribute.

As for you, Laura, you are to be commended for your caring in raising this issue about your friend and reaching out for help, and supported as you seek within yourself for the wisdom to find, and the courage to do, the right thing; whatever, in this situation, that may be.

I've just read "The Writing On My Forehead," a novel by Nafisa Haji, and recommend it.  Written from the point of view of a feisty and questioning first generation American daughter of Indian-Pakistani Muslim parents, it covers a lot of ground about culture, family, morals, choices, and  traditions, within the context of a very readable story.  I was interested in the characters and, at one or two points, it bought a tear to my eyes.  

When a Privacy Guarantee Means You Give Yours Up

A colleague, on a professional listserve, reports that she was contacted by a hospital's development department after her daughter had been treated there.  "Hello Barbara," said the development caller (not real names).  This is Jonathan calling from ____ Hospital.  How are you tonight?  I'm calling because your 12 year old daughter, Laurie, was a patient here.  Is that correct?"  Jonathan went on to ask Barbara for a donation to the hospital.  

Barbara was appalled, and asked her colleagues whether this wasn't a violation of privacy laws.  Another colleague did some research and found this on the hospital's website:  "In the continuing effort to enhance _____ Hospital's mission of service, periodic communications and invitations to consider philanthropic support may be sent to patient families..."  This language must have been included in fine print on one of the required documents, probably the one that guarantees patient privacy.  

So a guarantee of patient privacy is actually a violation of it.  You think you're signing a privacy guarantee when you are actually giving up your privacy.

The linguist Geoff Nunberg, commenting on Terri Gross' "Fresh Air" program on NPR, highlighted a similar deceptive use of language when he said that most "guarantees" were actually "limitations of liability."  

This kind of misuse of language is actually a violation of trust.  And right now I am seeing violations of trust as being at the heart of many psychological problems in human relationships in our society.

Although the legal structure, pace of life, social decentralization, constant persuasion pressure from multiple sources, and technological sophistication of our culture bring it to a whole new level, the basic problem of violation of privacy is a very old one.  Idries Shah, in his "Caravan of Dreams," quotes a tradition of the Prophet Mohammed about this:  "Whoever invades people's privacy corrupts them."  

There's probably no more important psychological issue today than the reestablishment of public trust in institutions, and for that, institutions are going to have to become more trustworthy.  One of the chief changes that will require is the recognition of the importance of institutional honor in the relationships that institutions have with people at all levels; consumers, investors, workers, regulators.  

We have no trust in our institutions because they have no honor.  The hospital's treatment of Barbara is, in a word, dishonorable; not in that it asks patients for donations, but in that it slips the permission to do so into a document that purports to be a guarantee of their privacy rights.      

Clean Slate

We never start with a clean slate, because we always have a history that is embedded in the neurons of our brains and bodies.  We always start with a clean slate, because awareness is always now.  

Obstruction

Hanging my bathrobe in the closet, I felt a tug as the belt came out of the loops and fell onto the floor.  Looking to see what the belt had gotten snagged on, I discovered that I was, in fact, standing on one end of it.  And I wondered how many times we try to accomplish something that doesn't work out--in this case, hanging up my robe--and think that something in the environment is obstructing our success--whatever the belt had become snagged upon--when in fact the obstruction comes from ourselves.  We are, so to speak, "standing on one end of the belt while trying to hang up the robe."  Of course, it's because we haven't noticed in the first place that one end is longer than the other, and dragging on the ground...

Self-Respect and Self-Esteem

Some people equate self-respect with self-esteem, which is usually understood as having a good opinion of oneself.  But I would say that self-respect is saying what needs to be said and doing what needs to be done.