The 360-Degree Feedback Validity Debate

We’ll examine the three crucial areas you must address to be certain that your 360 process is valid:

• Valid questionnaire design
• Careful preparation of participants
• Follow-up that leads to action

1. Questionnaire Validity

The LinkedIn debaters agree on one thing. 360-degree feedback is only as good as its questionnaire. Carelessly-written questions severely limit its value.

But there are opposing views about how to create a good questionnaire.

Statistic-driven validity

On one side, the techno-statistical people promote so-called “validated questionnaires” that allow them to collect responses to the same set of questions across a wide range of workplaces.

On the surface, it seems attractive to compare the responses in your organization to their averages. They claim their off-the-shelf questionnaires are superior to those created individually for each workplace.

Unfortunately, this one-size-fits-all approach could end up comparing both a small-town bank and a Wall Street giant to the same criteria.

Local validity

I’m with the professionals who are asking why you would want your workforce to emulate the results of a vast group from which they are likely to be significantly different.

They believe that the only true test is local validity. So they ask this: do the questions communicate effectively in your specific situation?

As it happens, it requires only a few simple steps to produce a highly effective, locally-appropriate, totally valid questionnaire:

Keep it short

If it exceeds 40 questions, responders will get tired, providing less thoughtful answers.

But resist the temptation to combine two thoughts into one confusing question.

Make it relevant

Restrict your questions to your organization’s core competencies, job-related tasks, and areas of significant concern to your employees.

Ask yourself, are the responses going to lead to action? A valid questionnaire will identify where the recipients are doing well and should continue on track, and where they need to improve – encouraging them to make changes.

Ensure it’s unambiguous

Check the questionnaire with representative groups prior to launch, including  responders, people being assessed, HR professionals, and senior management (since they’ll eventually use the aggregate data).

Ask them to flag any questions that aren’t relevant, and any wording that is not crystal clear. And you can ask your 360 provider for a professional evaluation.

That’s it! Creating a valid questionnaire is within every practitioner’s reach.

Next: We’ll describe how to prepare participants for a valid 360 process.

2. Preparation Of Feedback Responders

There’s a common misconception that you can ensure the validity of your 360-degree feedback project by building it on a particular questionnaire.

But the truth is that validity relies on much more than a set of questions. It depends, in fact, on one over-arching concern: Does the 360 process as a whole lead to the desired outcome?

That outcome is that the subjects (the people being assessed) become more clear about where they are seen to be most and least effective. With this information, they can be assisted in leveraging their highly productive behaviors, and improving the skills that need work.

One of the requirements for a valid feedback process is good preparation of the people who will provide it, touching on three major topics: Why it’s valuable to provide feedback. The commitment to confidentiality. And how to give critical feedback without alienating the person receiving it.

Why it’s valuable to provide feedback

The motivation of responders is crucial for success

A good information program informs responders that feedback is not only a generous gift to the subject, but that they themselves will receive value from providing it.

They should be reminded that subjects who receive feedback will become better employees, bosses, and colleagues. Feedback will help make the work environment more positive and work relationships more satisfying.

Commitment to confidentiality

They also need to know that they are safe, meaning that they can provide frank feedback anonymously without fear that their identity will be leaked to the subject. After all, that person may be their boss or a powerful colleague.

They should be reminded not to use language in their responses that could identify them.

(The exception is that the subjects’ supervisors do not require confidentiality, since they are not vulnerable in the same way. In fact, providing identifiable feedback is part of their job description.)

The commitment to confidentiality also means that the administrators who work on the 360 project will not attempt to identify who said what.

Ok, as a responder I’ve been informed that feedback is valuable, and that I’m protected by a firewall of confidentiality. So does that mean I can empty both barrels at that subject I’m so angry about?

Next: How responders can provide feedback that’s critical but doesn’t alienate the subject.

3. 360 Validity: Honey and Vinegar

Today we look at a key step in developing valid 360-degree feedback.

One of the toughest parts is preparing responders to provide feedback that’s critical, but in a way that the recipient won’t reject or ignore.

Feedback that’s hearable

“You’re an idiot who should be fired,” will not help anyone develop their skills.

Feedback is effective only when it meets three criteria. 1. It must be candid. 2. It must address not unchangeable personal attributes but specific behaviors that can actually be changed. 3. It must be kind.

Nowhere is the saying more true that you attract more flies with honey than vinegar. The problem is that it’s hard for responders to be kind when they have been frustrated and angry for a long time.

Instinctively, they’re tempted to lash out. But their dilemma is that if they do, their feedback will be discounted.

Brief training sessions are most effective

Remind responders that feedback can make even the most secure individuals anxious. And help them remember how they themselves reacted when they were criticized in a rude, exaggerated, or disrespectful way.

Of course they got angry, dug their heels in, refused to change, or totally ignored the feedback.

And, by the way, if you can’t provide in-person training for responders, your minimum responsibility may be to provide a brief written guide.

Candid, kind, and specific

Help responders discuss how to make their point without alienating the recipients. They should point out where the recipients have been effective, and where they have not.

Useless: “You’re a terrible leader.”

Valuable: “You’ve been pretty effective when you communicated openly with the team. But we should be asked for input on important matters more often.”

Useless:  “You play favorites.”

Valuable: “It’s frustrating that you give the interesting work to a few of your old favorites, and hand the rest of us the drudgery. We can’t be creative, and no one sees how capable we are. Give us all the chance to shine.”

Help your responders understand how to make their point without alienating the recipient, and you’re well on the way to a feedback process that’s unquestionably valid.

Next: How good preparation of recipients contributes to a valid feedback process.

4. Validity Of 360 Depends On Recipients

We’ve been addressing the myth that the validity of 360-degree feedback rests on the so-called “validated” questionnaire.

First we established how you can to prepare your own valid questionnaire.

Then we broadened the focus to motivating responders and helping them provide feedback in a form that’s accessible to the recipients.

Now, how the preparation of the subjects (recipients) contributes to validity.

The most effective method of preparation is to hold small meetings, so subjects can ask questions, discuss their concerns, and experience support. (If meetings are not possible, the following information should be provided in writing.)

Value of feedback to the organization

Subjects are often anxious about getting feedback. One way to counter this is to tell them why the organization invested in 360-degree feedback.

Clearly it was not to embarrass anyone. Or increase anxiety. Or find an excuse to fire them. Or provide a weapon for people with a grudge. Or focus on things they can’t change about themselves.

360-degree feedback helps the organization become more efficient, improve the workplace atmosphere, and encourage a culture of personal growth and satisfaction. These, of course, are the marks of organizations that are responsible, respected, efficient, and profitable.

Value to the recipient

At the individual level, everyone wants to perform at the best level possible. 360 provides subjects with explicit guidance about how changes in their work habits can improve their productivity.

Not only can it increase their perceived value within the organization, it provides the data they need to improve their career prospects.

As well, because most adults spend half their waking life inside the community of the workplace, 360 offers significant social benefits.

It helps subjects discover how they are seen by others. It points out how they can improve the work atmosphere and strengthen their relationships.

It’s win-win-win.

Finally, by expanding feedback beyond the traditional supervisor’s perspective, it provides a more broad-based perspective on their skills.

Next: How coaching subjects, post-360, increases the validity of the process by empowering positive change.

5. Effective Coaching Makes 360 Valid

There’s great fascination these days with the issue of validity in 360-degree feedback, so this final installment in a series about best practices for validity tackles the importance of supporting the recipient, post-360.

The challenge of feedback

When we receive our feedback reports, it’s normal to experience a range of reactions: anticipation, excitement, and satisfaction may be accompanied by discomfort, resentment, fear, and anxiety.

With such powerful and conflicting emotions, it’s easy to get stuck in negativity. A single critical comment in a 360 report will often preoccupy our minds.

We mutter ineffectual questions like, “Who said that?” “What do they mean?” “How can I get back at them?” “Does everyone think that?” “How will this affect my prospects?”

That’s particularly the case when we receive feedback on our own, and it reduces the validity of the process.

Coaching for positive change

To combat this tendency, it’s essential that we receive coaching as soon as possible after we receive the report.

Whether the coach is our manager or a trained professional, s/he can help re-frame our feedback not as a threat but a gift.

The effective coach interrupts a negative cycle of thinking by encouraging us to sustain our positive behaviors.

What can we change?

Validity measures the extent to which a process leads to positive change.

So the coach will steer discussion away from those things we can’t change (like personality qualities) to those we can (behaviors). The coach will help us figure out how to do things differently.

The validity of the 360 increases when the coach can point to resources that support sustainable growth, like mentoring and courses.

Subsequent coaching conversations encourage us to be accountable for our own development and fine-tune our efforts.

A final word

It’s tempting these days to focus on the dramatic but self-interested claims of the promoters of so-called “validated” questionnaires. But there is much more to validity than questionnaires.

Validity in 360-degree feedback includes the preparation of recipients and responders ahead of time, careful fine-tuning of the questionnaire to suit your organization, and effective follow-up with recipients.

Cover these areas, and you’ll have a 360 process where the validity can be measured by positive results.