Recent Breakthroughs in 360 Feedback Software:  Ease of Use, Versatility and Affordability

by Dennis E. Coates, Ph.D.

 

Today, desktop computers let organizations do simply and economically something that used to be cumbersome and expensive:  administer multi-source (360 degree) feedback.  HR professionals should be aware that 360 software has taken several leaps forward, and important new capabilities are now possible. 

 

For the HR professional, 360 feedback is one of the most useful tools to come along in a long time.  It creates reasonably objective, quantified data about vital areas of professional competence that are otherwise hard to measure.  These are the interpersonal aspects of work:  leading, managing, supervising, coaching, coordinating, communicating, interacting, contributing, selling, instructing, negotiating and consulting.  The rich feedback that flows from the 360 process helps people decide how to use valuable time, resources and effort to improve their effectiveness in these areas.

 

Multi-source feedback involves so many calculations and administrative steps that computers are a necessary component of the process.  The external scoring services introduced in the early 1980s made 360 possible for organizations.  The first publishers developed instruments and the software to score them, and they routinely charged about $200 per assessment to generate a report.  Because of the expense, 360 feedback was offered mostly to upper-level managers. 

 

Executives who have experienced multi-source feedback have affirmed it as a momentous step toward self-improvement - an approach that would facilitate the development of all employees.  But for many years, surveys were focused mostly on management and leadership; and costs made the widespread use of 360 impractical.  Today, advances in feedback software have made implementing 360 throughout an organization a viable option.  The most important developments are in three areas:  

·    Ease of use

·    Versatility

·    Affordability

 

The new capabilities are highly beneficial.  However, older programs are still on the market along with "next-generation" software, so its harder than ever to determine what's possible and to decide which approach will best serve an organization.  With more than fifty options from which to choose and no two configured or priced the same, comparisons are difficult.  This article will help eliminate some of the confusion by highlighting the most significant improvements.

 

Breakthrough #1:  Ease of Use

 

In the early years, 360 feedback was not only expensive, it was an administrative nightmare.  After receiving materials from the publisher, administrators had to collate, address and distribute packets to respondents.  They had to keep track of returned forms, checking each one for correctness.  Then they resorted, bundled and returned the forms to the publisher.  With luck, no problems held up the scoring process, and reports arrived on time.

 

On-site scoring.  Today, organizations everywhere have the PCs and laser printers needed to score their own 360 assessments and produce high-quality feedback reports.  While a few publishers still make scoring services their main business, most now license administration software for on-site use.  This puts control in the hands of the organization while reducing some of the time and hassle of administration.

 

On the other hand, some on-site scoring systems still use paper forms and scanners, so they require specially printed forms, scanner hardware and interface software.  Experts are needed to assemble such a system, get it running and explain how to use it. 

 

Paperless administration.  Until the 1990s, most 360 assessments used scannable forms, paper reports and development planning workbooks - an expensive, painstaking approach that made customization difficult.  The elimination of paper from the 360 process was a major breakthrough.  The ability to put assessments on diskettes took the complexity and cost out of customizing assessments.  It also eliminated the need for the extra hardware, software, training, time, labor and other expenses related to scanning forms. 

 

After diskette-based 360 came network-based 360 - the ultimate paperless platform.  Administrators distribute and collect assessments from all parts of the organization - even from remote locations, using Internet e-mail.  Some programs let an administrator complete entire projects without using any physical media.  The key is to select a program that’s compatible with the server platform (Novell, NT, HTML, etc.) and that produces diskettes or paper for people who don't have access to the network.

 

Some publishers have introduced web-based 360 scoring services.  This method also eliminates the need for scannable forms, but it requires that respondents have access to the Internet, a limiting circumstance in most organizations.

 

A few organizations use phone keypad systems.  Instead of forms, diskettes or networks, people enter feedback using the touch keys of a standard telephone.  This form of data entry makes automated 360 possible for organizations that don't have computers, but many people find this method slow and awkward, and the phones inability to accommodate comment feedback has made it a relatively unpopular option.

 

User-friendliness.  Not all 360 programs have the same capabilities.  Some have complicated set-up procedures; others use wizards.  Some require expensive customization, configuration and training; others install in minutes and are simple to use.

 

It's important to think of multi-source feedback as more than software.  Diagnosing strengths and weaknesses is a key first step toward improving performance.  A successful 360 process involves a number of actions before, during and after assessment.  Some 360 packages contain little more than software and instructions.  Others include development planning workbooks, which vary in thoroughness and usefulness.  In addition to participant materials, organizations need instructions, texts, references, lesson plans, worksheets, handouts and presentation aids to support each phase of the process.  Ideally, a multi-source feedback system contains all these resources.

 

Important questions to Ask About

·     Is on-site administration software available?    

·     Does the administration software eliminate the need for forms and scanners?    

·     Can it be installed in a few minutes without help?    

·     Is the program simple enough to be used without training?    

·     Does the system include all the resources needed before, during and after assessment?    

·     Does the software make it easy to customize survey parameters in-house?    

·    Does the software include on-screen reporting and development   planning programs?

·    Will it work on your network?    

·     Does it support paper-based assessment for people who don't have     computers?

·     Does it produce respondent diskettes for people who aren't connected to a network?

 

Breakthrough #2:  Versatility

 

First-generation 360 tools evolved from the tradition of psychological assessment.  Publishers claimed that their instruments were valid for all managers, regardless of industry or type of organization.  They featured a fixed set of management dimensions, and organizations were expected to use all survey items without modification.  HR professionals had to decide which instruments were backed by the most convincing research.

 

Customization.  Rigidly constructed instruments no longer appeal to most organizations, which are demanding surveys that reflect their unique culture and practices.  Most publishers have responded to this need; but in order to customize a paper-based assessment, they have to reprogram proprietary software and print new test booklets, which creates additional expenses for the user. 

 

On-site paperless 360 software has significant advantages.  Built-in commands let users specify locally validated survey items, scales, perspectives and report formats quickly and as often as needed - without forms and the accompanying expense.  Administrators can then copy the newly customized assessments to diskettes or make them available to respondents on the network.  Some systems let the user load any set of competencies on the software, making it easy to assess any desired area of workplace performance. 

 

All employees.  During the past decade, organizations have focused on developing effective teams:  executive groups, project teams, committees and work units of all kinds.  Members of these groups began asking for feedback from each other, not just from their boss.  Because of the way 360 focuses on priority areas for improvement, organizations have discovered the value of using it with employees at all levels.  As a result, many 360 publishers have introduced instruments that assess personal leadership, communication and team interaction skills.  And the ability to load any locally validated set of performance measures onto 360 software makes it possible to assess people in all types of positions.

 

Multiple applications.  Some 360s are designed for one purpose:  executive development.  Others are virtual assessment platforms, with a library of surveys to suit a range of needs, such as team effectiveness, competency development, needs assessment, performance management, evaluation of training, customer satisfaction, market research, employee attitude, organizational climate and organizational culture.  Some 360 systems will even process locally produced surveys. 

 

Report formats.  Most organizations have specific requirements about the kind of feedback given to their employees.  For example, a person receiving feedback may want detailed information, but the individuals supervisor may need only summary information.  Reports used for leader development are different from those used for group surveys. 

 

Most 360 systems feature three to five report sections presented in a standard format and sequence.  A few permit selection from a range of displays.  Some publishers charge extra for special reports, such as consolidated group or organization summaries.  The most advanced concept is a report format designer, which lets the user create and use original report formats in-house, at no charge.

 

Comment feedback.  Most people feel that numbers don't tell the whole story.  When areas are considered strong, they want to know why.  Scores may highlight opportunities for improvement, but people also want to know what coworkers expect.  End-of-report comments can supplement numerical feedback.  A few 360 systems make it easy to enter a variety of comments on-screen with each rating.  This capability not only eliminates transcription costs, it lets users display comments with item scores.

 

 

Developmental recommendations.  While its helpful to know which areas of performance need improvement, the next step is deciding what to do about it.  Some 360 systems include carefully researched developmental recommendations.  The best of these resources go beyond vague generalities to give detailed suggestions for low-rated behaviors.   They also let HR personnel add local resources to the suggestions.

 

On-screen development planning.  Development planning workbooks help feedback recipients analyze their feedback and set developmental goals.  A recent innovation is development planning software.  The most useful programs let the subject view, analyze and print the 360 feedback, as well as set goals, create a development plan and monitor progress.

 

Important Questions to Ask About VERSATILITY

·    Which survey parameters may be customized using the administration software?

·    Is a library of standard researched surveys included in the system?

·    Can survey items be added, deleted or modified to suit local requirements?

·    Can locally produced surveys be added to the system?

·    Can the system be used for organizational surveys, as well as individual 360 feedback?

·    Does the system make it easy to give comment feedback?

·    Are item-specific comments sorted and reported with item ratings?

·    Are detailed developmental recommendations available for frequently used surveys?

·    Does the system let the user design and save new report formats?

·    Is individual development planning software included, and what are its capabilities?

 

Breakthrough #3:  Affordability

 

While external scoring services traditionally cost in the range of $150 to $300 per assessment, on-site 360 involves two separate expenses.  The first relates to the initial installation--a fixed, one-time investment.  "Paperless" on-site systems are more automated and involve less hardware, so they are less expensive to install and use.

The second expense is the cost of assessment.  To facilitate assessment, software is generated by the system and given to every respondent.  The more advanced systems also contain reporting and planning software for every subject.  Thus, the cost typically varies according to the number of assessments.  Today, most organizations can administer on-site state-of-the-art 360 feedback for about one-fifth the typical expense of external scoring services.  This makes 360 affordable enough to be used with all employees in any kind of organization.

 

The biggest economies come from avoiding extra costs.  Many publishers have separate pricing for highly desirable features; others include them free.  High-tech typically means low-cost:  if a task is an automatic function of the software, there's no need to charge extra for it. 

 

Installation.  Most systems, especially those that use scannable forms, require front-end installation that can cost well over $10,000.  Others install and are ready for use in a few minutes - with no installation charges. 

 

Training.  Setting up a customized 360 project can involve setting a lot of variables.  If the software is user-friendly, this is a simple task.  If not, training may be needed to teach administrators to use the software.

 

Technical support.  Most publishers have an annual fee for technical support, because they expect users to have questions or problems when using the software.  For convenience, some publishers provide a toll-free number; others have no fees for technical support.

 

Upgrades.  Software improvements are a major cost to 360 publishers.  Most charge their customers for upgrades, although some provide upgrades at no extra charge.

 

Reports.  If the software generates reports automatically, there’s no need to charge for additional reports.  External scoring services usually charge for every report they produce.

 

Comments.  The comments of feedback providers often reveal more than the ratings.  The use of paper forms limits the feedback to end-of-survey comments and requires expensive transcription services.  Paperless feedback systems can collect a variety of comments automatically - with no costs passed on to the user.

 

Respondents.  Paper forms cost money - the more respondents, the more the expense.  Paperless systems make it possible to involve an unlimited number of respondents without adding to the cost.

 

Customization.  Most organizations want surveys tailored to meet their needs.  Rigidly configured surveys are hard to customize, and publishers that accommodate requests for customization must charge for reprogramming.  Software that helps users make changes eliminates these expenses.

 

Multiple Uses.  Long-term per-assessment costs are often the most significant expense.  If the software is flexible enough to be used for organizational surveys, market research and other applications, an organization can save money.  To further reduce costs, some publishers offer permanent, unlimited usage licenses.

 

Development Planning Guides.  The most important part of 360 feedback is what people do with it afterwards.  An individual development planning workbook is an extremely useful tool, although not all publishers provide them.  A few publishers offer individual development planning software that helps an individual review 360 feedback, analyze strengths and weaknesses, set goals and create a plan for self-improvement.  Automated planners can be more useful than paper ones, but they may involve an additional expense. 

 

 

Important Questions to Ask About AFFORDABILITY

·    What's involved in installing the 360 system, and what will this cost?

·    Do administrators need training to use the software, and what does it cost?

·    What are the charges for additional reports?

·    What extra charges are involved for comment feedback?

·    Is there a limit to the number of respondents or a cost for additional respondents?

·    Which aspects of assessment can be customized, and what will this cost?

·    Is a paper or electronic development planning guide available, and what does it cost?

Multi-source feedback, once used exclusively for executive development, is now a practical developmental tool for all employees. 

 

Advances in the ease of use, flexibility, and affordability of 360 software have made it possible.  With an awareness of these new developments, HR professionals can find solutions that will meet their organization's needs as they explore the continuously expanding array of options.

 

Copyright © 1998, Performance Support Systems, Inc.  All rights reserved.  Dr. Coates' article was published in slightly different form under the title, Breakthroughs in Multisource Feedback Software,  in the November/December 1998 issue of Human Resource Professional.  Used with permission.  This article may be reproduced for internal educational purposes only.  Embodiment of this material in products or resale in any form is strictly prohibited.