Choosing an Internet Assessment Vendor: A Practical Guide to Ensuring a Value-Added Partner

Over the last several years there has been an explosion in demand for Internet-based employee assessment solutions. Employee assessments in general are increasingly recognized as being critical for effective selection and development systems, and the advantages of the Internet are further increasing demand. Even in high stakes employee assessment situations (e.g., selection or certification tests administered in proctored settings), companies are seeking online solutions.

The number of Internet employee assessment vendors has likewise increased dramatically, and they differ widely in offerings, capabilities, technology, assessment expertise, customer support and other factors. Choosing the vendor who will best meet your needs and become a value-added partner can seem somewhat daunting. This Insight white paper provides some practical guidelines on meeting that challenge.

Do You Need an ASP?

Application Service Providers (ASPs) are firms that provide online applications and other services to their clients. Instead of purchasing or developing software and installing it in your organization, the ASP hosts and manages the application. Advantages of using an ASP include the following:

• Lower cost than developing and maintaining internal systems, and much faster deployment.
• Access to the latest technology. Good ASPs, with their need to remain competitive, offer leading edge technology.
• More powerful and robust administration and reporting capabilities than most internal systems would provide.
• Expertise in leveraging assessment information to improve organizational effectiveness.
• Allows clients to focus on core functions (e.g., selection, training, management development) instead of IT issues.

Depending on the ASP and a client's particular needs, disadvantages could include things like dependence on an outside party, inability to customize the application in very specific ways, uncertainty about customer support, concerns about data security, and the possibility of the ASP going out of business and leaving you stranded.

What may seem like another disadvantage — having to pay for ASP services — is usually not a factor because the total cost for doing everything internally is usually higher. Both the advantages and potential disadvantages, of course, depend on the ASP.
At any rate, before you go about the process of choosing an assessment vendor, you have to first decide whether you need one at all. It gets down to a build or buy decision. The following conditions would tend to increase the desirability of finding an ASP to partner with you:

• Having many employee assessment applications versus just one or two, assessments used for multiple purposes (e.g., selection and development), and a relatively high volume of assessments annually.
• Limited internal employee assessment and/or IT expertise (or capacity) to build and maintain your own applications.
• Need for employee assessment content that may already be developed (e.g., a validated selection test offered by a vendor) versus an employee assessment associated with an internal training program that you would have to develop anyway.
• Desire to implement a solution quickly, with low start-up costs.

Getting Started

Assuming you've decided an Internet assessment vendor makes sense for your business, the first step is to clarify (and document) your specific needs. This includes who's being assessed and why, requirements for employee assessment content (e.g., validation evidence for selection tests), the number of users and functionality needed, individual and group reporting desired, company access to the database, and a variety of technology issues.

Check with other departments in your organization to see if they may also have online employee assessment needs. There are strong advantages to having different functions working with the same vendor, and preferably that vendor having an integrated database and the ability to transfer information to the client's HRIS or LMS in a seamless fashion.

Criteria For Evaluating Employee Assessment ASPs

It is beyond the scope of this document to discuss the criteria for evaluating the content, psychometric properties, validity, etc. of the assessments themselves. However, those issues are more important than almost everything else, and should be carefully considered if you are seeking vendor employee assessments in addition to administration, scoring and reporting capabilities. An employee assessment that does not accurately measure what it purports to measure does no good, and may cause harm, no matter how well it is delivered.

Apart from the quality of the employee assessments that may be offered, what are the criteria for evaluating ASPs? The evaluation form included at the end of this white paper is a tool you may find helpful in answering that question. It is by no means comprehensive, but it does scope out the general areas you'll want to investigate.

You should first consider how important the criteria are in your particular situation. The ability of the vendor to handle foreign languages, for example, may be critically important, or not needed at all. Then add other criteria that aren't covered but are important to you.

Most of the criteria are self-explanatory, and are not discussed further here. However, please note the following points:

• In most cases, it is strongly preferable to partner with an employee assessment vendor who has both assessment and technology expertise in-house. This allows the vendor to offer a full range of services in a timely fashion, and to proactively look for ways to add value rather than merely respond to your requests.
• Where the employee assessment program is intended for development purposes, make sure the vendor can easily make the assessment information available to line managers at various levels. You hold managers accountable for developing their people; a good employee assessment platform can help them do that.
• A good vendor should be able to provide you with two important documents: a clear description of all the features and customizable options available, and a technology document that describes the architecture, performance and reliability, security, backup procedures, etc. in detail. (You may opt to pass off the latter to your IT person.)
• Make sure the employee assessment vendor has both the expertise and the capacity to provide good customer support, including technical support, and that the service will be available when you need it.

Suggested Steps

Here are the suggested steps for selecting an ASP-based employee assessment vendor:

1. Document your requirements (as discussed on the previous page).
   
   
2. Identify potential vendors via the Internet, trade journals, and talking with your peers in other companies.
   
   
3. Conduct preliminary discussions with vendors and let them show you their online capabilities.
   
   
4. For the vendors who still look promising, send them your requirements document and ask them to respond (including estimated pricing). Ask for copies of the two important documents mentioned on the previous page.
   
   
5. Have follow-up discussions and, using your requirements document, dig deeper on any areas that aren't clear to you. Don't despair if no assessment vendor can provide everything exactly as you want it. There may not be one.
   
   Depending on the complexity of the technology you're considering, and on requirements for different systems talking intelligently with each other and the like, you may want to get your IT people talking with the vendors' technical experts.
   
   
6. Check references, preferably those who are using employee assessments in a similar fashion as you will be. There's a tendency to over-emphasize the value of vendors having experience in the same industry, but in most cases that isn't as important as how the employee assessments are used (purpose, volume, reporting, etc.).
   
   The evaluation form included along with this white paper may be helpful in checking references, but be sure to get answers to these basic questions:
   
   

   —	Does the assessment platform always work like it's supposed to?
   —	Is it always available and is it fast?
   —	Does the assessment vendor provide outstanding customer and technical support?
   —	Do they add value beyond just scoring and reporting employee assessments?

   
   
7. Pick the assessment vendor who looks most promising, and run through some type of pilot or initial implementation phase. Watch carefully how the vendor performs during this phase - it's a good indicator of how they'll perform in the future.
   
   
8. Establish a clear understanding (sometimes with a formal contract or a service level agreement) of the expectations, deliverables, etc. Get a good understanding of what your company must do in order for the vendor to fully meet your needs.

Building a Partnership

The process isn't over when a contractual understanding is reached with the assessment vendor. Work now begins on building a true partnership, which requires intentional effort by both parties. Here are some things your company can do:

• Help the assessment vendor understand the "big picture" — how the assessments fit in and support what you're trying to do strategically. This better positions the vendor to add value.
• Meet with the vendor every 6-12 months to review how well the partnership is working and ways it might be improved.
• Provide the vendor with feedback on ways the platform could be enhanced to better meet your needs. (Vendors' best ideas come from their customers.) Over time, your requirements may evolve as you and the vendor identify opportunities. (This is the reason the arrow in the diagram above is pointing from Partnership back to Requirements.)
• Look for ways the vendor might meet assessment needs for others in your organization. This may actually reduce your costs, as well as help your partner, since most vendors provide significant per assessment discounts for increases in total volume.

Internet Assessment Vendor Evaluation Form

© 2007 Censeo Corporation

Return to Downloads page