The Final Step: Evaluating Your Learning Technology Shortlist and Making the Right Choice

In this final article of our three-part series, we focus on the ultimate stage: evaluating your shortlisted learning technology vendors and making that all-important decision.

Now, that you’ve crafted a robust RFP and narrowed your options to a select few, it’s time to decide who gets the final rose - or contract.

Here’s how to approach this critical step with clarity, fairness, and a touch of practicality.

1. Who Do You Bring Into the Room?

The decision-making process isn’t just about the vendors—it’s about the team you assemble to evaluate them. Bring in the right mix of people who understand both the technical requirements and the business needs. This might include your key stakeholders aligned to your business case, L&D specialists, IT representatives, compliance officers, and HR stakeholders.

However, keep the group lean enough to avoid inefficiency and ensure everyone has a clear role.

Example Anecdote: One of my clients was reviewing vendors for a project tied to new legislation. During the evaluation, a compliance officer noticed that a vendor hadn’t read the brief and dismissed the project as “not compliance-related.” That slip-up annoyed the stakeholder so much that the vendor’s bid was dead in the water—no matter how good their offer was.

Who Do the Vendors Bring? Vendors often bring their team to these evaluations, but it’s worth considering if their presence is overkill. I’ve attended presentations where vendors brought an entire entourage—and unsurprisingly, they were also the most expensive quote. While it’s good to see a vendor’s depth, an overblown team can be distracting and inflate costs unnecessarily.

Key Tip: Prepare stakeholders in advance with clear scoring criteria and ensure everyone knows what to look for. Don’t let subjective preferences dominate the discussion.

2. Don’t Overload the Day: The Pitfalls of Cognitive Bias

Scheduling multiple vendor reviews in a single day is tempting, but it can lead to cognitive overload. People often fall prey to:

• Recency effect: The last vendor leaves the strongest impression.

• Primacy effect: The first vendor sets the benchmark.

• Decision fatigue: Too many options in a short period clouds judgment.

Best Practice:

• Limit vendor reviews to two or three per day.

• Schedule breaks to debrief and document thoughts immediately after each presentation.

• Rotate the order of vendor presentations if evaluations span multiple days.

Example Anecdote: Personally, a pet peeve of mine is when vendors waste time telling us about our organisation—we already know that! Another annoyance is when they spend too much time on slides and not enough on the product itself. It’s crucial to see how their solution will solve your problems, not just hear generic pitches.

Key Tip: Consider how vendors allocate their budgets. Do they spend excessively on flashy exhibition stands and grand customer events? While these can be fun, it's the equivalent of paying supermodels to endorse beauty products, the cost trickles down to you. Prioritise vendors who invest in their product and customer experience instead of extravagance.

3. Scoring Your Shortlist: An Example Template

A scoring system is essential for objective evaluation. Use both quantitative and qualitative criteria, assigning weights to reflect the importance of each factor. Here’s an example:

Qualitative Inputs to Consider:

• Responsiveness during Q&A sessions.

• How engaged the vendor was in tailoring their solution to your needs.

• Attitude during interactions (e.g., dismissive or collaborative?)

Key Tip: Align your criteria with your RFP and business goals to ensure you’re comparing apples to apples

4. Transparency and Due Diligence

Vendor transparency is critical at this stage. Ask yourself:

• Are they upfront about their limitations?

• Do they disclose their partnerships with other vendors, especially for complex integrations?

• How realistic are their timelines for implementation?

• How good is their project management? 

• How much onus is on you to project manage and can you resource that? 

Implementation Timelines:

• Greenfield implementations typically take around 12 weeks.

• Complex integrations, especially for systems like skills platforms, can take significantly longer.

• Factor in change management and ensure the vendor is clear about potential challenges. 

Key Tip: Weigh the hidden costs of delays or data migration issues. Vendors who sugarcoat these challenges often lead to frustration down the line. Add contingency in your budget for potential delays - rarely, there aren't any! 

5. Making Your Decision: Balancing Cost, Value, and Relationships

When making the final decision, consider:

• Contract negotiation: Ensure all costs are clearly defined, including licensing, support, and upgrades. Push for transparency on SLAs and penalties for unmet expectations.

• Long-term relationship potential: Ask about customer retention rates. Why do clients leave them? What’s their approach to post-implementation support? What might it cost you to leave them? How easy is it to get your data to migrate to another vendor? 

• Alignment with strategic goals: Ensure the vendor’s solution will continue to meet your evolving needs. Like we have mentioned before - do they deliver on their roadmap? 

Conclusion: The Final Checklist

As you evaluate your shortlist, keep this checklist in mind:

• Have you included the right stakeholders in the evaluation process?

• Did you avoid cognitive biases by limiting daily vendor reviews?

• Are you using a scoring system with clear criteria and weightings?

• Have you assessed the vendor’s transparency about timelines, integrations, and limitations?

• Are all costs and SLAs clearly outlined in the contract?

• Have you factored in the implementation timeline and change management efforts?

Final Thoughts: This is the final step, so stay disciplined and thorough. Choosing the right vendor is about more than just ticking boxes—it’s about setting your organisation up for long-term success. You’ve now done the easy part—next is where the hard work starts.

New to L&D tech? How does the vendor support you with change management, adoption, comms, and engagement?

For more insights or support in navigating this process, drop us a line - click Contact Us below. 

ConsultHer 

Website created by ConsultHer

+447800572110

Contact

ConsultHer

We don’t just consult.

We do.

We sense-make and enable organisations to realise their people and skills-based strategies, aligned to business goals

Privacy Policy

Copyright © 2024 ConsultHer | All rights reserved