Our business team was evaluating several options for a major software purchase. We’d done our due diligence and had a detailed checklist of all our requirements. We rated each potential purchase against our checklist and the final choice seemed clear. So we made the purchase.

Then we started using the software. It turned out to be one of the most troublesome applications I’ve ever used. Tasks that should have been simple were unnecessarily complex (I shared an example in this post). Errors were frequent and error messages were obscurely worded (e.g., “Object reference not set to an instance of an object”). Features that were supposedly included actually required a fair amount of programming—by our programmers, not the vendor’s—in order to work.

We learned a valuable lesson from this experience: you can’t expect to evaluate usability of software with checklist—especially if you’re getting your answers from the company’s marketing brochures or sales representatives. When you’re evaluating usable design, there’s no substitute for hands-on, real-life experience.