A couple of months ago, I conducted usability tests and did customer training for a software project.
I didn't reach the ideal number of users for a usability study (only four in this case), but I got plenty of interesting information from just watching and asking questions. Looking at people while they use your software is very enlightening for a software engineer.
What surprised me the most was that even though I took special care in designing a user friendly interface and I had a pretty good guess about the users' computer know-how, I still overestimated their ability to work with the software and missed opportunities to make things simpler. A lot of software engineers are just like me I believe, they don't have a good grasp about what happens when someone sits down and starts working with their software.
The number one recommendation that I can make is: do usability testing. Odds are that you'll have an epiphany and find many ways to improve your project. In the mean time, maybe you'll find the following thoughts useful:
- In order to do usability testing you need at least one user, preferably three or more. The trick is that it has to be a real user, someone that will use/is already using/is likely to use your software. You can't just go to Bob the tester and ask him to allow you to watch him. Sure, watching Bob will help, but you won't get any mind-blowing insights from him; he probably has a much too similar way of thinking to you.
- Explain to your test partners that you're not testing them, you are testing the software and they can't do anything wrong or somehow break the computer. This should make them feel more comfortable and allow them to be as natural as possible.
- Tell them why they're there, what your software does and what you hope to achieve with your test. Ask them what they think about the software at first sight.
- Try to be general with your instructions: for instance I wouldn't say "Let's open the invoice window and add three items by clicking the 'Add item button'.", I would say "Let's try to create an invoice" or "Let's do an invoice for three widgets. How do you think we could do that?".
- Watch carefully. Having the users think out loud helps, but I managed fine just by telling them to stop whenever they'd like to ask me questions or point out something that grabbed their attention. Steve Krug recommends that the session should be recorded. Pen and paper are acceptable in my experience, even if you have to stop for a few moments while you're writing.
Having done more than ten usability sessions so far, I've made some general notes about what works and what doesn't when it comes to UI. I'll publish those in a following post.