|
What exactly is the "usability"
of a product? ñ
Usability is a measurable characteristic, that is present to a greater
or lesser degree, that describes how effectively a user can interact with a product.
A very general way of classifying it is to look at:
- how easy it is to learn
- how easy it is to use
Although it is inherently multi-dimensional, major elements of it are:
- Learnability
How easy is it for new users to find features and do common tasks? Can the user get a job done the first time they
sit down at a computer?
- Efficiency
Does the new product reduce the time to complete a task? Does the product match their existing work flow or obstruct
it?
- Memorability
How often do users need to re-learn a feature? How often do they need to consult a manual for a feature they have
used before? Are the operations of the product tangible to the user?
- Errors
How often to users make mistakes? Error rates are always related to product design (both of architecture and user
interface.) How can these rates be decreased by changing the design?
- Satisfaction
Did the user like your product? Was it a good first impression? Did they have a better impression of a competitor's
product? Is the product adding or decreasing stress on users?
- Productivity
Do they get more done before or after implementation of the product? Can their existing work flow be improved and
augmented to increase how much a worker can do? How much can you increase productivity while retaining user satisfaction?
- Training time
Does the product require training? Can you decrease this number? Some products are described as "walk up and
use" because they do not require a training time.
- Data input speed and interpretation of data
Can the structure of the product increase how fast you can enter information? How fast can you understand what
the computer is displaying for you? Are you missing important information?
- Technical support needed
A technical support call means the product failed for a user in one of the above categories. How can this be reduced?
- Maintenance costs
Is the product durable? Can it be made more stable? Can it be more simple? Can product updates be easier and faster?
What is the difference between a usability problem
and a bug? ñ
Usability problem: Part of the product design which is hard for a user
to understand, will impede the effective use and negatively affect the product.
Bug: A portion of the product which does not work as the designers and
programmers intended or which is clearly broken.
How do you find usability problems? ñ
Usability problems are often very hard to spot, even for professionals
with years of experience. This is true for a number of reasons:
- It is really hard to think like a novice when you're an expert.
- You often don't have the domain knowledge of your customer.
- You become biased toward certain designs as you invest time and energy
into them.
- Predicting human behavior is something even psychologists can't reliably
do.
Definitions:
- Usability review: A product critique by the usability staff (see below).
- Usability testing: Some say that this is synonymous with user testing, but many use it to refer to quantitative or qualitative data
gathering about ease of use in general.
- User testing: Any
testing that involves real, live, typical users (see below).
Methods of finding problems:
- User analysis: If you don't know who your users are you can't design for them.
- Task analysis: You need to know exactly how people currently work if you expect to simplify their work or improve their performance.
Many software packages actually hinder previous work flow patterns which have already been optimized through trial
and error.
- Exploratory test: Usually involves one user. Uses paper and pencil, text documents or very simple digital mock-ups. These are very
important early in the design cycle.
- Assessment test: Involves users. This is the most typical kind of test. It begins a quantitative and qualitative examination of
a partially working design. It catches smaller errors missed in previous tests and determines areas of difficulty
before they become hard to change.
- Validation test: Involves users. This is done late in the design cycle and extensively uses usability requirements. It confirms
that the software is performing as expected by gathering simple statistics of usage. The results are compared against
the requirements and changes made to areas that need improvement.
- Comparison test: Involves users. Sometimes it is important to compare your product to a competing product or a previous design with
a new design. This can be done by giving the same tasks to users using different programs and seeing who does better.
- Heuristic analysis: A style of expert review which uses a set of "rules of thumb" or short guidelines for design. Each expert
categorizes the problems they found into these categories so that they can be easily debated.
- Cognitive walkthrough: A style of expert review which heavily utilizes "user-scenarios" and attempts to view navigation through
the eyes of the user.
- Participatory design: A style of development that advocates having a user on the design team to provide domain-specific knowledge necessary
for development.
- Follow-up study: Determine how the product is being used, how it has affected the work environment and how it can be improved.
Note that any of the above methods are best used "iteratively."
For instance, you might want to run 2 exploratory tests in a row and then an assessment test. If the results from
the assessment test are bad, you go back to the drawing board and run more exploratory tests until you get it right.
The tests are designed to catch different granularities of problems at different times in the development cycle.
How can you measure usability? ñ
Well, maybe first you're asking "Why would I want to quantify usability?"
It is very useful to know how much more usable one design is over another.
Does it save new users 20 minutes while doing task A? Have usability related customer support calls dropped by
30% since you started doing user-testing on your product? Have you reduced the average number of clicks necessary
to purchase a product in your online shopping cart by 40%? Have the number of user errors reduced by 14 occurrences
since the last test you did? Did your test participants rate their subjective satisfaction of the product as an
average of 4/5 instead of the 2/5 they did last time?
Here are some typical types of measurements. Some companies will do very
fancy statistical examinations of user interactions. In cases like medical or military software, this is justified.
In most business cases, doing more testing is better than doing better testing.
- Time to complete a task
- Time to complete a task after a specified amount of time away from the
product
- Number of errors and type of errors per task
- Number of errors per unit of time
- Number of users making a specific type of error
- Number of users successfully completing a task
- Number of steps necessary to complete a task
- Amount of usage of online help or manuals
- Ability of users to satisfy usability requirements*
- Questionnaire results
- Expert analysis
- Quantity of usability related tech support calls
- Rating scales of usability problem severity
* Usability requirements are performance goals
for the product which are developed before the product is tested.
Example: 60% of novice users should be able to type a paragraph, bold the title, save the file and print it in
under 10 minutes.
Does an investment in usability really pay
for itself? ñ
Yes. Think of it as a trickle-down effect.
Take a minor problem that might take a team a day to recognize and fix
early on in the design.
If the problem remains in the final product you get:
- designs that you have to re-design later, taking weeks or months
- bad designs that simply stay in place because they are too hard to change
- new features have to build on an unusable foundation and hence have
decreased usability
- increased customer support calls
- potentially lost sales due to unhappy customers or competition which
is more usable
- increased maintenance costs
- increased training time which your customer unhappily pays
Costs of usability engineering include:
- salaries
- hardware
- test participants
- travel
Cases of cost-justification:
- There are daily instances of web sites improving usability for increased
sales, repeat customers, increased advertising, better reputations, etc. IBM recently reported that sales went
up 400 percent with an easier to navigate site.
- Telephone companies regularly report savings in the millions for shaving
seconds of usage times. [Karlin and Klemmer, 1989, Wayne D. Gray, et al 1992]
- Australian insurance company had annual savings of A$536,023 from redesigning
its application forms to make customer errors less likely. [Fisher and Sless 1990]
- Major computer company saved $41,700 the first day the system was in
use by making sign-on attempts faster for a security application. [Karat 1990]
- Improvement of a Boeing 757 flight deck interface to allow for only
two pilots instead of three. [Harris, 1984]
For further information please read: Cost-Justifying Usability (Bias
and Mayhew, 1994).
How can I implement usability in our software development? ñ
- Include usability people in the initial designs for a product or feature.
- Establish usability "check offs" by the usability dept. for
each stage in a project.
- Make sure all GUI changes go by the usability dept. Even the smallest
label change can throw off the usability of a page for new users.
- Familiarize yourself with usability engineering methods and try to use
them yourself and support usability goals in your products.
- Make sure you are in direct contact with your users and have watched
them use your product in their workplace.
Are you are a developer who just wants to quickly improve the usability
of your designs?
- Sketch out your designs on a whiteboard first. Don't just jump into
coding! Play with some different layouts and underlying architecture ideas before deciding on which one is most
simple and effective.
- Imagine trying to teach your mother how to use what you are creating.
Now simplify it. =)
- Grab someone else and ask them what they think things will do without
selecting anything. If you have to explain anything to them you should probably redesign it.
- Follow style guides ~ and use the most simple language you can think of.
- Think up a few typical scenarios of usage and imagine yourself knowing
nothing about the product. Now try and do the tasks.
- Run your designs by the usability team as early in the design process
as possible.
User Centered Design (UCD) is an old idea that is coming around again
under a new title. In short, it advocates increased time spent on better design, early on in the design cycle.
It results in fewer problems that need to be fixed late in the design cycle when they are expensive and often don't
get made. Despite the increased percentage of the cycle spent on design, it works out to about the same total time.
This is because of fewer re-designs, misunderstood product specs, and similar problems.
Think about it in terms of architecture. Do you want an architect to
build your house without a blueprint? Does an architect give you a two bedroom house when you have three kids?
Do you want a house with stairs when you are in a wheel chair? You want personal attention when you build your
house! Users want personal attention too and they don't like moving into houses that aren't built for them.
The initial design focuses extensively on the user. It includes an understanding
of who the users are, what their work environment is, what tasks they do and how they might be improved. UCD advocates
early and repeated testing of product mock-ups to catch design problems early on. Mock-ups start as paper and pencil
drawings that can easily be changed and re-thought and moves progressively into simple digital mock-ups and finally
into a working product that should be exactly what the customer needs.
For further information please read: The Usability Engineering Lifecycle
(Deborah J. Mayhew, 1999).
What other fields is usability related
to? ñ
Human computer interaction
Software engineering
Cognitive psychology
Experimental psychology
Ethnography
Systems engineering
Human factors
Ergonomics
Layout and design
How is usability different than QA? ñ
Developers rely on QA to test their code, report bugs, and verify bug
fixes. Very hip QA people also report potential usability problems to the usability team.
How is usability different than Technical Support? ñ
Tech support are the wonderful people who are patient enough to deal
with users who are trying to get their jobs done with programs that have usability problems and bugs. They usually
have great war stories to tell the usability team.
Why isn't the term user-friendly on this
list? ñ
Nearly every product that has had severe usability problems has at one
time or another claimed to be user-friendly. The term has therefore lost any meaning. Microsoft has always mentioned
that Windows is user-friendly in their advertising.
Much of the material in this document has been developed using the following
books for terminology and ideas:
Handbook of Usability Testing: Jeffrey Rubin, 1994
The Usability Engineering Lifecycle: Deborah J. Mayhew, 1999
Usability Engineering: Jakob Nielsen, 1993
User and Task Analysis for Interface Design: JoAnne T. Hackos and Janice C. Redish, 1998
|
