What customer data matters for UX design?
Optimizing your virtual floorspace requires an in-depth understanding of customers. To gain this understanding, companies undertake the expensive process of collecting customer data. For customer data to impact UX design, it must be collected in a manner that answers specific design questions. As a design strategy consultant, I am continually presented stacks of data about customers. Unfortunately, much of it is irrelevant or tangential to the understanding that is needed to optimize design work. Some of the research methods and data collection techniques that have been useful in Usography’s design strategy practice are listed below, along with a brief explanation.
Depth Interviews
Depth interviews are helpful to get a better understanding of why customers behave the way they do, and for developing mental models that serve as a basis for interaction design. Depth interviews are most useful at the beginning of a project, before the product has taken shape.
Web analytics
Web usage data can be collected, aggregated, analysed and reported by a number of web analytics packages. Every action taken by a site visitor can be tracked by most of these packages. They vary in their ability to analyze and visualize usage data. Some packages allow you to zoom into individual sessions and observe customers who meet specific criteria.
Surveys
Surveys are easy to conduct using tools like Survey Monkey. However, they are not easy to construct so that their results are meaningful to designers. The construction of a reliable, valid survey requires the support of people who have a significant understanding of statistics.
User testing
Observing customers while they are using the tools you (or others) have designed can be invaluable, but drawing conclusions that are useful to designers depends on framing the exercises to evaluate specific design issues, rather than randomly walking through a series of site tasks. User testing should begin with the first prototype, and be repeated with each major revision, including the final release.
Ethnography
Ethnographic studies involve the capture of rich data related to the full context of an activity for which a web site or information system is being developed. Originally an anthropological method for studying little known cultures in undeveloped nations, ethnographic studies have been gaining favor across a wide range of commercial design situations. Ethnography is most often used as a data gathering tool when the cost of research is significantly outweighed by the potential revenue of creating a superior product. Ethnographic methods may include one or more of the following: observing customers in a natural setting (participant observation), asking them to keep journals, webcam diaries, video observation, artifact collection, or in-context interviews.
A/B and multivariate testing
Design options can be empirically tested by introducing one variation on a design element, sending some users to the old design and some users to the new design. Statistics about the relative conversion rates reveal which design, A or B, is likely to be more successful. The duration of the test and the number of users sent to the experimental design, option B, need to be sufficient to obtain statistically valid results. Tests I’m aware of have sent up to 90% of the site traffic to the current site( option A) during the test, and 10% of the site traffic to option B. The specific test parameters are closely related to the non-test state analytics of the system being tested.
A/B testing usually requires approvals from a variety of stakeholders, and is expensive because of the resources involved and potential negative impact. Therefore, companies often want to test several design variations at once. When more than one design variation is tested, the math becomes more complicated, because it is difficult to tell which change has resulted in the conversion rate differences. Multivariate analysis is required to reliably sort out the results.
Help desk / customer service
This is one of my favorite streams of customer data, and one that is often overlooked in the design process. Customer service reps and logs are excellent sources of customer feedback related to design issues, although they are not necessarily the people who should be involved in developing design solutions.
Social media / discussion forums
Many customers who would never volunteer for a depth interview or user testing express themselves freely online in discussion forums. While this is great fodder for strategy meetings with executives, the issue of prevalence is one that suggests that quantitative validation is needed before any serious conclusions can be drawn.
Secondary research reports
Companies like Forrester, comScore and Nielsen produce research reports with customer data and trends that are very relevant to web design. Data about the needs of specific segments of the population is helpful for designing solutions for those segments of the population.
Market data
I use market data to understand the size of the market, the potential of the market, the demographics of customers, and trends. However, I don’t usually draw many conclusions about specific user experience design issues from such data. It is usually too highly aggregated for this purpose, and does not provide the kind of behavioral data I need to make design strategy decisions.