Virtual Floorspace » design research

Ethnography 101: What makes it an Ethnography?

Paul Bryan, Usography Corporation (www.usography.com) — Wed, 03 Feb 2010 15:56:06 +0000

Yesterday I started listing 7 characteristics of ethnography to distinguish it from other forms of research, for people unfamiliar with ethnography.

2. Observation is a primary data collection method, resulting in “thick” descriptions.

Ethnography is important as a research method when there are key factors that participants either can’t or prefer not to verbalize for one reason or another. For this reason, observation is an essential ingredient for ethnographic research. Observation is either unstrucure or semi-structured. If the variables are understood to a degree that a structured approach can be formulated, then ethnography is not the best method to use. Field notes provide a rich, or “thick,” description of behaviors and events.

Linked In: http://www.linkedin.com/in/uxexperts

Ethnography 101: Intro

Paul Bryan, Usography Corporation (www.usography.com) — Tue, 12 Jan 2010 15:28:16 +0000

Ethnography is a field research method developed by anthropologists to understand cultures other than their own. At one time, anthropologists theorized about other cultures from the comfort of their own life context. Ethnography was developed as a method to study the culture from within, as a participant, and at the same time as an observer. Ethnography is by definition and in practice a systematic approach for understanding cultural impacts on behaviors, attitudes, beliefs and rituals.

In recent years, ethnographic studies have gained favor across a wide range of commercial design situations. In the world of web design, the term ethnography can be used rather loosely to mean any research that is conducted on location, or in context of the activity for which a web-based system is being designed. Ethnography is most often used as a data gathering tool when the cost of research is significantly outweighed by the potential revenue of creating an innovative product. Ethnography is especially applicable to projects that are multi-cultural in nature, but is not limited to those types of projects. In design-related ethnographic research, the “culture” being studied may be a segment of a population that shares a set of common characteristics or goals.

Some of the methods associated with ethnographic research are: participant observation, in-depth interviews, participant diaries, informant debriefs, and context mapping. My presentation at UPA 2010 in Munich will focus on the application and adaptation of ethnographic research methods to the field of web design and usability, rather than a rigorous definition of anthropology research methods.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Preparing the Research Location

Paul Bryan, Usography Corporation (www.usography.com) — Fri, 11 Dec 2009 14:06:58 +0000

The research team should arrive at the interview location at least an hour before the first interview will take place. This gives you time to prepare the setting and agree on any mechanics of the interviews that are still open. If interviews will take place at different locations throughout the day, then my team usually meets for breakfast to run through the day’s activities and to make sure all materials are ready for use.

If interviews will take place in a central location, then a list of the current day’s interview participants should be printed out and posted in the area where they will take place. This list should also be emailed to team members and stakeholders.

Check with security prior to the day of the sessions to figure out what kind of documentation will be required to bring participants into the research area. Checking ahead with security is relevant in all kinds of research settings, from office buildings to retail locations, to special facilities like airports. This is often overlooked, and can lead to big headaches. At a minimum, security should be given a list of each day’s participants. They usually have some kind of guest badges or passes that can be issued at the check-in desk.

The research protocol you wrote identified the materials that will be required for your customer interviews. The materials identified in the research protocol should be refreshed at the beginning of each day. Copy machines for some reason seem to break more often the day that interviews take place, so we usually make lots of copies prior to the day we will need them. We are especially careful to have extra copies of any paper forms that participants will write on directly, because if we are lacking those, then we will be lacking data we need. Recording devices, media for data and copies, markers, pads, etc., should all be checked at the beginning and end of each research day to avoid unpleasant surprises.

As a point of reference, the following are materials that Usography typically requires for customer interviews:

Copies of research protocol
Release form (permission to use likeness and responses)
Payment received form
Cards with content types, features, concepts
Photos for brand impression analysis, photo elicitation
Grid with topic relevance on one axis, perceived value on the other
Digital camera, recorder, or video camera
Map of physical location where activities or interactions take place
Markers
Blank wireframes for screen sketches

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interview: Evaluate existing design

Paul Bryan, Usography Corporation (www.usography.com) — Fri, 27 Nov 2009 13:41:51 +0000

If a site design already exists at the time the research is conducted, then in most cases you will ask customers about that design in the customer interview. The design may exist as either current release of the site that customers are using, or it may be a non-released design in the form of page comps or a prototypeprototype. The case for omitting an interview module that evaluates this design can be made when you are conducting research for a completely new design that will not rely on previous design work. I’ll discuss the case of the customer interview that focuses solely on design evaluation in a more detailed post later.

Objectives of the design evalution module include:

Understand the gaps between the desired design and the actual design
Obtain specific feedback as to which design elements need to be increased or decreased in prominence
Determine components that participants would like to eliminate from the design
Determine components that participants consider missing from the design

Linked In: http://www.linkedin.com/in/uxexperts

In the next post, I’ll include an example protocol and script for a design evaluation exercise.

Linked In: http://www.linkedin.com/in/uxexperts

In-Depth Interviews: Participatory Design Exercise

Paul Bryan, Usography Corporation (www.usography.com) — Mon, 23 Nov 2009 13:23:46 +0000

The last post discussed a participatory design module for in-depth customer interviews. This post gives an example exercise and script.

To determine participant’s concept of the ideal Kiosk system for on-site booking of reservations and events.

1) Introduce list of 50 tasks used in card sort, to provide more raw material for participatory design exercise.

2) Provide blank wireframe with placeholders for header, navigation, portlets

3) Ask participant to design their own Home Page using their personal task list created above, and the list of 50 tasks provided

4) Probe regarding design elements

5) Offer possibilities based on previous comments if participant is stuck

Data to capture:

Wireframe sketch

Notes regarding optimal kiosk design

Script:

Introduce topic

In this exercise, I’d like to ask you to create the ideal Guest Kiosk Home Page. I’m not so much concerned about what everybody else would want on the Home Page. I’m concerned right now with what the ideal would be for you personally.

Before we start this exercise, I’d like to remind you of the tasks and content you’ve already mentioned. I’d also like to present you with a list of tasks have come up before, so that you can have plenty of raw material for doing this exercise. But again, we’re interested in what’s most useful and interesting for you.

Please take a moment to read through this list quickly, to jog your memory about things you would like to see represented at the very top of the kiosk Home Page.

Give participant time to review list and mark desired tasks.

This is a blank page, on which I’d like to ask you to draw your ideal kiosk home page. If you need more room, we can just make a notation on the front of the page, and continue that element or list on the back of the page.

First of all, what do you think the main sections you would like to see here at the top, or perhaps along the side?

What kinds of news or other content would you like to see spelled out here on the Home Page with a paragraph and perhaps a picture?

What kinds of links would you like to see?

What kinds of spotlights or highlights of content would you like to see here?

Would you want those to be just links, or part of the story as well?

What else would you like to place on the Home Page? Where would you place the search box? Do you want to be able to print this page? What about save this information for later viewing.

What kinds of functionality would you like to see on this page? How do you think it should work? Do you expect to have to sign in to use it?

Allow participant to fill in page. Probe as necessary using participant’s lists of tasks and content, as well as the pre-determined list of tasks.

Design review

Let’s talk about the page you’ve created, to make sure I understand what you’ve represented.

Review design, and ask for explanations as needed.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Customer Interviews: Task Analysis Exercise 1

Paul Bryan, Usography Corporation (www.usography.com) — Sun, 08 Nov 2009 10:35:22 +0000

In the previous post I introduced the task analysis module for in-depth customer interviews. One way to start the task analysis part of the interview is to have the customer create a task list. A task list is a list of activities that participants recall when asked about a given subject matter. For example, in research for a resort web site, the interviewer asks what types of content and functionality the participant currently uses on travel web sites. Although analytics give a more accurate picture of the prevalence of different activities in the system in question, an activity list or index helps characterize the individual being interviewed in terms of usage patterns, preferences, breadth of activities etc. The activity index is useful later in the interview when the research team asks participants to discuss a common activity in detail, noting barriers and opportunities for each.

Some activities in the list may be related to the system being researched, but may not specifically be conducted within the system. For example, participants may be asked to list resorts they have visited, or which areas within a resort they typically use, both of which are broader than asking what tasks they typically undertake using a guest information system. The purpose for capturing these broader activity indexes is to characterize the person being interviewed. The data can help the research team formulate an experience model.

When constructing an activity list, it is helpful to ask participants how they expect to go about the activity using a web site or information device. Their expected task path can be used to support the mental model and current or proposed interaction model.

Linked In: http://www.linkedin.com/in/uxexperts

Depth interviews in design research: Planning data analysis

Paul Bryan, Usography Corporation (www.usography.com) — Mon, 31 Aug 2009 20:19:41 +0000

Depth interviews typically result in a mountain of data, including researcher notes, results of exercises and other research artifacts, video or audio recordings, and transcripts. Reviewing and analyzing this data is very time-consuming. You need a plan before the interviews start for what will be captured and how it will be processed prior to presenting results. As a rule of thumb, we allocate at least two hours for every hour of interviews for data review and analysis: 1 hour to review, 1 hour to analyze and formulate findings. It is easy to underestimate the time this will take. Watching a one hour video typically takes more than an hour, because you not only have to watch it through to make sure you catch all the information, but you will probably start and stop it to highlight the most significant sections, write notes, get the timecode precise for the clips you want, etc. Then you need time to formulate higher level findings, produce clips, and trim down the highlighted data for final presentation.

An important aspect of the data analysis plan is data reduction, i.e. how you will go from raw data to the first step of processed data. Usography standardizes this procedure so that all researchers use the same approach to data analysis, for comparison and corroboration purposes. Each researcher goes through the detailed transcript and notes at least once, recording notes about themes and high-level findings as we go. We then note recurring topics and place them into a table. Example data elements that we look for consistently across all participants are:

HHI
Internet IQ
Shopping IQ
Goals
Drivers
Categories
Business value
Typical purchase process
Barriers
Dropoff
Requested content/features
Competitive sites
Quotes
Method for tracking this user type with analytics

If we have previously conducted research in the area, or if we have done extensive secondary research prior to conducting primary research, we may have a preliminary coding system going into the study. In this case, research assistants are labeling notes with these codes as they record or transcribe them. Those codes are then used to form the data summary table for each participant. For example, in cell phone research we may designate “communication enabler” as a predefined code. Research assistants type “ce” everytime a participant makes a comment that matches this theme. This is part of the analysis planning that takes place before we conduct the research, and is a big time saver.

Written by Paul Bryan, Usography Corporation (www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Written by Paul Bryan, Usography Corporation (www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Depth interviews in design research: An overview

Paul Bryan, Usography Corporation (www.usography.com) — Tue, 25 Aug 2009 22:07:35 +0000

Depth interviews are a useful research technique when user needs, motivations, usage context, and personal success factors need to be understood in a greater degree of detail than is possible using other research methods, such as surveys. Depth interviews usually last between 60 and 90 minutes. The research team conducting depth interviews usually consists of an experienced moderator or senior researcher who facilitates the interview, and a research assistant who captures data.

The volume of data in depth interviews is high, and participants tend to wander around various topics unless strongly facilitated to stay on track. Therefore, having a senior researcher on hand to conduct the interviews is highly desirable, preferably one who is skilled in both interviewing techniques and web design (wink wink). The research assistant captures notes about what the participant says and does, as well as observations. Video or audio recording equipment is typically used to supplement the research notes with a visual and/or audio transcript.

Additional components of a successful depth interview include:

Participants who accurately represent the audience that the research is targeting
Detailed research protocol
Materials for completing research exercises

Interviewing people in the place where they work, or purchase goods, or use the information tool you are designing is called an in-context interview. Conducting the research in context provides many clues about the kind of design that will be successful, in terms of how users expect the web site to communicate with them, and the kinds of additional support they may need beyond basic transactions. When interviews are conducted in context, participants may refer to artifacts present in the activity context that affect their ability to use the information system. They may also point out offline or peripheral factors that significantly impact their ability to successfully use an online system, which may not be discussed if the interview were to take place in a meeting room or lab.

Because of the logistics and expense involved, depth interviews involve a lot of planning to run smoothly. I’ll write about the depth interview process as a set of manageable steps and provide examples to help you plan for successful depth interviews in subsequent blog posts.

Written by Paul Bryan, Usography Corporation (www.usography.com)