Virtual Floorspace

6. Researchers add data collection methods as appropriate.

Ethnography is very flexible in terms of data collection. Some ethnographers develop their methodology in the field as events unfold. The primary data collection methods, observation and interviews, may be supplemented by a wide variety of field research techniques. Some additional data collection techniques commonly deployed by design agencies include passive video capture, video participant shadowing, photo elicitation, visual diaries, and concept collages. Research in archives as well as social media web sites can yield supplemental findings that are difficult to capture or observe in the field.

Results from ethnographic studies are often triangulated with quantitative methods, such as surveys or analytics, for corroboration, extension, or quantification purposes.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized design strategy, e-commerce design strategy, e-retail, ethnography, interview protocol, methodology

3. Interviews are used to clarify what is observed and to gain a deeper understanding of behavior.

After  period of observation in the field, we use intercept or in-depth interviews to elicit explanations of terminology, decision factors, perception of what’s most important, motivations and beliefs behind behaviors, and other variables of interest. The interviews may be with participants that we have observed, or may be with “informants” who understand behaviors from an insider perspective, and who collaborate with us in a structured way. The informant may be a salesperson who has just finished interacting with a customer, or may be a manager or gatekeeper in other business settings. Interviews may be unstructured or semi-structured. A snowball or chain sampling method may deployed if the interviews are unstructured, but in commercial settings it is more common to screen participants on the basis of a set of pre-defined criteria.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer research, design strategy, e-retail, ethnography, interview protocol, methodology

Once participants arrive, the interviews are mainly a matter of ensuring that the data you need to capture as outlined in the interview protocol is captured. After the first couple of interviews, the research lead will develop a sense of topics that are most likely to result in data that will be useful to innovative design work in the future. The research team should debrief after each interview to discuss whether the research protocol should be modified for any reason. Some questions or exercises don’t work as well as planned in terms of data capture and participation, and need to be amended or omitted. In my experience, the most changes are made either after the first session, the first day, or the first round of interviews.

As time goes by, the interview moderator may modify the interview for each participant to get the most value out of the interview. Some people feel that this is problematic because modifications to the interview precludes comparisons of data across participants for a given set of variables. My feeling is that assigning such weight and rigor to variables that are being measured qualitatively is to misunderstand the nature of the data being collected in customer interviews. I feel that it is best to get the most value out of each interview and to characterize each participant as clearly as possible, and then to compare variables across participants to the degree that it makes sense for the type of data being collected. If a participant is leading me down an interesting trail of discovery toward a potential opportunity for design innovation, then I will follow that trail in lieu of other sections of the interview that I feel will not be as fruitful. My sense of where to take the interview has developed over many years of interviewing customers for the purpose of creating innovative web sites.

Uncategorized customer interviews, customer research, Depth interviews, e-commerce design strategy, interview protocol

In addition to the materials required to conduct the research exercises and to capture data, you will need to organize food and drink according to the standard practices of the organization sponsoring the research, and as allowed by the project budget. Sometimes the food and drink offering is elaborate, with a full lunch or dinner provided as a perk to the participants, researchers, and/or project sponsors. In other cases, it is very simple: “The snack machines are over there.” At a minimum, bottled water should be provided for the people who will be talking the most: the participant and the moderator. I prefer a middle of the road set up, because lots of food tends to cause unwanted distractions, and participants fumble with plates and trying to eat while being observed. I prefer a selection of soft drinks and juices, with a tray of assorted snacks like trail mix, power bars, or other easy to eat items.
Research days are often 10 to 12 hours long, and so a lunch tray for the research team is always welcome. I prefer to order lunch and have it delivered to the area where the team is located, rather than going out to lunch. Walking to lunch spots with the project sponsors and team is a good opportunity to bond, but in some cases it leads to many opportunities for getting off-track for the afternoon sessions. Project sponsors or wayward team members start window shopping or stopping for little errands, and then we’re late for the first afternoon session. Being cooped up in the same small room for 12 hours is no joy, but it tends to keep people talking over impressions, which can be very fruitful.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer interviews, Depth interviews, interview protocol, methodology, planning user research

The research protocol should include a detailed schedule for each day. The following is a typical daily schedule for Usography research projects:

-       Prepare participant testing station and observation room

-       Make copies of script if necessary

-       Set up video equipment when research assistant not present

-       Replenish materials for exercises if necessary

-       Prepare water and/or other refreshments

-       Conduct Interview 1

-       Debrief

-       Replenish materials and prepare rooms, as needed

-       Conduct Interview 2

-       Debrief

-       Replenish materials and prepare rooms, as needed

-       Conduct Interview 3

-       Debrief

-       Replenish materials and prepare rooms, as needed

-       Conduct Interview 4

-       Debrief

-       Assemble and label materials completed during the day

-       Burn data to DVD

Research sessions are being scheduled at 2 hour intervals. The interviews usually last 75 to 90 minutes. There is a break between sessions of 30 to 45 minutes, when the team talks about what was observed in the previous session and the changes, if any that should be made to the research protocol. If the team takes a lunch break, it usually lasts 30 minutes and is taken on-site.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer interviews, e-commerce design strategy, interview protocol, methodology

The research team should arrive at the interview location at least an hour before the first interview will take place. This gives you time to prepare the setting and agree on any mechanics of the interviews that are still open. If interviews will take place at different locations throughout the day, then my team usually meets for breakfast to run through the day’s activities and to make sure all materials are ready for use.

If interviews will take place in a central location, then a list of the current day’s interview participants should be printed out and posted in the area where they will take place. This list should also be emailed to team members and stakeholders.

Check with security prior to the day of the sessions to figure out what kind of documentation will be required to bring participants into the research area. Checking ahead with security is relevant in all kinds of research settings, from office buildings to retail locations, to special facilities like airports. This is often overlooked, and can lead to big headaches. At a minimum, security should be given a list of each day’s participants. They usually have some kind of guest badges or passes that can be issued at the check-in desk.

The research protocol you wrote identified the materials that will be required for your customer interviews. The materials identified in the research protocol should be refreshed at the beginning of each day. Copy machines for some reason seem to break more often the day that interviews take place, so we usually make lots of copies prior to the day we will need them. We are especially careful to have extra copies of any paper forms that participants will write on directly, because if we are lacking those, then we will be lacking data we need. Recording devices, media for data and copies, markers, pads, etc., should all be checked at the beginning and end of each research day to avoid unpleasant surprises.

As a point of reference, the following are materials that Usography typically requires for customer interviews:

• Copies of research protocol
• Release form (permission to use likeness and responses)
• Payment received form
• Cards with content types, features, concepts
• Photos for brand impression analysis, photo elicitation
• Grid with topic relevance on one axis, perceived value on the other
• Digital camera, recorder, or video camera
• Map of physical location where activities or interactions take place
• Markers
• Blank wireframes for screen sketches

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer research, Depth interviews, design research, e-commerce design strategy, in-depth interviews, interview protocol

For design projects that focus more on innovation than improvement, one helpful interview technique is to ask customers about the ideal experience. This exercise is much more specific than simply brainstorming a solution with customers. The ideal experience interview module helps uncover different facets of the participant’s mental model of the activity apart from the experiences they’ve had to date. The ideal should be constrained to tasks within the activity domain for which you are conducting the research. Ask participants to go into some detail about their ideal process interactions, step by step. It’s unlikely that you will be able to accommodate the blue sky thinking of your participants, but you may be able to get closer than you think using available technology.

In cases where there is consensus among participants on ideal interactions and experiences, you should create an interaction model that represents the compass and an overarching design goal to shoot for. Steve Jobs presented some very specific goals to his engineering team that designed the Ipod. It seemed impossible to some on the team that they could meet the goal, but after a long and arduous effort, they did reach it: Your music collection in your pocket. This vision was not contiguous with existing solutions. For innovative solutions, companies can’t rely on incremental improvements that result from assessment-focused research methods such as A/B testing and web analytics.

Objectives of the Ideal Experience module:

• Understand each participant’s view of the ideal web site in your field
• Obtain a “best case” experience that is generalized from all the participant interviews
• Understand what customers ultimately want, and the intermediate steps that could be taken to ultimately reach that point

Participatory exercises can be used to determine a participant’s ideal experience (the ideal design is in the next module). For example, participants can be asked to describe their ideal concept of a web shopping experience or application, or the ideal store section. A single customer’s view of the ideal experience will not necessarily be significant on its own, unless a customer happens to come up with a brilliant idea for a future design. However, when combined with the results of other similar exercises with other participants, a model may emerges of a future innovative design concept that would meet the needs of many users.

If the participatory exercises result in scenarios or descriptions of experiences that are fundamentally different, then the research team needs to ponder what this result means for the future design. It could mean that a personalized experience is required, since everyone’s ideal solution was different. Or it could mean that the solution is not obvious to users, and must be driven by the design team’s experience and skills rather than customer-initiated design ideas.

Components found within the ideal experience or design should be tabulated across participants to understand trends, although quantitative methods are necessary before the results can be generalized to a population of users.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer interviews, e-commerce design strategy, in-depth interviews, interview protocol

Customers access business web sites to achieve a goal or specific purpose. They are rarely there just to look around, unless they are looking for a job and want to understand the company better. Customer goals can often be grouped into distinct modes, such as reading content, finding products or documents, purchasing products or services, learning new processes or procedures, etc. Understanding these modes is important when designing the interactive space, to enable customers to easily enter a mode that clearly facilitates the activity and presents options related to that mode in a consistent way.

For example, in a resort web site or kiosk, guests may be looking for leisure activities, finding products they forgot, reading about the history of the area, etc. Each of these modes should be simple and straightforward to find in terms of access points, make it easy to achieve the most common and most valuable goals, and represent the task using design components that appear especially suited to the activity (i.e. high affordance). The modes should not be cluttered with lots of options that are unrelated to the mode customers have indicated they want to work in. Marketers often want to surround and interject this experience with lots of selling options, but many times this is interpreted by users as visual noise that damages the perception of the experience and isn’t effective. Looking at the analytics for such off-task design elements, I’ve nearly always found clicktrhroughs to be near zero. If they have to be there, make sure they are not obstructing progress in the primary activity. Billboards are okay, but on the side of the road, not in the middle of the road. And spaced apart so that the visual signal to noise ratio is at a reasonable level.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized analytics, customer interviews, e-commerce design strategy, e-retail, in-depth interviews, interview protocol, methodology, planning user research, qualitative research, task analysis, user research, web design

In the previous post I introduced the task analysis module for in-depth customer interviews. One way to start the task analysis part of the interview is to have the customer create a task list. A task list is a list of activities that participants recall when asked about a given subject matter. For example, in research for a resort web site, the interviewer asks what types of content and functionality the participant currently uses on travel web sites. Although analytics give a more accurate picture of the prevalence of different activities in the system in question, an activity list or index helps characterize the individual being interviewed in terms of usage patterns, preferences, breadth of activities etc. The activity index is useful later in the interview when the research team asks participants to discuss a common activity in detail, noting barriers and opportunities for each.

Some activities in the list may be related to the system being researched, but may not specifically be conducted within the system. For example, participants may be asked to list resorts they have visited, or which areas within a resort they typically use, both of which are broader than asking what tasks they typically undertake using a guest information system. The purpose for capturing these broader activity indexes is to characterize the person being interviewed. The data can help the research team formulate an experience model.

When constructing an activity list, it is helpful to ask participants how they expect to go about the activity using a web site or information device. Their expected task path can be used to support the mental model and current or proposed interaction model.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer interviews, customer research, Depth interviews, design research, e-commerce design strategy, in-depth interviews, interview protocol

Discovering motivations can take place through the interview process, through a photo-elicitation or card-sorting exercise. Motivations can also be discovered by observing an online or offline activity and then asking questions about it afterwards. The researcher should be careful to avoid suggesting motivations to the participant and asking them to confirm or deny the motivation. This is leading the participant, and the accuracy of response will depend heavily on the personality of the participant. Constructing a motive and then reading into the motivation can obviously lead to misinterpretations.

Another method for discovering motivations is to use a Likert scale. [xxx example Likert scale questions.] Likert scales in an interview setting can be used to understand the motivations of the individual participant, which may also help understand a composite user archetype. The scales can be used to compare and contrast different participants. But they can’t be used in the interview context to generate conclusions about the larger population of users. That would be crossing over from qualitative data capture to quantitative data reporting, which is never a reliable approach.

Photo elicitation

Photo-elicitation is an interview technique in which participants are shown a series of images, and they are asked to comment on those images. The images may be photos that the participant has brought or that have been assembled by the research team. They may have been created as part of the research process, for example the participant may have been asked to create them as homework in preparation for the interview. Or, they may have been existing photos that were assembled for the interview, for example collected by the research team from a stock photo web site as illustrations of a concept related to the research topic. In a photo-elicitation exercise, the researcher shows participants a series of images and asks them to comment on them, possibly asking questions that lead the participant to talk about specific topics related to the research.

Sample script: Photo-elicitation for resort booking system

What is the first thing that comes to mind when you see these images?

Show series of resort photos from brochures and web site.

Show series of photos of people on vacation from stock photo site.

When customer responds positively to an image: What are some words you would use to describe how you would feel if you were part of this scene?

Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized customer research, design strategy, interview protocol, research protocol, user motivations, user research