Archive

Posts Tagged ‘research protocol’

In-Depth Customer Interviews: Design Evaluation Exercise

December 1st, 2009
In my last post, I discussed an interview module that asks customers to evaluate an existing design. This post is an example of that module.

Interview outline:

-       Review the ReGIS design comps of key screens with participants

-       Ask participant to describe each of the elements on each page, and what they think each component does

-       Ask participant to describe the destination of the most important links on the page

-       Ask participant to mark up the page using a highlighter, specifically indicating with components should increase or decrease in prominence

-       List components that the participant considers as missing

Data to capture:

-       Marked up design comp

-       Identification of design elements that cause confusion

-       Notes about participant’s design preferences

-       Research assistant’s coding of participant reactions (see “coding” on page xxx)

Example Script:

Please take a look at this design for the start page of ReGIS. I would like to get your your feedback about this design. This design was not created by anyone in the room, or anyone from the Company for that matter, so nobody’s feelings will be hurt. Be as direct and opinionated as possible.

As we talk through this design, I’m going to ask you to mark it up to show your thoughts and opinions. To do that, please use these highlighters and this pen. When you see a type of information or tool that you would be likely to read or use, highlight it with the green highlighter. When you see a type of information or tool that you don’t understand what it means or how to use it, highlight it with the yellow highlighter. When you see a type of information or tool that you would not be likely to read or use, mark through it with the black marker.

Okay, now I would like to ask you to begin in the upper left hand corner, and describe what you see in each area of the page. Describe what you think it is for, and what you think will happen if you click it.

Probe regarding participant’s understanding of each design element on the page.

What is your understanding of how the “Bring It Now” feature works? What products do you think you would find there? Do you think there is an extra service charge to have these products delivered immediately? What do you think that service charge is? What do you think it should be?

What is your understanding of this element in the middle of the page (Coupons for on-site amenities)? How do you think it works? What will happen if you click one of these links?

Remind participant to describe what the expected outcome is of clicking each major link on the comp.

If participant omits component, ask question to make sure they can identify component when prompted.

Missing components

Now you’ve responded to the design elements on the page. Let’s talk about what’s not on the page, that you would like to see added in. You can review the ideal guest site you designed earlier or the lists of tasks we covered earlier in the interview if you want to.

What components would you like to add to the page?

For example: “How do you think community tools such as Locate Guest, Post Pics, or Post Video should be accommodated in this design, if at all? If they were included, how often would you use them?”

Specific outcomes of the design evaluation module include:
  • Customer-generated requests for design modifications
  • Opportunities for innovative design features and functionality
  • Identification of interaction design elements that confuse customers
Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)
Linked In: http://www.linkedin.com/in/uxexperts

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized , , , , ,

In-Depth Interviews: Participatory Design Exercise

November 23rd, 2009

The last post discussed a participatory design module for in-depth customer interviews. This post gives an example exercise and script.

To determine participant’s concept of the ideal Kiosk system for on-site booking of reservations and events.

1) Introduce list of 50 tasks used in card sort, to provide more raw material for participatory design exercise.

2) Provide blank wireframe with placeholders for header, navigation, portlets

3) Ask participant to design their own Home Page using their personal task list created above, and the list of 50 tasks provided

4) Probe regarding design elements

5) Offer possibilities based on previous comments if participant is stuck

Data to capture:

Wireframe sketch

Notes regarding optimal kiosk design

Script:

Introduce topic

In this exercise, I’d like to ask you to create the ideal Guest Kiosk Home Page. I’m not so much concerned about what everybody else would want on the Home Page. I’m concerned right now with what the ideal would be for you personally.

Before we start this exercise, I’d like to remind you of the tasks and content you’ve already mentioned. I’d also like to present you with a list of tasks have come up before, so that you can have plenty of raw material for doing this exercise. But again, we’re interested in what’s most useful and interesting for you.

Please take a moment to read through this list quickly, to jog your memory about things you would like to see represented at the very top of the kiosk Home Page.

Give participant time to review list and mark desired tasks.

This is a blank page, on which I’d like to ask you to draw your ideal kiosk home page. If you need more room, we can just make a notation on the front of the page, and continue that element or list on the back of the page.

First of all, what do you think the main sections you would like to see here at the top, or perhaps along the side?

What kinds of news or other content would you like to see spelled out here on the Home Page with a paragraph and perhaps a picture?

What kinds of links would you like to see?

What kinds of spotlights or highlights of content would you like to see here?

Would you want those to be just links, or part of the story as well?

What else would you like to place on the Home Page? Where would you place the search box? Do you want to be able to print this page? What about save this information for later viewing.

What kinds of functionality would you like to see on this page? How do you think it should work? Do you expect to have to sign in to use it?

Allow participant to fill in page. Probe as necessary using participant’s lists of tasks and content, as well as the pre-determined list of tasks.

Design review

Let’s talk about the page you’ve created, to make sure I understand what you’ve represented.

Review design, and ask for explanations as needed.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)
Linked In: http://www.linkedin.com/in/uxexperts

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized , , , ,

In-Depth Interviews: Capturing the Ideal Experience, part 2

November 13th, 2009

The last post talked about capturing a customer’s ideal experience. This post gives a brief example of how to do that.

Objective: Explore the ideal guest information system experience for resort visitors with young children who spend an average of $200 per day at the resort beyond room fees. Determine gaps between the actual and ideal. Prioritize the gaps based on cost to implement and business value.

Script:
“I would like to ask you about the perfect mobile guest information system. You mentioned earlier that some of the features you would like to see incorporated into our guest information system include…”

“But now I’d like to take it a step further and discuss the ideal system. Don’t be concerned about the impossibility of it, or the cost. Let me worry about that. For now I’d like to ask you to think with me about the best system you can imagine. Think back upon your stay so far at the resort. When would you have liked to pull out a mobile device and use it to get information or order something, regardless of your location.”

Allow participant to describe scenarios. If participant needs help, probe gently first with locations, such as, “What about when you were at the pool? What about when you were in your room?…” Then probe with features that are currently under consideration: “Have you ever considered what it would be like if you could request dinner recommendations and make reservations no matter where you were at the resort?”

This is different from blue sky thinking or brainstorming. This exercise elicits the normal task process, but without constraints of what currently exists. There may be a simple solution for providing what the customer is asking for that doesn’t break the scope bank.

Copyright 2009, Paul Bryan, Usography Corporation (http://www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized , , , ,

In-Depth Interviews: User Motivations, Example

October 23rd, 2009

The following is a sample script for the user motivations module of an in-depth interview. The objective of this sample exercise is to understand the overall context before travel, during travel, and after travel that may involve needs associated with web site usage.

Sample script:

Establishing the behavioral and motivational context

Previous experiences

Now I’d like to talk about your last three vacations. Can you tell me about them?

Where did you go? What was it like? How did you get there? What kind of hotel or resort did you stay in? What was it like inside? Outside?

Please describe your room or suite in detail. How much did the room or suite cost?

Overall approach

When you think about taking a vacation, what do you think about first?

What are your favorite types of vacation? How do you go about planning them?

What is your approach for selecting one hotel or resort over another?

If prompting needed: For example, leisure, fun activities, mix with other people, be alone, beach, mountains, woods, water sports?

If you had to select one of these basic approaches to selecting a resort, which would you say you focus on most?

-       Luxurious sparkle

-       The time of my life

-       Unique experiences

-       Rejuvenation, R&R

-       Meet interesting people

-       Good times for good value

Where would you like to go, but haven’t yet had the time or resources to go there?

Is the choice of location and resort yours alone, or is it shared with someone else?

If shared: What is your role in terms of selecting the hotel or resort and location?

Information resources

When you were planning the trips, how did you find out about the places you would stay? What resources did you use?

Probe for comprehensive list.

What other resorts did you read about but didn’t go to? Why not?

4. Summarize motivation

As time permits, restate the approach in terms of steps

Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)
Linked In: http://www.linkedin.com/in/uxexperts
Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized , , , , , ,

In-Depth Interviews: User Motivations, part II

October 21st, 2009

Discovering motivations can take place through the interview process, through a photo-elicitation or card-sorting exercise. Motivations can also be discovered by observing an online or offline activity and then asking questions about it afterwards. The researcher should be careful to avoid suggesting motivations to the participant and asking them to confirm or deny the motivation. This is leading the participant, and the accuracy of response will depend heavily on the personality of the participant. Constructing a motive and then reading into the motivation can obviously lead to misinterpretations.

Another method for discovering motivations is to use a Likert scale. [xxx example Likert scale questions.] Likert scales in an interview setting can be used to understand the motivations of the individual participant, which may also help understand a composite user archetype. The scales can be used to compare and contrast different participants. But they can’t be used in the interview context to generate conclusions about the larger population of users. That would be crossing over from qualitative data capture to quantitative data reporting, which is never a reliable approach.

Photo elicitation

Photo-elicitation is an interview technique in which participants are shown a series of images, and they are asked to comment on those images. The images may be photos that the participant has brought or that have been assembled by the research team. They may have been created as part of the research process, for example the participant may have been asked to create them as homework in preparation for the interview. Or, they may have been existing photos that were assembled for the interview, for example collected by the research team from a stock photo web site as illustrations of a concept related to the research topic. In a photo-elicitation exercise, the researcher shows participants a series of images and asks them to comment on them, possibly asking questions that lead the participant to talk about specific topics related to the research.

Sample script: Photo-elicitation for resort booking system

What is the first thing that comes to mind when you see these images?

Show series of resort photos from brochures and web site.

Show series of photos of people on vacation from stock photo site.

When customer responds positively to an image: What are some words you would use to describe how you would feel if you were part of this scene?

Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)
Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized , , , , ,

In-Depth Interviews: The Interview Protocol

September 17th, 2009

The most important component of planning in-depth interviews is the Interview Protocol. I’m going to describe the kinds of interview sections I include in the interview protocol to capture data that is directly applicable to innovative web design. Today, I’m going to write a brief overview of the interview protocol, in case you’re not familiar with what it is. And the next entry will start getting into the actual contents of an interview protocol that is targeted for design innovation.

The interview protocol describes in as much detail as possible everything that will occur on the day of the interview sessions. Interview protocols take many forms in different organizations and even vary within organizations for different projects or teams. Therefore, I will describe what is typical in my Usography experience and professional practice, and readers can adapt this to their particular environment as necessary. Although the format and contents of interview protocols can vary widely, their purpose is consistent: to elicit the responses and data from customer interviews that will provide the insights and understanding needed to guide design of a web site.

Interview protocols have two main components, which may or may not be physically separate documents. The first component provides overview information for the project team and stakeholders about objectives, procedures, data capture and recording, pre-determined thematic codes for note taking, interview materials, maps, security, refreshments, and any other topic that people need to know regarding the reception of participants and the interview sessions. The second component of the interview protocol is the discussion guide, consisting of the interview script as well as notes to the interviewer to ensure that topics are fully covered and that necessary data is captured. 

 

Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)
Linked In: http://www.linkedin.com/in/uxexperts

Copyright 2009, Paul Bryan, Usography Corporation (www.usography.com)

Linked In: http://www.linkedin.com/in/uxexperts

Uncategorized , , ,