Halloween night was the date when I conducted an interview and task observation with a good friend who regularly attends folk events and festivals. (Please note that the interviewee is wearing the microphone.)
I prepared a question sheet for the interview and practised on my teenage son, so as to check the flow of questions, as suggested by Kuniavsky (2003). I downloaded a free version of Microsoft Expression Encoder 4 in order to record the observation tasks and I practised using this software (again, on my surprisingly obliging son). I was impressed by the quality of the screen capture and used the headphones and mike pictured above to record the participant’s audio very clearly. However, I decided not to include the webcam feature as this lagged behind the audio and screen capture.
The data capture sheet had separate boxes for each screen as a reminder of the 4 key questions for each screen. I felt that this was important to ensure consistency in our approach across all tasks and all participants. During my test task observation I became quickly aware that it is almost impossible to take notes whilst talking to a participant and I agree with Kuniavsky (2003) who suggests that this exercise should be conducted by 2 people.
Conducting the interviews and task observations
We encountered a range of issues which I have categorised as follows:
1. Technical Issues
I had to find, download and practise using the software; this initially seemed straight forward but it became difficult to troubleshoot when we encountered problems with Internet connections; firstly with teenage children playing online game during the task observation with Participant 1 and secondly on campus where the Eduroam wireless system failed, forcing us to rely on audio recording for Participant 3 and 4. Although I had tested the wireless system on campus at 9.30am, I hadn’t made allowances for the notoriously high demand on the wireless network in the Library between 11am and 2pm. In the future, I would minimise the chance of this happening by connecting via an ethernet. Indeed, as I write, I am also wondering why I didn’t video the task observations using my phone…
2. Lack of interviewing experience
Having conducted the interview/task observation with a familiar primary user, I had become aware that the series of questions which we had agreed as a group were very specific to the website and that they were not going to provide as much background information as we might need to understand the lifestyle, interests and motivations behind the behaviours of the interviewees, particularly the primary users – about whom the group had little information. We needed to ask additional questions but this proved to be difficult as two of the interviewees were line managers for two members of the group! Although this wasn’t ideal, there were advantages to using senior staff. Firstly, we were allowed to conduct the interviews during work hours and secondly, we were able to access a suitable venue (a group study room in the Library). Consequently, given the budget and time restrictions of the project, I feel that we gathered enough information to fulfill our user research objectives. In a future project, I would include additional questions to enhance our understanding of the primary users.
The second issue that arose due to my lack of interviewing experience was deciding when to interrupt and re-focus the interviewee’s attention to the question/task as opposed to just keeping quiet and listening! This is obviously a skill which needs to be developed over time; recording and listening back to interviews has certainly helped me to analyse my performance. For example, when observing task 3, I insisted that Participant 1 should try and find the Facebook page from the website, even though she had proposed an alternative route which would have worked in real life. This is discussed by Krug (2006) who outlines a range of motives for users not choosing a designed route, including the idea that users will revert to previously successful strategies for solving problems, even if they are unorthodox. The result of my intervention was to force the participant to undertake additional searching which was invalid as it was not what the participant would have done under normal circumstances.
3. Interviewee Issues
There was evidence that interviewees were slightly nervous about the interview/task observations. Participant 2 arrived late and flustered and seemed less focussed in his approach to the tasks. I had booked the room for 5 hours to suit participants’ needs. Unfortunately, the interviews were arranged by another group member with only 15 minute breaks between each participant’s 30 minute slot. In the future, I would leave longer periods between interviews to accommodate over-runs and to allow time for re-setting equipment.
Participant 3 wanted to give the “right” answers, as shown by the transcript of one of the tasks: Participant 3 Observation task 1buying a ticket. We tried to minimise this anxiety by preparing an introduction speech, as discussed by Kuniavsky (2003) which was read to all participants to explain the purpose of the task. Unfortunately, this participant is an experienced manager who would probably not wish to appear unable to complete a task in front of her junior staff (even if difficulties were due to the website design). In future, I would therefore avoid any potential conflict of interest by not interviewing my own line manager, unless the person was a key stakeholder to the project.
4. Analysis Issues
The original plan had been for the interviewers to share the transcription process with the other members of the group. Unfortunately, as 2 of the interviews were affected by the technical problems, we thought that it would be very difficult for the rest of the group to transcribe the audio-only recordings of the practical tasks. Consequently, we have yet to analyse the data. I am conscious of the fact that transcripts are raw data and that the interviews and task observation results need to be summarised and analysed to obtain the information required to build archetypal personas. We will try and complete this before next week’s lecture.
What did I learn?
- Interviewing is difficult; asking searching, open-ended questions without leading a participant or imposing one’s views is a skilled job
- Researchers need to work in pairs and allow extra time for delays and discussion
- Researchers should have back-up recording equipment in case there are technical problems
- Audiovisual recording of the research provides an accurate record intervieweee responses
Was this research useful?
Overall, despite some of the issues outlined above, I can see that this user research has provided us with real information required to map realistic scenarios and journeys for the primary users which had not been available to us from the SurveyMonkey survey. In the future, I would undertake similar research, not only to support the development of personas and user journeys but also to provide an evidence base. Bowles and Box (2011) discuss how this would be particularly important where stakeholders may have conflicting priorities, as it would help focus everyone’s attention on the key requirements. As with the persona development, I can see how this is going to provide the basis for the design aspect of the website.
Bowles, C. and Box, J. 2011. Undercover user experience : learn how to do great UX work with tiny budgets, no time, and limited support. Berkeley, CA: New Riders.
Evans, W. 2012. Customer research & persona development [online]. San Francisco: slideshare. Available at: http://www.slideshare.net/willevans/customer-research-persona-development [Accessed 01 November 2012].
Krug, S. 2006. Don’t make me think! : a common sense approach to web usability. 2nd ed.Berkeley, CA: New Riders.
Kuniavsky, M. 2003. Observing the user experience: a practitioner’s guide to user research. San Francisco, CA: Morgan Kaufmann