Product Management Knowledge Base
Page tree

Versions Compared

Old Version 1

changes.mady.by.user Bert

Saved on

compared with

New Version 2

changes.mady.by.user Bert

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Preparation for the session

Email with instructions

Google doc with the email template.


Pre test user instruction (to be read out loud and can be adopted for online use)

____________________________________________________________________

Dear [User],


Thank you very much for taking the time to help us testing and improving our platform!

My name is [Tester], I’m a [Tester position] at our company.


At the moment we are working on a [feature description]. Before this feature goes into production we want to test that it works as intended. For this purpose, we have built a clickable prototype. This prototype is not completely functional or interactive but it should be sufficient to give you an impression of the feature.


When the test starts I will ask you to perform an activity on the prototype as if you were alone doing it. Particularly, I would like you to think out loud as you are using the prototype. You can take a look at this 1-minute example of how to think aloud during the session: https://www.youtube.com/watch?v=g34tOmyKaMM&t=1s.

Basically, when you click on something or making a decision, we would like to know how do you think about it and why you are doing it. If something doesn't work as you expected, explain that to me. Also if you really like something, let me know.

And don’t worry that you’re going to hurt someone’s feelings. We are doing this to improve the site, so we need to hear your honest reactions.


With your permission, we will record the session because it’s hard to capture everything by note taking and we don’t have the possibility to go back if something is missing. The recording will only be used internally and seen by the people working on the project.

 


The session is scheduled on the [Xth of X XXXX at YY:YY] Berlin time. Please, ensure that you will have a high-speed Internet connection and approximately one hour of your time. Also, it is really important that you choose a quiet place without any disturbances to conduct the test.

 


[We are going to use lookback.io software for conducting a user test. You will receive the link to the test and the instructions 30 minutes prior to the test. Additionally, we would like to ask you to ensure that you have Google Chrome browser installed, as our testing software supports only this option.] / [We are still figuring out which tool we are going to use to do the session. I will probably send you a link shortly beforehand to invite you for screen sharing.] In the case of technical problems, we can connect beforehand via Skype. Therefore, could you please send me your Skype contact details?

[I will update you tomorrow morning on how exactly we will connect.]


Thanks again for helping us out with this, we appreciate your input a lot.

[I’m also sending you warm greetings from Diana Forster from our support team who had the idea of contacting you.]

If you have any further questions please don’t hesitate to get in touch.


Kind regards,

our company team


____________________________________________________________________

During the session

Before

  • Thank user
  • Explain why we are doing this test
  • Explain they can't do anything wrong
  • Explain that they are testing a prototype
  • Explain to really click on things, but not everything works, but we want to learn so it's good
  • Optional - Sign agreements (NDA, recording permission)
  • Explain to think out loud
  • Explain the role of the user
    • Explain the task of the user



    Cheatsheet text for pre-test: 

    Google doc _ Think Aloud pretest

    PDF Think Aloud pretest - version 09/06/17

    After

    Ask about the overall impression of the tested feature
    1. Do you think you would like to use this feature frequently?
    2. Was the process complex or straightforward for you? Could you predict well the result of the actions?
    3. How would you assess the integration of the various functions in the tested feature?
    4. Was the system consistent?
    5. In your opinion, how easy for other people would be to learn to use this feature?
    6. Did you feel confident using the feature?
  • Thank user, tell him/her how this helps
  • Tell him/her what we're going to do with is, when it goes live
  • Remind them to wait until the video uploading will be done completely in their browser (if using lookback.io)

    • Tips

    The focus is only on the pure user intentions and thoughts, reduce bias as much as possible:
    1. Don't explain, only ask questions. However, avoid straightforward questions which can be biasing as well.
    2. Don't answer questions during the session. Apologise and say that this is exactly the question which cannot be answered right now, but we can discuss it in the end of the test.
    3. As less "if" as possible (for example, "if this won't be here, if you were needing this")
  • Avoid tech or designer talk (shortcuts like UI, terms and specifications).
    • Don't interrupt the tester if he is reading or thinking, but sometimes do ask questions, especially if you see facial expressions.
  • Best practices for before: use Steve Krug's Usability testing script, show a brief video as an example of a think aloud test.
  • The strong recommendation is to use testing script during the process
    • Best practices for after: SUS (System Usability Scale)


    Reading:

    1. Different techniques of talking in order not to bias a user and to take the most from the observation (not a conversation): NNgroup - Talking with Participants During a Usability Test
    2. How to really observe users behaviour instead of relying on their words: NNgroup - First Rule of Usability? Don't Listen to Users
    3. Helpful usability.gov instructions to remote usability testing.
    4. NNgroup - Turn User Goals into Task Scenarios for Usability Testing