Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Decide if you’re using the introduction script (click to download) or use the points in the section "General Tester Instruction at Start of the Test". Print it if used. Translate it if needed.
  • Prepare the Scenario's you want to test, by adjusting the templates below. A scenario is 1 persona in 1 point in time in 1 system state (i.e. new user in empty system). (see examples below) (i.e. think of Scenario's (Scenario's) and tasks per Scenario, and what test data you need for that). By searching for the word "todo" you will find all area's to fill in.buying a book) 
    • Per Scenario define (sub) tasks you will give the user, i.e. search for the latest harry potter book, add the audio book version to your shopping basket, buy it with this creditcard. 
  • Determine what Test data you need for each Scenario and write it down below under "Scenario specific Preparation tasks before Test Day" (i.e. your bookshop will need relevant books, another scenario might need an existing user who has bought a book before to test the returns process)
  • Setup the system with this test data.
  • If you are testing on a Design or Product that's actively beind developed you might want to ask for a seperate stable version for the test period. 

...

Scenario specific Preparation tasks before Test Day

Scenario 1

  • Add a User with no (this is specific to your app. here are 2 example preparation points for a real estate app test)
  • Add a User with no purchased objects and login as below. And no profile image. One user for each tester that is coming in and one backup.
  • For each user: 4 objects to explore, 2 outside New York.(TODO: fill in your actual test (data) to prepare before)


Excerpt Include
Product Processes home
Product Processes home
nopaneltrue

Run a User Test

Process to be used during the actual test when the tester has arrived. 

Agenda

  1. Introduction Round (5 mins)
  2. General Instruction(5 mins)
  3. General User Interview Questions (20 mins)
  4. Scenario's  (40 mins)
  5. General After Scenario questions (15 mins)
  6. General After Entire Test script (5 mins)

General Tester Instruction at Start of the Test

  1. Thank user for his/her help.
  2. Explain why we are doing this test
  3. Get recording permission and start recording

User Interview questions

  1. What company do you work for?
  2. How many employees work there?
  3. Do you use any similar software?
  4. (if yes) What device(s) do you use it on?

Scenario 1 - Facilitator part

General Tester Instruction at Start of the Scenario

  • Explain they can't do anything wrong
  • Explain we want honest opinions, don’t worry about hurting our feelings
  • Explain that they are testing a prototype
  • Explain to really click on things, but not everything works, but we want to learn so it's good
  • Explain to think out loud
  • Explain the role of the user   - NOT NEEDED: Is on tester handout.
  • Explain the task of the user - NOT NEEDED: Is on tester handout.
  • Handout the Scenario Tester handout.
  • Cover up tasks 2 and further (we reveal them 1 by 1).
  • Ask the user to read and start with task 1 afterwards.

Questions after a Task

Question for after a Task was completed. The numbers of the questions below are the same as the task numbers in tester handout

  1. What would you do now the password is not working?
  2. What’s your first impression? Is it clear to you what the different parts of the application are for (objects/wishlist/etc)?
  3. Did you notice what cities the objects are from?
  4. Is it clear to you what you bought exactly? Did you have all the information you needed? What do you expect will happen next?
  5. What do you think the difference is between the 2 emails? What do you think the picture to the top left is for (on objects screen)?
  6. (TODO: replace these examples with your actual questions)

Questions after Scenario

General questions for after the scenario was completed  all Scenarios

  • Ask about the overall impression of the tested feature (or have him/her fill in the SUS (System Usability Scale) first and then discuss it).
    • Do you think you would like to use this feature frequently?
    • Was the process complex or straightforward for you? Could you predict well the result of the actions?
    • How would you assess the integration of the various functions in the tested feature?
    • Was the system consistent?
    • In your opinion, how easy for other people would be to learn to use this feature?
    • Did you feel confident using the feature?
    • How would you improve this app?
    • (TODO: replace these examples with your actual questions) 

 Scenario Specific questions  for after the scenario was completed

  • (TODO: replace these examples)

(TODO: Copy paste the next scenario here if there is one, otherwise remove this line.)

After all Scenario's have been completed

  • Thank user, tell him/her how this helps
  • Tell him/her what we're going to do with is, when it goes live
  • Handout the gift voucher or tell him/her when it will be mailed. 

After Tester left

  • Reinstall app to clear out login data.
  • Make sure everything is ready for next test
  • If time allows: Evaluate.

Scenario 1 - Tester Handout

Tip

Scenario

You are a Agent who work at a small 3 person company in New York. You signed up with us because offer potential clients to you (you are paying a certain service fee). They told you in an email that you can use their service via the app or via an Iphone app and you decided to give the Iphone app a go. The login details that they emailed you are below. For this test please ignore spelling errors and the fact that many words are not translated to German yet. Also ignore the menu in the bottom of the screen.  Remember to think out loud.

Login data

Username:

Password: 

Tasks

Remember: Please think out loud the whole time. No worries, you cannot do or say something wrong.

...


...

Scenario 1 - Tester Handout


Tip

Scenario

You are a Agent who work at a small 3 person company in New York. You signed up with us because offer potential clients to you (you are paying a certain service fee). They told you in an email that you can use their service via the app or via an Iphone app and you decided to give the Iphone app a go. The login details that they emailed you are below. For this test please ignore spelling errors and the fact that many words are not translated to German yet. Also ignore the menu in the bottom of the screen.  Remember to think out loud.

Login data

Username:

Password: 

Tasks

Remember: Please think out loud the whole time. No worries, you cannot do or say something wrong.

  1. Try to Login to the system.
  2. The correct password is xxxxxxx. Login to the system and tell us your first impression. If any questions or concerns popup let us know. Do not yet Edit or Add something yet.
  3. Compare objects and tell us which one looks most interesting to you
  4. Try to buy an object you like. You want to pay by invoice. Go through it step by step and tell us what you are thinking.
  5. You want change the email that you use to login. Find out if its possible (no need to actually change it, if it’s possible).


...


Run a User Test

Process to be used during the actual test when the tester has arrived. 

Agenda

  1. Introduction Round (5 mins)
  2. General Instruction(5 mins)
  3. General User Interview Questions (20 mins)
  4. Scenario's  (40 mins)
  5. General After Scenario questions (15 mins)
  6. General After Entire Test script (5 mins)

General Tester Instruction at Start of the Test

  1. Thank user for his/her help.
  2. Explain why we are doing this test
  3. Get recording permission and start recording

User Interview questions

  1. What company do you work for?
  2. How many employees work there?
  3. Do you use any similar software?
  4. (if yes) What device(s) do you use it on?

Scenario 1 - Facilitator part

General Tester Instruction at Start of the Scenario

  • Explain they can't do anything wrong
  • Explain we want honest opinions, don’t worry about hurting our feelings
  • Explain that they are testing a prototype
  • Explain to really click on things, but not everything works, but we want to learn so it's good
  • Explain to think out loud
  • Explain the role of the user   - NOT NEEDED: Is on tester handout.
  • Explain the task of the user - NOT NEEDED: Is on tester handout.
  • Handout the Scenario Tester handout.
  • Cover up tasks 2 and further (we reveal them 1 by 1).
  • Ask the user to read and start with task 1 afterwards.

Questions after a Task

Question for after a Task was completed. The numbers of the questions below are the same as the task numbers in tester handout

  1. What would you do now the password is not working?
  2. What’s your first impression? Is it clear to you what the different parts of the application are for (objects/wishlist/etc)?
  3. Did you notice what cities the objects are from?
  4. Is it clear to you what you bought exactly? Did you have all the information you needed? What do you expect will happen next?
  5. What do you think the difference is between the 2 emails? What do you think the picture to the top left is for (on objects screen)?
  6. (TODO: replace these examples with your actual questions)

Questions after Scenario

General questions for after the scenario was completed  all Scenarios

  • Ask about the overall impression of the tested feature (or have him/her fill in the SUS (System Usability Scale) first and then discuss it).
    • Do you think you would like to use this feature frequently?
    • Was the process complex or straightforward for you? Could you predict well the result of the actions?
    • How would you assess the integration of the various functions in the tested feature?
    • Was the system consistent?
    • In your opinion, how easy for other people would be to learn to use this feature?
    • Did you feel confident using the feature?
    • How would you improve this app?
    • (TODO: replace these examples with your actual questions) 

 Scenario Specific questions  for after the scenario was completed

  • (TODO: replace these examples)


(TODO: Copy paste the next scenario here if there is one, otherwise remove this line.)

After all Scenario's have been completed

  • Thank user, tell him/her how this helps
  • Tell him/her what we're going to do with is, when it goes live
  • Handout the gift voucher or tell him/her when it will be mailed. 

After Tester left

  • Reinstall app to clear out login data.
  • Make sure everything is ready for next test
  • If time allows: Evaluate.


...

Excerpt Include
Product Processes home
Product Processes home
nopaneltrue