OurStories+Usability+Test

toc **OurStories** **Usability Test** by **Payam Azadi** (azadi@umd.edu), **Brandon Britto** (britto.brandon@gmail.com), **Dan Sonthichartkul** (dsonthic@umd.edu), **Kevin Heiting** (kheiting@terpmail.umd.edu)

Final usability study for OurStories interface; actual interface test using informed test subjects with the goal of discovering whether our interface is on the right track, and which parts need to be changed and in what ways.
 * Key: A= Kevin, B= Dan, C, D = Brandon

=Abstract=

Following prototyping an elementary level of functionality, essentially allowing stories to be submitted, viewed, and searched, we wanted to set out to see if our conceptual model would actually make sense to a lay person and that they would be able to figure out what the site does at first glance, as well, to feel invited by the interface rather than daunted.

The goal of this project rests in the desire of people to find, view, and tell stories, as well, to instill a deeper sense of connection with people around the world. We predicted that people would not have trouble figuring out how to perform these basic functions, but we could not be sure whether users would actually want to interact with it. Our prediction of "usability" was to a greater or lesser degree confirmed, however, it seems we still have some serious work to do in addressing the interface, and the fact that we are only in a prototype stage is only part of the problem. It is clear some of our methods need to be thought through more thoroughly, particularly our navigation and our story collection model.

The usability test was designed to be as simple as possible and many 'special considerations' or testing tactics were purposely not employed. For example, one test subject would be able to see changes made to the system (particularly story submissions) made by other users; independence was not established. This was because the interaction scheme is critical to overall function. Also we did not feel it necessary to monitor or record our subjects in the course of the study, however we did make it clear that we would be asking follow-up questions about their experience using the system. The testing was simply a task list of essential functions, but followed a particular sequence in order to test the quality of navigation. Prior to the test, non-identifying biographical information was collected from each subject.

=Testing Plan & Execution=

The test plan is to inform the subject that they will be part of a study on the usability of a new social website and will be asked questions about their experience afterward.. Subsequently they will be asked quick pre-test questions that will identify their demographic, then given a link to the OurStories main page and given a list of seven tasks to complete. Afterward we present follow-up questions and get feedback about their experience before concluding the subject experiment.

Pre-test questions

 * 1) What is your age?
 * 2) What is your gender?
 * 3) How would you rank your computer savviness on a scale from 1-5
 * 4) Occupation/Major?

Task List

 * 1) Find a random story from the home-page
 * 2) Submit a story
 * 3) Leave a comment on your story.
 * 4) Flag your story as inappropriate.
 * 5) Upvote your story. Now downvote your story.
 * 6) Go to the advanced search page and find your story.
 * 7) View your story

Subjective Feedback (5 star system)

 * 1) your comfort telling a story; if uncomfortable, why?
 * 2) ease of navigation; why was it easy/difficult?
 * 3) density of the homepage?
 * 4) attractiveness of the website?
 * 5) ease in finding stories?

Open-ended interview

 * 1) what does this website do?
 * 2) how easy did you find it to accomplish your task list?
 * 3) was the interface for telling a story inviting?
 * 4) was there any functionality you expected to have but was not available?
 * 5) at any time were you unable to navigate to where you wanted to go, i.e. hit a dead end?
 * 6) were the search fields sufficient
 * 7) was the search page intuitive?
 * 8) did you feel more connected to the international community

=Results=

This section will be divided into analysis of the individual test subjects' experience, followed by thorough analysis (including statistical) about the overall results, and accordingly an action plan for areas of interface that need improvement as part of the final product.

Subject A

 * Demographic**: 50 years old (older/elder), female, self-rated 1/5 star computer skill, entrepreneur


 * Comfort telling story**: //3//. Not used to the format of the form
 * Ease of navigation:** //3//. Not all buttons are in the same location/panel
 * Density of homepage:** //1//. Not dense, but confusing with the fake stories listed
 * Attractiveness of website:** //4//

A: to share your options and stories with others, share feedback, viewing short stories
 * Q: What does this website do?**

A: not intuitive at first, but easy once discovered the functions/links
 * Q: How easy did you find it to accomplish your task list?**

A: no
 * Q: Was the interface for telling a story inviting?**

A: wouldn't know what is expected for this type of website
 * Q: Was there any functionality you expected to have but was not available?**

A: needed assistance with finding buttons/links, but never hit a dead end
 * Q: At any time were you unable to navigate to where you wanted to go, i.e. hit a dead end?**

A: Yes
 * Q: Were the search fields sufficient**

A: Yes
 * Q: was the search page intuitive?**

A: Yes
 * Q: Did you feel more connected to the international community**

I observed some serious difficulty in navigating the website do to the lack of a well defined navigation panel. The use of links "share yours" and "Random Story" are not intuitive to users that are not computer savy. I believe we need to have a navigation panel with "Home", "Search", "Random Story", and "Submit Story" buttons. The basic search form in the side panel was also difficult to find and understand, especially with the "Go" button rather than "Search". The basic search panel should be made more intuitive and visible. My user seemed to not notice the "Surprise Me!" button during the find a random story task, and instead used the "Random Story" link. Unless other users found the "Surprise Me!" button to be useful, we should probably get rid of it. My user was also confused by the false links such "Story One", "Story Two".
 * Observations:**

Subject B

 * Demographic:** 25 years old (around target age), female, self-rated 3 star computer skill, accountant


 * Comfort telling story**: //2//. Not used to the format of the form
 * Ease of navigation:** //2//. Not all buttons are in the same location/panel
 * Density of homepage:** //1//. Not dense, but confusing with the fake stories listed
 * Attractiveness of website:** //4//

A: Allows users to read stories and share their own stories.
 * Q: What does this website do?**

A: Moderate. The interface is simple and easy to use.
 * Q: How easy did you find it to accomplish your task list?**

A: No, too formal and it looks outdated.
 * Q: Was the interface for telling a story inviting?**

A: Yes, like users can recommend stories to others by adding them into the current story's recommendation.
 * Q: Was there any functionality you expected to have but was not available?**

A: Yes, at first I can't find create a story link. It's at the top bar, but I think it's easier to find if it's at left menu panel. I couldn't find flag button because there's no description for the button. Also, I couldn't find advanced search page, so I used search from the left panel, then I can see the advanced search link.
 * Q: At any time were you unable to navigate to where you wanted to go, i.e. hit a dead end?**

A: Yes
 * Q: Were the search fields sufficient**

A: No, I put my firstname as an author and part of my story's title as a keyword but it doesn't return anything.
 * Q: was the search page intuitive?**

A: Yes In general the subject did not have trouble using the interface though at times it seemed that she had to spend a few moments thinking and looking at the different navigation areas in order to figure out how to proceed. However she did not ask any questions and was able to figure out how to accomplish all of her tasks without assistance. This is good news that someone who is close to our "typical user" demographic was able to figure out how to perform all pertinent tasks without getting stuck or overly confused. The major issue seems to be with consistency in the navigation and how daunting the submission form is.
 * Q: Did you feel more connected to the international community**
 * Observations:**

Subject C

 * Demographic:** 22 years old (inside target range), male, 5-star computer skill, computer science


 * Comfort telling story**: //5//
 * Ease of navigation:** //4.// some UI elements are unlabeled (flag, advanced search), but I was still able to find them with simple reasoning
 * Density of homepage:** //1// (not cluttered at all)
 * Attractiveness of website:** //3// (very clean, minimalist design, but not much color)

A: Allow users to share stories and find others' stories
 * Q: What does this website do?**

A: easy, although some of the terminology in the task list was different from the terminology on the page (page doesn't say "advanced search" on search box; only on results page)
 * Q: How easy did you find it to accomplish your task list?**

A: it was very intuitive, but the alignment of the form fields makes it look somewhat cluttered, and it makes it seem like there is more to do than there really is.
 * Q: Was the interface for telling a story inviting?**

A: Ability to list and browse through pages of all stories
 * Q: Was there any functionality you expected to have but was not available?**

A: When looking for "random stories" and "new story", I hadn't thought to look in the top navigation bar; I first looked to the left navbar, then the footer, then the top navbar
 * Q: At any time were you unable to navigate to where you wanted to go, i.e. hit a dead end?**

A: The advanced search fields allow me to search by any criteria I'd want. The basic search lists "United States" twice.
 * Q: Were the search fields sufficient**

A: yes, although I used the regular search box in lieu of the advanced search page
 * Q: was the search page intuitive?**

A: No, since I never saw any other users' stories
 * Q: Did you feel more connected to the international community**

As a computer scientist, some of the comments and feedback are not very useful to us, such as his ability to use reasoning to figure out UI features such as flagging. On the other hand many of his insights are very valuable, such as going to the header //last// order to find random story/submission links. Also, we never once thought of being able to page through all stories when it is such a painfully obvious and needed ability. His annoyance at the terminology and existence of 'advanced search' echoes that of all users tested through this point.
 * Observations**

Subject D

 * Demographic:** 21 years old (inside target range) female, 5-star computer skill, computer science


 * Comfort telling story**: //4//
 * Ease of navigation:** //4.//
 * Density of homepage:** //2//
 * Attractiveness of website:** //1//

A: Allowing you to share and view stories around the world
 * Q: What does this website do?**

A: Somewhat easy, confusing at times but I think that's just an issue with the interface
 * Q: How easy did you find it to accomplish your task list?**

A: Not really. Maybe adding some color would spice things up. I do appreciate the globe, ties in with the stories around the world theme
 * Q: Was the interface for telling a story inviting?**

A: Not necessarily
 * Q: Was there any functionality you expected to have but was not available?**

A: Yes there were multiple times I didn’t navigate to where I wanted. Finding a random story was not an easy task. I pretty much clicked on every link on the website, and was disappointed because it appeared that nothing was happening. I then glanced up, only to see that there was a link that said "Random Story." At first, I thought I would have to click a story and then click "surprise me". I really think that the "Share Yours" should be "Submit a story" or "Post a story" or "Share a story" or "Share your story" and should be on the left side. The exclamation mark was definitely not intuitive enough for me to associate it as the button to click for flagging inappropriate items. I find it very weird that there is nothing like "advanced search" on the homepage and that you need to search for some random thing for the "advanced search" button to appear.
 * Q: At any time were you unable to navigate to where you wanted to go, i.e. hit a dead end?**

A: Yes
 * Q: Were the search fields sufficient**

A: Yes
 * Q: was the search page intuitive?**


 * Q: Did you feel more connected to the international community** A: Not really

Leave it to a computer scientist to give such a critical and thorough beating of our interface. However it is to our advantage and everything said is echoed by all the other test subjects. According to this user our initial ideas about terminology and verbiage being friendly and concise opened a can of words that probably won't be fixed by consistency in links alone.
 * Observation**

Analysis


Overall using averages on the subjective ratings seems not to give us a very clear picture on where we need to work on, because the best 'interface density' score is 1 not a 5, and also the study overall is somewhat skewed due to half of the participants being very-savvy computer users.

However the truly valuable data comes out of the feedback that we received - the comments by each participant were almost identical. The most common theme was that people had great difficulty finding the 'random story' feature. Oddly, a lonely location at the top of the header seemed to be the last place people ever looked. This user's feelings seems to summarize the general feeling: "Finding a random story was not an easy task. I pretty much clicked on every link on the website, and was disappointed because it appeared that nothing was happening". The other most repeated comment is that there is this ambiguity between the terminology on our task list and that on the website. As predicted, the notion of a specifically named 'advanced search' seems to have caused a lot of confusion ("I find it very weird that there is nothing like "advanced search" on the homepage").

Other points of agreement among all subjects are lack of colors, loudness of story submission form, awkward terminology for form submission buttons, difficulty in finding the 'flag' feature, and ambiguity in purpose among the top header, the left navigation panel, and the footer navigation.

As a matter of priority, it is clear that the first challenge we must tackle is attaching strict purpose for navigation to the header, panel, and footer, and to stress consistency once function is decided upon. Secondly, terminology must be consistent, and we must be careful in mixing the terminology we use as developers with the terminology that is present on the website, specifically, 'advanced search'. Tertiary objectives are refining prettiness, buttons, etc and were known ahead of time to be causes of dissatisfaction. To be precise:

Priorities

 * 5 stars**: expressed purpose for header, panel, and footer navigations
 * 5 stars**: consistent terminology and linking structure across all pages
 * 3 stars**: Interface 'bugs', such as 'flagging' icon considerations
 * 2 stars:** Aesthetics; color, pictures, borders, etc.

=MS Word Original= Please note that this Wiki page is the most recent and authoritative source of the project; attached is the original document in MS Office 2007 before translation into Wikitext and accounting for minor revisions.