Barbee, M. A. (2004). Web design and usability testing for school web sites. Instructional Technololgy Monographs 1 (1). Retrieved <insert date>, from http://projects.coe.uga.edu/itm/archives/fall2004/BarbeeAppProj.htm.

 

Web Design and Usability Testing for School Web Sites

By

Mary A. Barbee
University of Georgia

Abstract

The web site design elements commonly accepted and applied in the development of commercial sites improve the usability of school web sites.  In this study, key elements of good web design were developed from the literature, and then applied to a school web site to determine whether application of these principles made the site easier to use.  A school web site was evaluated, revised, and then usability testing began.  Participants in the study were identified as representatives of target users for the school web site.  Each participant was asked to complete tasks on the web site.  Video and audio taping of the participants as they completed tasks revealed problem areas, and another round of revisions to the web site was completed.  Using this third iteration of the web site, new participants were given the same tasks and their responses were recorded.  Ultimately, web site design improvements positively impacted the usability of the site.

Introduction

It is commonplace now for businesses, organizations, individuals and schools to use web sites to communicate information to targeted audiences.  The World Wide Web has exploded tremendously in the last five years, and schools are struggling to develop sites that can keep up with the sophisticated sites that businesses are now offering.  As consumers more frequently turn to their computers to research and make purchases, as well as to investigate travel options or look up a phone number, their expectations are raised to higher levels.  Combined with the fact that school personnel such as technology specialists and media specialists have had little formal training on web design or web development tools, school web sites are often poor in quality and difficult to use in comparison to commercial sites.  Though much information is there, visitors to the site have a hard time finding it or do not use the site because it is not appealing.

At the same time, school administrators expect technology coordinators, media specialists and sometimes teachers to maintain web sites.  Parent expectations that educators should be current with this technology have forced schools to develop a web presence with or without the training and the tools.  As computer use in the home increases, more parents and students go first to the school web site for information.  In addition, new community members will use the web to investigate educational opportunities for their children.  As time goes by, a parent’s first impression of a school will come from his/her computer screen.

The Research Question

Can the application of elements of good web design developed in the business world contribute to the usability of a school web site?  Research was conducted on the basic elements of web design and usability.  From the literature in this area, guidelines for effective web design were established.  Once established, these guidelines were applied to an existing school web site. Standard practices in usability testing were applied to the testing of the school site after implementation of design changes based on general guidelines.  Testing was repeated in an iterative approach to improving the usability of the school web site.  Acknowledging that accurate and relevant content is essential to the effectiveness of a well-designed site, questions or tasks developed for the usability test represented identified key information elements for a school web site.

As most schools and media centers already have a web site, the first issue to be researched was the elements of good web design and their use as an assessment tool for evaluating existing school web sites.  Elements of good web design have been established through research and seem to apply to all types of web sites.  Determining what those elements of good design are and being able to assess how well your site meets them is the first step in this process.

This research focused on general web design with an emphasis on school web pages that are part of a school web site.  Usability testing focused on the school home page and navigation to subsequent pages as it explored the elements that the typical user would be looking for.  The elements of web design and the usability testing methodology were applied to the entire school web site.

Limitations

            A usability test by definition uses a very small sample of participants as representative users.  Participants should represent all major groups of targeted users who will have a broad range of familiarity with the school web site and the web in general.  In addition, participants should represent the socio-economic, gender, age and ethnic makeup of the targeted audience.  With such a small sample, it is difficult to thoroughly represent all of these groups.  Also, with only three participants in each round of study, it is possible that some participant responses will be more unique to the participant than representative of the target audience. 

            The local school technology coordinator and webmaster of the school site being tested assisted in the research by recruiting the participants.  As outlined by the researcher, the participants selected represented a sample of genders and age groups in the school population.  A broad range of computer and web skills were represented as well.  At the same time, each of these participants worked with the technology coordinator in some capacity.  Teachers have a professional relationship with the technology coordinator; students interact with her as a teacher and authority figure, and both parents had worked with her on school projects as volunteers.  The researcher observed that all participants were reluctant to make any negative comments regarding the web site and its design.  Though the technology coordinator’s voluntary participation was emphasized in the pre-test instructions, all seemed hesitant in the post-test interview to make comments that might reflect negatively on her.   Improvements in subsequent iterations were developed almost solely on the actual usability test, while the post-test interview was most useful for highlighting the positive features of the site.  Unlike the usability testing of a commercial site where participants have no relationship with the owner or webmaster of the site, school site testing includes an inherent bias that is difficult to eliminate.

            None of the participants were novice computer users and all had experience on the web.  The results of this study are somewhat limited in that they do not evaluate what an inexperienced web user would encounter if trying to use the school web site.  Many of the elements of good web design rely on conventions that have become commonly accepted, such as using blue font color for hyperlinks.  The research suggests that a site should be developed with the assumption that users have no prerequisite web skills (Mok, 1996).  Using only inexperienced participants could affect results.  Therefore, one of the limitations of this study is that it does not measure usability of the site for the novice web or computer user.

Literature ReviewLiterature Review

Elements of Web Design

The primary elements of good Web design are summarized in Table 1.  The text that follows describes the major elements as gleaned from the literature.

Table 1

Elements of Good Web Design

Leaders in the field of web design have defined the elements of good web design in a number of different ways.  Mok (1996) provided basic design guidelines in his “4Cs” model that deal with content, control, consistency and corroboration.  The initial step in designing a site according to Mok (1996) is establishing the content or information that will be presented.  Control elements involve the layout of the page, navigation of the site and the location of information.  The basic rule of thumb here is to create a site as if every visitor is a first time visitor.  Most users are interested in visiting the site, getting the information they need and exiting as quickly as possible.  Although experienced web users may have become accustomed to conventions such as navigation bars on the left or underlined hyperlinks, there are still many users who may be exploring the web without such familiarity.  Developers should not assume any level of prerequisite knowledge on the part of the user.  

The third C in Mok’s model is consistency.  Interactivity should be simple and consistent.  Fonts, colors, location of text and icons should remain the same from page to page.  Background colors should be solid colors that provide contrast with the text.  The overall effect should allure users to your site.  Finally, corroboration is a key element of design for Mok.  Often overlooked, items like contact information, copyright statements and the date of the last web site revision are very important to the user.  The author of the site establishes credibility with the users through these design elements.

In the design of a school web site, Mok’s 4 C’s are clearly relevant.  First, the technologist (assumed here to be the author of the site) must establish the goals for the site including the content to be included.  Visitors to the site will change as frequently as the student population does.  Consistency should contribute to successful navigation of the site as it facilitates quick and easy access to information.  Corroboration is important for the technologist as the school may be an unfamiliar entity for many users and therefore will not have previously established credibility.

D’Angelo (2000) examined current research on web design as well as studies in related areas such as the use of color in screen design.  He then completed his own study, which resulted in a research-based set of guidelines for design of successful web pages commonly known as the D’Angelo standards.  These standards assume that the purpose of any web page is to communicate information and that the design of the page is important to the comprehension of that information.

D’Angelo’s standards built on previous research and often reflect those findings.  He echoes the admonition to sign and date each page giving credibility or corroboration to the information contained within.  Contact information is critical and should include a mail-to command or a form that the user can submit with questions.  In addition, D’Angelo recommends placing the site URL on each printed page so that users can easily revisit the origin of the printed text.

The remaining D’Angelo standards fall into categories that deal with visual quality.  Visual characteristics of a well-designed web page include a simple and organized layout using short, well-structured sentences.  Links should be descriptive, and navigational options should appear at the top of each page.  Fonts should be consistent and bold, and italics should be used sparingly, while all caps should never be used for text.

The navigational characteristics addressed by D’Angelo include links on every page to the home page and to a help page.  Links to a single location should be consistent.  Be judicious in the use of links to outside pages, with links to related organizations only, preferred.  Menus should be clear and understandable to the user, not the author.

Finally, D’Angelo establishes some practical considerations.  Again the theme of consistency as a key element to good web design is emphasized.  Consistent use of icons, button styles, bullets and navigational items is imperative.  Use of graphics should be limited to no more than three per page and should be designed to load quickly.  Use only light blue, white or gray for backgrounds with contrasting text.  Indicate action with warm colors, and use bright colors for emphasis. 

As content is the primary consideration in developing a web site, D’Angelo suggests putting the most important material at the top of the page and including a search capability.  He suggests taking care to avoid abbreviations that your users may not understand.  Finally, avoid placing information on your site that must be updated in more than one place.

Many elementary school web sites violate the majority of these standards.  School sites often enjoy the use of a lot of color, animated graphics and varied fonts.  Like a kindergarten classroom, the home page represents a flurry of activity with more to see than you can take in at a glance.  However, bold use of color is hard on the eyes as is the use of some fonts.  The sensation that there is too much to see in a quick look will drive many web users away.  The problem with this view of web design is that the research does not support it.  Jakob Nielsen (2000), a renowned expert on Web usability, contends that ninety percent of web sites are poorly designed and difficult to use.  Developers’ biggest mistake is attempting to take what works in print and transform it to the web.  He advises an approach that is unique to the web.  Text should be concise, graphics minimal and organization should be simple.

            Nielsen (2000) asserts that another major flaw in web design can be traced to a lack of user focus.  Developers concentrate on complex designs, forgetting what the user wants.  Most users want to quickly solve a problem or answer a question and leave the site.  If they are not able to navigate quickly to the solution, they will just leave.  At all times, Nielsen says, the user should know where he is and what this site can do for him.  Another key element to keeping a user coming back to your site is content that is relevant and current.

The principles of good web design become a complex set of rules for the web developer whose primary concern is content.   A decision to make a change to one page requires many more changes to other pages to maintain the consistency and control desired.  Web developers who are attempting to keep information current actually become consumed by the time it takes to update web pages.  Lehman (2003) suggests the use of the Cascading Style Sheet (CSS) to maintain up-to-date information.  Once awkward for the novice to use because of how they were handled by different browsers, CSS now allows complex pages to be rendered and updated consistently. 

Another often-overlooked but key element to maintaining a good web site is the “8-second rule.”  In 1999, a Zona Research report, “The 30-second rule” (2002) stated that if a company’s web page did not download in eight seconds, users would go elsewhere.  Nielsen (2000) allows up to ten seconds, but warns that any more than that and the user will turn to other tasks or leave the site altogether.  Loh (2002) in a study on student perceptions of web sites, attempted to determine what children felt were the most important elements of a “cool” web site.  Children in this study ranked quick loading as the top feature of a cool web site above navigational features, interactive features and multimedia features.

Again, the research should be relevant for school web sites.  Users are not likely to be more patient with the challenges of inconsistent labeling, poor navigational layout and slow page loading because they are looking at a school page.  In fact, a school risks losing credibility with its audience when its web site reflects poor planning and/or inadequate technological skills of its developers. 

Summary

Research regarding the elements of good web design in the business world is very consistent.   There is little disagreement on what makes a web page effective.  Nor is there much argument with the concept that the primary purpose of a school web page is to communicate information to a targeted audience.  So why do so few school web sites follow the guidelines created by the experts?  Does the targeted audience for a school site have different needs and expectations?  Or, do they need and expect what they are seeing on commercial sites and therefore become frustrated with school web pages?  A usability study will help determine whether good design will make a school web site more effective.

Methods

In order to identify the key elements of a well designed school web site, usability testing was selected as the methodology.  Nielsen (2000) contends that the most important thing designers can do to test their sites is to use real people and direct observation.  Surfers who arrive at your page should be able to quickly find what they want and know immediately why they should stay.

More important than what users say about your site is how they act when visiting it.  Usability testing is a user-centered approach to discovering how easily a web site visitor can accomplish common tasks critical to the site’s target audience.  As Dickstein (2000) outlines, key to the success of usability testing is the iterative approach that allows a small subset of users to view a prototype and identify problems or suggest changes.  Feedback provided results in another version of the page or pages to be evaluated and the testing continues.   Each version of the web pages provides additional insight on what makes your site usable for visitors.

Instruments

            The usability test itself was a list of tasks or scenarios that participants were asked to complete.  The complete test took no more than forty-five minutes and was limited to eight to ten tasks.  Scenarios were developed by the local school technology coordinator and researcher from a list of top reasons users visit the school web site and represented common tasks that representative users would attempt upon visiting the site. 

            A script was then used to administer the test (see Appendix A).  The script provided consistency and assured that information provided was the same for all participants.  A key element in the script was the expectation that the participant think out loud as he attempted each task.  Participants were reminded that it is the site that was being “tested” not the user.  Every attempt was made to reassure users that they were not being evaluated or critiqued.

            After the script was read, the participant was presented with the list of tasks or scenarios to be completed (see Appendix B).  A post-test interview was also conducted with each participant to get more information about their experience with the web site (see Appendix C).

Participants

            Participants for a usability test should be a representative sample of the target audience for the web site that is being tested (Nielsen, 2000).  In the case of a school web site, school personnel can be asked to identify the target groups.  Parents, teachers and students were among those that were questioned regarding their interests in using a school web site.  When the primary target audiences were identified, two representatives were selected from each group.  No more than five to seven participants are required to participate in a usability study. 

            For this study, the Britt Elementary School web site was used for testing.  The participants represented the target audiences as identified by the Local School Technology Coordinator (LSTC).  Participants included a parent, a teacher and a student and are described with attributes as of the time of the study. Parent one was very active in the local school Parent Teacher Association and volunteered at the school often.  She had two children at the participating elementary school.  Parent one was an inexperienced web user who had no familiarity with the school web site before the usability test.  She had a home computer, but admitted that her children were more proficient at using it than she feels she was.  Parent Two, like Parent One, was very active at the school.  In addition to volunteering, she also substituted for teachers at times.  She had one child at this elementary school.  Though Parent Two described herself as a frequent web user, she had little experience with the school web site.  In her post-test interview, Parent Two mentioned that her children could probably have completed these tasks more easily than she did.

Teacher One was an experienced web user who had used a previous iteration of the school web site.  Of all participants, she was most familiar with the content of the site and with web site conventions in general.  Teacher One had worked with the school technology coordinator on class projects involving technology integration.  She seemed the most comfortable with the testing environment.  Like Teacher One, Teacher Two was an experienced web user who seemed comfortable completing the assigned tasks.  However, Teacher Two was not as familiar with the school web site as she had rarely used it before.

Student One, a fourth-grader, was an experienced web user who had never visited the school web site.  While he was unfamiliar with the content of the school site, he was very familiar with navigation and some web site conventions.  Student Two, a fifth-grader, was also an experienced web user who expressed no concern over navigation or exploration of the school web site.

None of the participants expressed concern over the video or audio taping of the test.  All participants were asked to think aloud as they worked through the test; the two teacher participants were the most comfortable with this portion of the test.  The students had to be coaxed during the test to express their thoughts and still had little to say.

Procedure

            Once the participants listened to the instructions provided by the script and were presented with the tasks to be completed, the researcher’s job was to observe how the participants responded.  Direct observation was accomplished in a number of ways.  Video taping of the users’ facial responses to the tasks and their attempts to complete them along with a video record of the steps they actually completed was the most reliable way to measure the difficulty of tasks for given users.  Other measures included the amount of time it took the participant to complete the task as well as the number of clicks or screens they went through until done.

Audio taping of the participants thinking out loud while they worked was another important measure of the usability of a site.  Users revealed that link labels were confusing or that the location of navigation items was unexpected as they talked through their thought processes.  How the site made users feel was also revealed by both the tone and the content of their comments.

Results and Discussion

The school Web site changed dramatically as the result of the design recommendations made before the first usability test.  The webmaster made a concentrated effort to adhere to the guidelines provided by the researcher when revising the site.  As a result, the site tested out in the first round as highly usable and posed few problems for even the most inexperienced user.

            One frustration common to the users was the lack of a Home button to link back to the home page from any other page.  In fact, such a link was available, but it was not labeled Home.  If users had scrolled over the star logo in the upper left corner of all screens, they would have seen that this graphic was a link to the home page.  The usability test demonstrated that though this was a clever approach, it was all but wasted on these test users.

            Vague language was also a common frustration.  Though each user found the Cafeteria menu with no problem, they were confused as to which option to explore to discover the cost of breakfast or lunch.  The label “Meal Pay” suggested to these users that it should provide pricing information.  However, it led to a vendor web site used for putting money on the child’s lunch account.   On his first attempt to get pricing information, Student One, immediately chose the Meal Pay option and spent a great deal of time exploring that site for pricing information.  Even after he eventually found the information under Cafeteria Policies, he did not remember that experience when he again went searching for pricing information.  To complete the second task regarding meal cost, Student One again went to Meal Pay first.  When the other web site launched, he immediately recognized that he was in the wrong place, exited and selected the correct option.

            Parent One, an inexperienced web user had trouble locating the link for making a request to add information to the web site.  She was unfamiliar with the common location for such a link and so did not even know where to look.  Only Teacher One found the webmaster link and knew that it was used to submit requests for enhancements or additions to the web site. 

            The drop-down menus were an asset to all users in the first round of testing.  The ability to roll over a word and see the next level of menus was utilized frequently.  Rather than take the time to consider what might be under each link, users would roll across them and look at the options.

            Broken or dead links were another area of frustration for first round test participants.  In the case of the mission and vision task, the link appeared on some pages and was missing on others.  For that reason, Parent One found the mission and vision right away.  Teacher One, who began on a different page, had to go through several screens to find the appropriate information.

            Information that was important enough to put on the front page as news was virtually overlooked by all users in the first round of testing.  The continuous paragraphs of text were not considered when participants were looking for information.  On the home page, users would check links and consult the calendar, but the news section did not help them at all.

            Most of the areas of concern were addressed in the second round of testing.  A Home link was added to every page except the index page.  Every user in the second round of testing used the Home link to navigate back to the home page. 

            The Cafeteria menu represented the most problematic link labels.  Confusion between Meal Pay and Cafeteria Policies seemed to be the result of vague or inexact language.  As Meal Pay is the name of the service provided by the web site that launches, the webmaster wanted to keep that language.  However, she added the word Account so that the link now reads Meal Pay Account.  Cafeteria Policies was changed to Cafeteria Pricing and Policies.  For two of the three users, these changes made a difference.  The student still went to Meal Pay before Cafeteria Pricing to find the cost of a meal.  Once he realized he was in the wrong place, he went back, reviewed the menu again and went directly to the correct location.

            The webmaster link was difficult for both groups of participants.  The task was to place a request to add registration information to the web site.  Only Teacher One understood that emailing the webmaster was the way to accomplish this task.  Parent Two, in her post-test interview admitted that she saw the webmaster link, but was not sure who that was and if he/she would be the right person to talk to about registration information.  Parent One and Teacher Two never noticed the link at all.

            The most effective navigational tool on this site was the rollover drop-down menus.  Users in both groups used the strategy of scanning the menus before choosing a link to click on.  Scanning the menus amounted to looking at a site map for these users.  In later tasks, they would frequently think aloud by saying, “I saw that under Students.”  In scanning for other information, they became more familiar with the web site as a whole without making errant clicks.

            Broken links did not pose as much of a problem in the second round of testing as they did in the first.   Two different users each found a dead link.  In both cases, they expressed frustration at having found what they were looking for and then not having it work.

            Finally, information that was lost in the news section for the first group, was quickly located in the second group.  The webmaster added headlines that were very effective.  Users in the second group read the questions and in some cases began scanning menus immediately.  However, as they scanned, their eyes would pick up the key words of the headlines and they would move to that portion of the home page for the appropriate information.   Two users were asked to locate information in the top story on the page.  Student Two had to look in the second story which still appeared on the screen without scrolling.  Student Two found the information he needed while Student One never did; however, it took Student Two longer than the other members of his group whose headline was closer to the top of the page.

            Usability testing is an efficient and effective method of determining ease of use for a web site (Nielsen, 2000).  Using only six participants, the researcher was able to identify key weaknesses in the web site.  Because participants represented the key target audiences and a range of computer skills, recommendations covered a broad area of concern. 

            This school site definitely benefited from the application of web design principles.  Some factors that were incorporated in the first iteration that was tested were actually elements that participants commented on in the post-usability interview.  The use of the dark font on the yellow background was popular with all participants.  Limiting the number of fonts used contribute to the easy-to-read, pleasant feel of the site.  The simple design and clearly labeled links were assets to the user in locating needed information. 

            Some of the improvements made to the site validate the research on web design.  The Home link on all pages has become expected and caused tremendous confusion for the participants in this study.  Navigational times were good in both rounds of testing, but greatly improved when the Home link was used. 

Minor adjustments to the language of the Cafeteria menu made a significant difference in the ease with which users found the information they needed regarding the price of meals.  In general, users were complimentary of the plain, easy to understand language used on the site.  All rated the site as easier to use than other sites they visit regularly.

            Another consistent problem area was the use of the home page for news stories.  It is common practice for schools to put important or timely stories on the home page, but this test confirmed that web users scan, not read.  All participants scanned menus for information that was actually on the page they were looking at.  In the first group, no one read the news stories.  In the second group, users found the information because the larger font headline jumped out at them as a link does.  Once the news story was identified as the place to get the information needed, one user still jumped immediately to an embedded link.  Again, he did not take the time to read the text, but assumed that a list of winners would appear on a subsequent screen.  If a web site uses a news section, it must be clearly labeled with words that will catch the scanner’s attention.

            A commonly neglected area of web design is the maintenance of links (D’Angelo & Little, 1998).  Participants in both groups were impatient and frustrated with broken links.  Though the webmaster made an attempt to correct this problem in the second iteration, broken links still surfaced.  Only regular monitoring of all of the links on the site will correct this potential problem.

            Do the elements of good web design impact the usability of a school web site?  This research suggests that indeed it does.  For each element that was updated per the researcher’s recommendations, subsequent users were more successful.  For the one element that was not revised, the webmaster link, participants continued to have problems completing the task.  Using the elements of web design as a model for improvement against simple usability testing is a solid approach for the development of an effective web site. 

References

The 30‑second rule. (2002, July 22). Network World, 39. Retrieved April 13, 2004, from Infotrac database.

Barnd, S. M., & Yu, C. (2002, May/June). Creating an effective web site. Journal of Physical Education, Recreation and Dance, 73(5), 11‑13.

D’Angelo, J., & Little, S. (1998). Successful web pages: What are they and do they exist? Information Technology and Libraries, 17(2), 71-81.

D’Angelo, J., & Twining, J. (2000, September). Comprehension by clicks: D’Angelo standard for web page design, and time, comprehension and preference. Information Technology and Libraries, 19(3), 125‑136.

Dickstein, R., & Mills, V. (2000, September). Usability testing at the University of Arizona Library: how to let the users in on the design. Information Technology and Libraries, 19(3), 144‑152.

Lehman, T. E. (2003, June). Mastering the language of web design. Information Technology and Libraries, 22(2), 93‑95.

Loh, C. S., & Williams, M. D. (2002, Spring). What’s in a web site? Student perceptions. Journal of Research on Technology in Education, 34(3), 351‑363.

Mok, C. (1996). Designing business: Multiple media, multiple disciplines. San Jose, CA: Adobe Press.

Nielsen, J. (2000). Designing web usability: The practice of simplicity.  Berkeley, CA: New Riders Press.

Riccardi, M., Easton, D., & Small, R. (2004, February). Web site evaluation: how would your school’s web site measure up? Teacher Librarian, 31(3), 19‑22.

Warlick, D. (2002, September). Plan it. Design it. Build it. Put your web site to work. Technology & Learning, 23(2), 22-31.

 Appendix A

School Web Site Usability

Pre-test Instructions Script

 

Welcome.

My name is Mary Barbee, and I will be conducting today’s usability test.

Today we are evaluating how easy the school’s web site is to use by measuring how quickly and easily users can accomplish certain tasks.  We are doing this for research purposes regarding web design and to help us make improvements to the site so that parents, students and teachers will find it more helpful. 

This is not a test of your ability on the computer or of how good you are at finding information on a web site.  It is a test of how well the site directs users to the information they need.  Please perform and react in whatever manner is comfortable or typical for you.  Once all sessions are completed, I will compile the feedback received from each participant, along with my notes so that I can make suggestions for revisions of the school web site.

Explain the hardware setup and confirm that the user is comfortable with it.

We will be videotaping this session to help us understand how users feel and what they are thinking as they work through the tasks.  These videotapes will be erased by December 31, 2004.

Explain the camera setup and usability lab setup and confirm that the user is comfortable with it.

The video and audio of your session will be confidential and will be viewed by the researcher only.  Please sign this consent form indicating that you are participating voluntarily.

Have participant read and sign both copies of consent form.

Do you have any questions?

Give the participant the task list.

This usability test consists of a series of tasks that are evaluated separately and sequentially.  Work through this task list in order.  As you work through each task verbalize your thoughts.  Thinking out loud will allow the researcher to understand how you view the different elements of the web page. 

I will not be able to help you complete the tasks; however, I may prompt you to share your thoughts as you work.

Complete a practice task with the participant, modeling the thinking aloud process. 

Do you have any questions?

Have participant complete as many tasks as possible.

After all tasks are complete or the time expires, each participant will be debriefed by the researcher.  The debriefing will include the following:

·          Completion of a brief post evaluation interview in which the participant shares his opinions on the web site’s usability, appearance of screens, and general impressions of the site

·          Participant’s overall comments about his or her experience

·          Participant’s responses to probes from the researcher about specific errors or problems encountered during the evaluation

Complete post-test interview with participant.

Thank you for participating in our study.  Your feedback will help make a better web site for your school.

Appendix B

School Web Site Usability

Participant Task List

Parents

Task 1

Locate directions to Britt Elementary School.

Task 2

Add money to your child’s cafeteria account.

Task 3

Find the phone number for Britt Elementary School.

Task 4

When is Breakfast with Grandparents for 4^th grade?

Task 5

How much is lunch? 

Task 6

Locate the mission and vision for Britt ES.

Task 7

Find contact information for the PTA president.

Task 8

Locate this year’s attendance policy according to the student handbook.

Task 9

Place a request to add registration information to the web site.

Appendix C

School Web Site Usability

Post-test Interview

Have you used the school web site before?  How often?  For what purposes?

Do you use other web sites often?  How often?  Can you give me some examples of web sites you use frequently?

How would you describe your computer skills?  Do you use the computer often?

Did you feel that you found information easily on this web site?

How did you feel as you were completing these tasks?

Which task was easiest to complete?  Hardest?

Compared to other web sites you visit, how would you rate this one?

What three words would you use to describe this site?

What was confusing about this site?

Did you like the colors used on this site?  The fonts?  The menus?

How did you feel about the navigational features of the site?

Was there anything you would like to change about this site?

Appendix D

School Web Site Usability

Participant Task List Results

Parents

Task 1

Locate directions to Britt Elementary School.

• Both participants located information in minimal time and clicks.

Task 2

Add money to your child’s cafeteria account.

• Parent One completed in minimal time and clicks.
• Parent Two first used the Meal Pay link found on the Cafeteria Pricing page, and that link was broken.  She was able to navigate back to the Home page and locate the Meal Pay link from the Cafeteria menu.

Task 3

Find the phone number for Britt Elementary School.

• Parent One immediately located the number in the header of the current page.
• Parent Two began exploring the School drop-down menu when she noticed the information at the top of the page.

Task 4

When is Breakfast with Grandparents Day for 4^th grade?

• Parent One was unable to locate this information after repeated false starts.  She explored every drop-down menu on the home page looking for relevant links.
• Parent Two located this information immediately on the home page as part of a news article that included a headline regarding Breakfast with Grandparents.

Task 5

How much is lunch? 

• Parent One voiced uncertainty regarding which of the Cafeteria menu options would include that information.  Meal Pay and Cafeteria Policies were both considered.  She elected to try the first option on the list, Cafeteria Policies, and found the information there.
• Parent Two went immediately to the Cafeteria menu options and selected Cafeteria Pricing and Policies to find the information in minimal time and clicks.

Task 6

Locate the mission and vision for Britt ES.

• Both participants found this information in minimal time and clicks.

Task 7

Find contact information for the PTA president.

• Both participants found this information in minimal time and clicks.

Task 8

Locate this year’s attendance policy according to the school handbook.

• Parent One located the student handbook under PTA link.  Once she opened the pdf file, she was uncertain how to navigate to the desired section and abandoned the task.
• Parent Two found this information in minimal time and clicks.

Task 9

Place a request to add registration information to the web site.

• Parent One never scrolled to the bottom of the home page and was unable to complete this task.
• Parent Two explored the home page and located the “email the webmaster link,” but was unsure of its purpose.  After exploring the drop-down menus, this participant abandoned the task.

School Web Site Usability

Participant Task List Results

Students

Task 1

Locate a resource that provides information on rainforests.

• Both participants found this information in minimal time and clicks.

Task 2

Find an Accelerated Reader book worth 1 point with a reading level range of 3.0 – 5.0.

• Both participants scanned the drop-down menus to find the Accelerated Reader link.  Once the link was located, they navigated to the appropriate information in minimal time.

Task 3

How much is lunch?  

• Student One had several false starts beginning with the Meal Pay option which led to a vendor web site.  Then he tried Lunch Menu link which did not include pricing.  He went back to Meal Pay again before finally trying Cafeteria Policies.
• Student Two went to Meal Pay first after scanning options.  When the information was not immediately obvious on Meal Pay site, this participant scanned the menus again and found the Cafeteria Pricing link where the information was found.

Task 4

Who won the Advance essay contest from Ms. Malone’s homeroom?

• Student One was unable to complete this task.  He began with the Faculty link and attempted to click on Ms. Malone’s name expecting that it was a hyperlink.  This page was merely a list of faculty and when he could not find the needed information, he scanned the menus again, then abandoned the task.
• While scanning the menus, the headline on the home page caught Student Two’s attention.  He immediately navigated to a Click Here link imbedded in the story and clicked on it to find the list of winners of the essay contest.

Task 5

When are early release days?

• Student One looked for a calendar link, but could not find one.  He next began looking for a link back to the home page and could not find that either.  He scanned the other menus again, before abandoning this task.
• Student Two found this information in minimal time and clicks.

Task 6

How much is breakfast?

• Student One went directly to the Cafeteria menu and selected Meal Pay.  When the vendor web site launched, he immediately went back to the Cafeteria menu and selected Cafeteria Policies to find the appropriate information.
• Student Two found this information in minimal time and clicks.

Task 7

What is for lunch on November 16?

• Both participants found this information in minimal time and clicks.

Task 8

Where is the school store?

• Both participants found this information in minimal time and clicks.

School Web Site Usability

Participant Task List Results

Teachers

Task 1

Find a link for creating a Language Arts puzzle sheet.

• Both participants found this information in minimal time and clicks.

Task 2

Locate a web site with suggested Thanksgiving activities.

• Both participants found this information in minimal time and clicks.
• The Thanksgiving link was broken and Teacher One immediately tried to find another link that worked.  The researcher interrupted and asked her to go on to the next task.

Task 3

Where is the school store?

• After scanning the drop-down menus to find the School Store link, both participants found this information in minimal clicks.

Task 4

When is Eat with your Grandparents Day for Kindergarten?

• Teacher One immediately wanted to navigate to the Home page, but could not find a link to do so and used the Back button instead.  On the home page, she immediately located the information in a news story.
• Teacher Two found this information in minimal time and clicks.

Task 5

Place a request to add registration information to the web site.

• Teacher One immediately scrolled to the bottom of the home page and then spent several seconds scanning the screen.  Eventually she located the “email the webmaster” link and clicked.  The link was broken, but Teacher One knew that she had located the appropriate link.
• Teacher Two scanned menus looking for email or registration links but could not find anything that she felt applied to the task.  Teacher Two eventually abandoned this task.

Task 6

What is for lunch on November 16?

• Both participants found this information in minimal time and clicks.

Task 7

Locate the mission and vision for Britt Elementary School.

• Teacher One immediately went to the School menu, but did not find a Mission and Vision link.  After exploring other menus on the home page, she went to the student handbook found on the PTA menu.  After opening the handbook, she located the mission and vision in the table of contents and then scrolled to the appropriate page.
• Teacher Two found this information in minimal time and clicks.

Task 8

How much is lunch? 

• Both participants found this information in minimal time and clicks.

Task 9

Locate contact information for the PTA president.

• Both participants found this information in minimal time and clicks.