Syllabus Collection >
Please use this identifier to cite or link to this item:
|Title: ||Usability Engineering|
|Issue Date: |
|Abstract: ||CS 5714- Usability Engineering - Dr. Hartson - Spring 2004
Home | Syllabus | Calendar and Class Notes | Projects | Homework | Participation | Grades
Usability has become an essential requirement of interactive software. In response, human-computer interaction (HCI) has grown into a serious field of research, development, and application. To meet the specific needs of practitioners for methods and tools to develop user interaction designs with provably high levels of usability, Usability Engineering (UE) has emerged as a sub-discipline within HCI. Despite this attention to usability on some levels, much interactive software is still too difficult to useï¿½ as any computer user knows. It is turning out to be more difficult to design for high usability than many first imagined. A major cause underlying poor usability is a lack of understanding of a usability engineering process for developing usable interaction designs. Although many software development teams now have roles for a usability engineer or usability specialist, software developers often still have primary responsibility for developing interactive systems. Most software engineers are not trained in usability methods and, therefore, do not have the knowledge and skills to include usability methods in their life cycle activities. Many software developers believe that usability engineering is merely usability testing, done near the end of the development process. This course, which has been designed to focus on usability methods and a usability engineering development process, uses an integrative and cross-disciplinary approach to bring together a broad variety of topics together in relation to the problem of developing quality user interaction designs.
Among the topics studied are the design and evaluation of effective user interaction designs, including principles and guidelines for designing the product: effective interaction design content and interaction style. Additionally, much emphasis is given to the development process for user interaction designs as an integral, but different, part of interactive software development. The development process includes management of a life cycle that must be both iterative and cost effective. User interaction development activities include requirements and task analysis, usability specifications, design, prototyping, and evaluation. It is a goal of this course to help students realize that usability engineering is an ongoing process throughout the full product life cycle, and developing the human-computer interface is not something to be done at the last minute, when the "rest of the system" is finished.
This is an active learning course, meaning that you, the student, have more responsibility than usual in the learning process. A major student responsibility is to read and study the class notes applicable to each class before that class meets. It is the student's job to manage the pace of this reading so that it fits the class schedule. No fixed reading assignments will be announced. Class time will be used for highlighting course materials, questions, and discussion. The main use of class time will be for in-class activities to demonstrate techniques and principles and to practice the skills described in the class notes. Outside of the classroom, students will acquire hands-on experience in applying these skills and techniques in a semester-long team project. In this project, students will develop a usable interaction design for their own application system in a real development project for a real client selected from the local community.
SUBJECT MATTER OBJECTIVES
The outcome-oriented objectives for the course are that each student, upon successful completion, will be able to:
* participate as an HCI- or user interface-oriented development team member to represent usability issues within an interactive software system development project, and
* perform individual guidelines-based critical analysis and evaluation of a user interaction design.
These translate into the following specific instructional objective, that each student will be able to:
* apply human factors principles and interaction design guidelines in the design and critical evaluation of interaction designs,
* relate some basic cognitive characteristics of users to interaction design,
* apply an iterative evaluation-centered life cycle for user interaction development within a broader traditional software/ system development process,
* formulate quantitative, measurable usability specifications for user interaction designs,
* construct paper and computer-based rapid prototypes,
* conduct a formal process of formative usability evaluation using appropriate users as subjects/participants,
* analyze quantitative and qualitative usability evaluation data, and
* perform cost/importance and other analyses for management of the iterative life cycle.
MY PERSONAL GOALS FOR YOU IN THE COURSE
In addition to the content-specific objectives listed above, I have these personal goals for each student:
* to get you to think deeply and carefully about the subject,
* to help you to genuinely like the subject, and
* to engender a deeper interest (perhaps in some of you) that can be pursued beyond this course.
Graduate standing in any major (despite what you might read elsewhere, e.g., the university catalog) is the nominal prerequisite for CS5714. Some qualified undergraduate seniors may also be admitted by permission of the instructor. No official coursework is required as a prerequisite, but students should have substantial experience with computers, especially their interactive use, and an intense interest in making them easier to use. Some knowledge of software engineering fundamentals, gained either from coursework or from practice, would be very useful. We expect most students in this course to be from Computer Science and Human Factors Engineering (in ISE). We do welcome the diversity of others (e.g., art and design, psychology, communications studies, technical writing, music, etc.), but they must be interested and willing to devote some extra effort in learning the technical aspects of development processes. If concepts such as interaction design evaluation, usability, prototyping, life cycles, and development methodology mean nothing to you, this may not be the class for you.
WARNING: At the other end of the spectrum, occasionally we get students with considerable experience in HCI and usability. You are still welcome to participate in this course, but be warned that this is not an advanced course in usability engineering. Although this course gives thorough treatment to the usability engineering process, this is an introductory course.
D. Hix and H. R. Hartson, Developing User Interfaces: Ensuring Usability Through Product and Process, John Wiley, 1993. This book is optional. It is getting a bit old, but much of its content is still current. Quite a bit of the material for this course still comes from the book and it makes a good reference book in usability engineering. The book is not required to be successful in this course. The class notes are very complete and are definitely up to date. You decide.
ADDITIONAL STUDY MATERIALS
Class notes are posted on the course website in the forms of download links ( in the course calendar) to PowerPoint files, the same PowerPoint presentations I will use in class. This is the major source of content and discussion for the course.
You are required to print these slides and bring them to every class in a notebook, so we can refer to them and you can use them for note-taking in class. Most of what we do in class and in your team projects will be based on these notes.
User Interaction Development Project*
Project 1: Topic, Client, and Product Concept Statement
Project 2: Systems Analysis
Project 3: Scenarios, Screen Design, and Early Prototype
Project 4: Usability Specifications, Hi-Fi Prototype, and Demo
Project 5: Formative Usability Evaluation
Project 6: Design Iteration
Project 7: Oral Project Presentation
*Individual project grades will be weighted by amount of participation in team effort (see Team Member Evaluations below).
Occasional homework will be assigned as appropriate.ï¿½
There are several kinds of participation activities, including in-class activities, research project participation, and contribution to the class collection of usability problem descriptions and analyses. To achieve a perfect grade in the Participation category (see Grading above), you are expected to complete both in-class and research-oriented participation credit (PC) requirements. Contribution to the class collection of usability problem samples is a backup possibility to research-oriented participation, in case you are not able to find a research project that needs a participant.
Getting full in-class participation credit is easy: Just be willing to participate in an in-class activity and do a good job. As a normal part of active learning in the classroom, individuals and small groups will be called upon from the class roll to come to the front of the class and go through part of an ongoing analysis or design exercise in class, to illustrate the application of concepts covered in the class notes and class discussion. In assessing the "do a good job" part of this activity, we'll be looking for:
* Presence or absence of the individual
* Preparedness, knowledge of material
* Care and correctness in applying it
* Intangibles (getting into role, etc.)
Research project participation
To expose you to the experience of being part of an HCI research project/study, this component of the participation grade can be fulfilled by serving as a participant/subject (or other needed role) in one or more active HCI research projects on campus (or other location), for a total of four hours of research project participation. We have posted descriptions of some research project participation opportunities, along with the contact information for volunteering to participate. Your participation does not have to be as a study subject; it can be in almost any role that exposes you to how such projects work. Other research projects are also an option, subject to our approval. After participating in one of these projects, fill out this participation certification form, have the person in charge of the project sign it, and hand it in before the last day of class.
Alternative participation option
As a class, we will be accumulating a collection of usability problem cases. If you cannot locate a research project needing a participant, you may fulfill your research participation component of the participation credit requirement by constructing and submitting several usability problem cases (problem descriptions and analyses), using our Problem Reporting Tool. See the instructor or GTA for further information about this option.
The major work component for the course is the semester team-oriented development project. It involves defining, analyzing, specifying, designing, prototyping, evaluating, and iterating the interaction design for the user interface of an interactive system for a real-world client of your choice. The purpose of the project is to give you real-world exposure to all steps involved in developing a significant user interaction design. It is not always necessary to develop a complete interaction design to learn the usability engineering process but it is important to follow all the steps in the process for at least part of an interaction design.
This is a team project. I will assign students to teams, trying to balance knowledge, skills, and backgrounds. All development activities, including writing the deliverables, are team activities. All team members are to participate in all development activities. Do not divide the overall process among the team members. Even though this might seem like a more efficient way to proceed, this leads to a kind of specialization that poses a barrier to each person learning the overall process. The is especially true for a person who gets the job of programming, at the price of not learning usability engineering skills.
The grading process
The process. It is the professor's responsibility to establish grading standards and work with the GTA in grading. The GTA has the primary responsibility for grading homework. The GTA and the instructor work together in grading project reports, with the bulk of the grading being done by the GTA. The process begins with the GTA carefully reading each report and writing comments on it. The GTA and instructor meet and have a detailed discussion of each report.
The objective part. The first thing we assess objectively is whether all requirements are met. Mechanical aspects such as formatting, labeling, grammar, spelling, following instructions, etc. are easy to grade because they are objective. Since these mechanical aspects are just expected, we don't give positive points for those, but we do deduct points if they are wrong or missing.
The subjective part. The hard part in grading is the subjective part, which is about quality of content. We lay the reports out on a table in approximate order of overall quality. During a discussion of their relative merits, a given report may be moved up or down in the sorting as we calibrate our judgments and make adjustments. There are two components to this subjective evaluation: how well requirements are met (how well you did the job) and how well you reported it. Our evaluation of these components is based on our own knowledge and experience and is necessarily somewhat relative among the project teams of the class. The "how well you met requirements" part is based on our perception of how much you put into it, how completely you pursued the assignment, and how well you understood, interpreted, and applied the material covered in class to your project.
We try to write comments on your reports about these qualitative parts, so you know what aspects of your work and writing are possible issues. Comments of this type are along the lines of: why didn't you also do such-and-such, this isn't really what we are asking for here, wording not tight, not quite a complete as it could be, a little bit rambling, grammar and spelling not the best, etc.
An example of what we mean. Here is an example from our perspective, so you can relate to what we are faced with in this part of the grading. Team A reports their findings of a client interview in a crisp, easy to digest form using summary tables and short clear descriptive paragraphs. You report your findings in a somewhat lengthy and verbose, narrative form. You cover all the important points but it is not as easy for the reader to pick them out as it is in Team A's report.
So we write on Team A's report: Nice job of clearly presenting the important points. On yours, we write: Slightly verbose; need to clarify/highlight important points. Then you come to see us and ask: Where is it verbose? We say that verbosity is a distributed thing but, as one example, here are two sentences that you could have tightened into one short one. It's your job to extrapolate that comment to the rest of the report.
The point here is that we cannot always say exactly where a specific number of points was deducted to arrive at your grade. In fact, it is usually the case that points are not deducted but your score is assigned to reflect your relative standing among all the reports, as explained in the next paragraph.
Grades are based on overall quality. Please remember that, beyond any comments we write on your report, the grades are based on *overall* perceived and relative quality. So, let's say your team got an 85 and Team X got 90 and the only apparent (noted in comments) difference between you and Team X was that you got a comment about completeness. That does NOT mean that the completeness issue was necessarily worth 5 points. The completeness comment contributes to where your report finally falls on the table, but its relative position in that sorting is what determines the numeric grade. So *that* is the answer when you come back to us and ask why you got the grade you got.
You may also ask what you can do to get a better grade. Often there is not a simple quick answer, like 'do such and such'. We are willing, however, to sit down with you and go through your report in detail and point out some aspects of the writing that could be improved. And maybe we might be able to make a general conclusion about a particular aspect of your writing to focus on.
Bottom line. If you get a grade of, for example, 80 or 85 and another team gets, say, 90 or 95, it does not mean that you did a poor job. We have to assign some number as a grade and we use it to distinguish among the project reports from all the teams. We evaluate your assignments in detail. You can be sure this is done in the fairest way possible.
Keep this in mind, especially if you get discouraged about the project workload: Our experience in teaching this course has been that the project work separates the class into two groups: those who simply cannot keep up and who drop the course before the end and those who hang in there and do a good job on the project. Students who stay in the class, stand up under the heavy workload, and carry out the team project with dedication and enthusiasm will do well in the course in the end.
Grading appeals. All assignments (homework, project, and final exam) are designed by the instructor. The homeworks and projects are graded by the GTA, with consultation by the instructor. If you have any questions about the grading of these items or feel that something has been graded incorrectly or unfairly, you should contact the GTA first. If you do not get satisfaction from the GTA, you can appeal to the instructor by requesting a regrading. Regrading requests must be submitted to the instructor within three class meetings after the graded work was returned to the class. A regrade will entail a COMPLETE RE-EVALUATION OF THE ENTIRE ASSIGNMENT. This may increase or decrease your original score. The regraded score is final.
This risk of a grade reduction is not to discourage you from discussing your project grade with the GTA or instructor. This is to address the problem of students who mainly want to squeeze out more points in their scores. You need not worry about this if your desire to discuss project grading is motivated toward learning and not all about getting points.
Keep all graded work until the end of the semester. You should check the posting of your grades periodically to be sure our records reflect your correct grades. In case your grade is incorrectly recorded, you will need to bring in the graded original in order for the recorded grade to be changed.
Team member evaluations
The parts of the project assignments are described separately in the course Website, under "Projects". Each member of the team is expected to contribute equally to each part of the project. It is possible that the most difficult part of the project assignments is working well together in a group. Be aware of possible group problems and be ready to solve them. Don't make the mistake of taking this aspect for granted.
Sometimes, despite our best efforts, some team members end up not pulling their fair share of the weight. To ensure that each team member is given a project grade reflecting individual contributions, the final project assignment is a Team Member Evaluation. Each team must INDIVIDUALLY turn in a paper copy of the Team Member Evaluation Form (print from Web) as a required deliverable to report the relative effort/contribution of each person on your project team over the whole project (including yourself).
This form is not optional. Be professional and give a careful rating. The ratings on these forms will be used as weightings, as explained at the beginning of the semester, to convert team project grades into individual student project grades. The team is given a grade for each part of the project. Each individual team member's grade for each project assignment is a weighting of the team grade, where the weighting is based on an evaluation of individual contributions, collected from each team member at the end of the semester (and moderated as necessary by the instructor).
You are responsible for keeping up with the reading of class notes per the schedule given in the course calendar. You are also responsible for knowing where we are in our class discussions, with respect to finding your place in the class notes, and setting your own reading pace to keep ahead enough to be prepared for class discussions and activities. There will also be a few academic papers assigned as reading associated with various class topics.
Skipping through material in class
Sometimes students say "Please don't flash through the slides in class." I need to explain that it is sometimes necessary for me to skim over some less important material in order to highlight the most important things. You should be able to keep up with this kind of occasional skimming because:
1. You are supposed to have just read the notes for what we're covering at any point in class. If you have questions about parts I don't cover in class, you are welcome to ask.
2. Class time is for review, highlights, examples, and explanations. It's not the traditional lecture format and I think the examples, etc. not in the class notes are more important than my total coverage of what you can read for yourself.
3. You should have the Class Notes with you in class, so you can follow along easily without getting lost, even though we might skim over some material. You can even mark what we skip in class, to go over it yourself at home.
Due dates and times for handing in homework and project assignments
All homework and project assignments must be turned in at the beginning of class on the due date. You should think of all due dates for assignments as firm. The tight schedule of deliverables throughout the whole semester makes it nearly impossible to slip or extend due dates. Any assignment that you do not hand in on time will be penalized in grading. If you are not able to complete an assignment by the due date, it would be best for you to hand in as much of it as you have done. You must prepare your assignments using a word processor and hand in a hardcopy by the due date/time. Assignments may not be submitted via email to either the professor or a GTA (some exceptions may be negotiated in advance).
Attendance is not required for the course, nor does it play a direct part in the course grade. Beyond the occasional need to be absent from class for a good reason, please consider that much of the learning for the course occurs in class. You cannot participate in this learning if you are not present.
If you have to miss class for an extended period due to a protracted illness or similar reason, we will treat your needs as a special case and I will do everything I can to help you survive.
No extra credit work
Students sometimes ask for some extra credit work near the end of the semester in an attempt to bring up sagging grades. However, university policy does not allow extra credit work to be given to any student on an individual basis.
If the university is closed (for any reason) on an assignment due date, the assignment will be due (in a box or pile outside Dr. Hartson's door) by 2 p.m. on the first day the university reopens.
Responding to e-mail
The professor and GTA will make every effort to answer your email in a timely fashion. However, you should not necessarily expect to get a reply in less than 24 hours or over a weekend. Many times you may get a reply in less than 24 hours, but you should not count on it (e.g., to answer questions about a homework or project assignment within the last few hours before that assignment is due). Please put "CS5714" as the subject line of your email; that will help us identify your emails more quickly.
Grades via e-mail
Neither the professor nor the GTA will be able to reply to individual email requests for final exam and/or class grades at the end of the semester, but grades will be posted on the Blackboard system by the end of the day of the final exam.
The Virginia Tech honor code is in effect for all work, whether performed individually or in teams. All assignments submitted shall be considered graded work unless otherwise noted. All aspects of your coursework are covered by the honor system. Any suspected violations of the honor code will be promptly reported to the honor system. Honesty in your academic work will develop into professional integrity. The faculty and students of Virginia Tech will not tolerate any form of academic dishonesty.
We wish to make any accommodations needed by any student because of a disability. Please contact the instructor during the first week of class.|
|Appears in Collections:||Syllabus|
Files in This Item:
All items in DSpace are protected by copyright, with all rights reserved.