An important part of this course is the term project.
As a member of a design team of around 5 students, you
will apply the methods and skills learned in class
(and hopefully elsewhere) to the design, prototypic
implementation, and evaluation of a realistic application.
The project consists of several parts, which will be graded separately.
Each team has to produce joint deliverables,
which will be the basis for the grades of all team members.
The team members will also be asked for feedback on the
performance of the other team members. This subjective
feedback may be used to adjust individual scores.
Team members are also required to document their
activities, e.g. in the form of work sheets.
Project Organization
The first part of the task will be to select an application,
and to establish the requirements that will serve as the basis for the
later evaluation of the applicatoin.
Then a prototype of your selected application will be implemented.
Note that in many cases, due to time constraints, this
prototype will not be fully functional, but it should
have a "front end" that can be used for evaluation purposes.
In the last part, the prototype will be evaluated.
Since it is obviously difficult to evaluate your own product,
the evaluation will be performed by a different team.
Finally, there will be a presentation on the project.
This presentation will focus on the last part, the evaluation,
and must include information on the initial user requirements,
design, and implementation performed by the original project
team. Both for the evaluation and the presentation it is
very important to provide the evaluators with good documentation.
This will be done with the help of design notes described next.
The following paragraphs provide you with more details
on the different parts of the project. Included is information
on the time frame, and the contribution of the part to the
overall grade. For exact information on the time frame,
please consult the class schedule.
Milestone Week 2: Requirements and Evaluation Criteria
The first part of the project has three main objectives:
Specification of the user requirements.
Definition of the corresponding evaluation criteria.
Determination of dates when the requirements will be
realized in the system.
Your team needs to identify an application to be working on,
and to clearly specify the task the system is supposed to solve.
In many cases, it will be useful to look at existing systems
for the selected or a similar application, and identify
features that are missing or can be improved.
In the ideal case, a summative evaluation should be performed
on a fully developed system that is used in a real-world
application by the actual users. Since this is practically
impossible in a classroom setting, you will have to "switch hats"
from your customary designer or programmer's perspective, and
look at the task to be performed from a user's point of view.
To establish the required functionality as well as usability aspects
is also known as requirements elicitation. For large systems,
these requirements are sometimes translated into a formal system
specification, which serves as blueprint for the actual implementation,
and as the basis for the evaluation and testing of the system.
Again, it is not practical here to perform a full requirements
elicitation or a formal specification, but you need to put
down the crucial functional features of your system together
with important usability aspects. Please note that especially
the usability aspects will be used by the evaluation team to
perform the assessment of your project. By providing concise
and clear definitions of these aspects, you will be able to
focus more on the technical aspects of the prototype implementation,
and it will be much easier for the evaluation team to derive
the evaluation criteria for the assessment of the system.
Implementation of a first prototype (alpha version).
Preparation of a testing and evaluation plan.
After clarifying the requirements for your system,
you will implement prototype versions of it.
The first prototype (alpha) should demonstrate
the most important functions or features of the
system. They should correspond to the requirements
defined earlier.
In addition to the actual prototype of the system,
you will work on a a plan for the testing and
evaluation of the system.
You should describe a collection of usage
scenarios, ideally in combination with some
clearly defined benchmarks that allow a well-founded
evaluation of the system, and its comparison with
other, similar ones. You should show the overall
user interface to potential users as early as
possible, even if the prototype is not finished,
and get some feedback from them.
Since we don't have the time
and resources to do use more formal approaches,
you should show your prototype to some friends,
ask them for feedback, and observe how they use the system.
This will help you to incorporate changes early in the design
stages, when they are relatively cheap to implement.
Make sure to have a record of the user feedback,
either as video, audio tape, or interview notes.
The testing and evaluation plan should be based
on principles and methods from Software Engineering
and Human Computer Interface Design.
Keep in mind that this plan
will be the basis for the evaluation performed
by another team; even if you have a wonderful
prototype, a lousy testing and evaluation plan
is an excellent basis for an unsatisfactory
evaluation ;-)
In order to facilitate collaboration between teams,
all implementations and documents must be accessible
via the Internet, either as Web-based application,
or by making the executable downloadable.
Implementation of additional functionality, as specified
in the requirements.
Elimination of errors and bugs.
Improvements in appearance, user interface, and performance.
Revision of the design based on user feedback and experiences
gathered from the first prototype (alpha version).
Due to time and resource constraints, most likely the alpha version
will be quite restricted in functionality and appearance.
The beta version should add additional functionality in
accordance with the requirements specified in the beginning.
In addition, there will probably be some usability and performance
aspects that can be improved.
Some of these improvements may be triggered
by your own insights and experiences, while others may come
from the feedback you receive from the users who took your
system for a test drive.
Implementation of the full functionality of the system,
according to the requirements and evaluation criteria.
Elimination of all major errors and bugs.
Minor modifications concerning appearance, user interface,
and performance.
The final version should implement the full functionality of the
system as initially specified in the requirements.
The system should be free of major errors.
Additional modifications can be made to increase performance and
usability, but the overall design of the system should be stable.
For this part, each team will evaluate another team's project.
The evaluation team will perform the tests and evaluations
described in evaluation plan provided by the development team.
A short evaluation will be performed after each milestone,
and should be completed the week after the milestone is due.
These evaluations should be available from the Web pages of
the evaluation team, with links from the development team pages.
They should address at least the following aspects:
overview of the system or the major changes,
as seen from the evaluation team's perspective
identification of the requirements relevant for this version
application of the evaluation criteria to this version
description and interpretation of the results
recommendations to the development team
You can add additional information, such as feedback on the
testing and evaluation plan, on the documentation provided,
or on the system design and implementation.
This evaluation is done on various prototypes of the system so that
changes are still relatively easy to incorporate into the design.
Due to the tight schedule, it is advisable to establish good
contacts between the development and evaluation teams.
Ideally, the evaluation team should receive ongoing information
on the status of the project, and report back any evaluation
results as quickly as possible.
The overall evaluation will be presented to class in a final presentation,
and should be documented in a final evaluation report.
The final presentations for the project will be given as a joint
presentation by the development and the evaluation team.
The development team will present the task, purpose, design,
and implementation of the system, and the evaluation team
will report on the evaluation of the system. The overall presentation
must not be longer than 20 minutes, plus 5 minutes for discussion.
The development team has 15 minutes for its presentation,
and the evaluation team 5 minutes.
Presentations that go over their time limit may be
interrupted, get a lower grade, or both.
A quick demo of the main functionality of the system should
be integrated into the development team's part.
The discussion time can be used for questions and feedback from
the audience, or for the clarification of misunderstandings
or controversies between the development and the evaluation team.
Since it can be difficult for me to judge the contributions
of individual team members to the overall effort, I am asking
for your feedback on the performance of your team mates.
Follow this link to a template Mutual Team Member Evaluation Sheet
as an Excel file.
I reserve the right to change the allocation of points
for extreme cases, however (e.g. if everybody
agrees to give all the other team members the full 20 points).
I may also ask you for further documentation to support
your contribution to the team, or your evaluation of a
team member's contribution.
The overall evaluation will be presented to class in a final presentation,
and should be documented in a final evaluation report.
Grading
Project Grading Scheme
Milestone Week 2
10
Milestone Week 4
10
Milestone Week 6
10
Milestone Week 8
10
Team Evaluations
20
Presentations
20
Mutual Team Member Evaluation
20
The overall score for the project is 100 points.
80 out of these 100 points come out
of my evaluation of the team project, and usually every
team member gets the same score. Up to 20 points come
from an evaluation of your team mates, calculated as
the average of all your team mates' scores for your work.
A Few Notes on Teamwork
A substantial degree of your grade in this class depends on
the overall performance of your team. This can be good
(you do nothing, and still get a good grade) or bad
(you do all the work, but it is not enough). Ideally,
every team member should contribute a roughly equal share.
In reality, this is not the case because team members have
different backgrounds, experience, work habits, cultures,
etc. Just like in a professional work environment, you
have to find a balance between looking after your own interest,
and contributing to the overall team effort.
If at any point you feel that there are serious problems
with your project team, feel free to talk to me, and we
will try to find a solution.