CSE 432S/533S: Project Evaluation I and Midterm Project Report

This page is intended to help each project team define both its evaluation approach for the first project cycle, and the format of its Midterm Project Report. The evaluation approach defined in this stage will help set the stage for defining the requirements for the second project cycle, and the project report will serve as a record of the first project cycle, both for grading and for the teams to review throughout the second project cycle.

Guidelines for Evaluation of your First Project Cycle

Each team should evaluate their project in a way that both (1) is similar to the "happy path" demo for the implementation stage in that it exercises the major subsystems of the project implementation, and (2) goes well beyond the "happy path" to explore (and potentially reveal design and implementation problems with) other scenarios. One of the key ways to think about your evaluation is to ask "what are the major ways our system could behave (both good an bad), and what scenarios could we define to try to drive the system into those behaviors".

For the good behaviors, your scenarios would be similar to the "happy path" scenario in demonstrating what the system can do, but would aim to demonstrate a broader range of the system's potential operation. For the potential bad behaviors, your scenarios would be designed more to test (and if the potential bad behavior actually appeared would allow you to fix) how the system responds to things like bad user inputs, different orders of actions from the user, etc.

In general, the goal in this stage is to demonstrate a system that is demonstrably more robust in avoiding or gracefully handling problems (and in achieving that is potentially somewhat more refined in its implementation) than in the previous development stage. Although time constraints may still mean that you won't be able to find and fix all remaining bugs in your system, it is important to try to do that to the extent possible, and document open problems as well as solved problems and their solutions.

Deliverable: Midterm Project Report

Using Word, LaTeX, or other editing tools as you prefer, you will produce a carefully edited document that summarizes the first cycle of your team's project, including your evaluation for the current development stage. In developing this report you may freely re-use materials from your team's previous deliverables, but in any case you should please make a thorough pass through each section to make sure that the discussion and details in each one reflects your understanding of the project as of the end of the evaluation stage (simple cut and paste won't cut it here, so to speak ;-). Your project report should contain the following sections:

  1. a cover page with the title of the project, the nature of the document (Midterm Project Report), the names of the team members who contributed materially to the document, and the date the document was submitted.

  2. a project overview section that describes what your project has achieved as of the end of the first evaluation stage. This should be an English narrative that summarizes what the system does, the scenarios that you have used to demonstrate and test its behavior, and high level observations and lessons learned during the first cycle of your project.

  3. a requirements and design forces section that gives a reasonably detailed description of the project requirements that have survived to the end of the evaluation stage of the first project cycle. You should comment on which of these were identified in the initial requirements stage of your project, and which of them were added after that initial requirements stage. If any requirements were added after the initial requirements stage, you should please comment on how you discovered the requirement, and your motivation for adding it. Finally, if any requirements that had been listed in your initial requirements stage were dropped from your project or deferred to the second project spiral, please indicate your reasoning for why the requirement was dropped or deferred and give some justification for why that was an appropriate decision.

    For each of the requirements that has survived to the end of the evaluation stage of the first project cycle, please describe the design forces that each requirements imposes on the design.

  4. a design evolution section that gives a narrative explanation of how you mapped the design forces from your requirements into your high level and low level design decisions. Please comment especially on which design patterns you applied in order to resolve particular design forces from your requirements or from previously applied patterns.

  5. a technical design section consisting of (1) class structure diagrams, interaction diagrams, and/or collaboration diagrams that show the structural and behavioral relationships in your design; (2) an English description of what those diagrams show, including which roles are played by the different classes in the design patterns you applied (as described in your previous section).

  6. a project code file listing section that lists the contents of the code files containing the declarations and definitions of the classes, functions, member functions, and member variables in your project. The comments in the listing should reflect your design, implementation, and evaluation decisions in detail.

  7. an evaluation scenarios section that describes each of the scenarios you used to evaluate your project (including the "happy path" scenario you defined for the implementation stage of your project). For each scenario, please explain what it is designed to test, the steps taken in the scenario, and the results you got when running the scenario. If any problems were encountered while evaluating your design, please describe them as well, along with saying what you did (or would do if they're still open problems) to fix them.

  8. an instructions for building and running section, that gives step by step instructions for building and running each of the scenarios on a particular platform. If there are any libraries, etc. that need to be downloaded in order to build or run your prototype, you should also say what they are and give step by step instructions for downloading them and using them in your project. One option is to set up a demo environment in your team account on the CEC machines, and give us directions (and access) for running your evaluation there.

Participation: every member of your team should be significantly involved in the evaluation of your project and the writing of the midterm project report, though how you structure the interactions within the team is up to you.

Deadline: by 11:59pm on Wednesday June 17, 2009, please e-mail your team's midterm project report to Prof. Gill at cdgill@cse.wustl.edu.