This course leverages peer feedback/evaluations in many ways. In particular, we do several rounds of peer evaluations using TEAMMATES.
Submitting peer evaluations is compulsory. If you routinely miss submitting peer evaluations, you can lose participation marks.
Session: Prelim Peer Evaluation
Objective: to give you a chance to give early feedback to team members, but also, a chance to familiarize with the TEAMMATES tool.
Submission is compulsory. Ratings/responses you receive will not be considered for grading .
Although this is done early in the project timeline, we recommend that you give sincere/accurate feedback to others as much as possible.Held early in the semester.
The questions in this session are similar to the
Midterm Peer Evaluation, but has the following additional question:Q How much do you plan to contribute to the team project?
- An equal share
- A bit more than an equal share
- A lot more than an equal share
- A bit less than an equal share (e.g., due to other commitments)
- A lot less than an equal share
- Can only do the bare minimum to pass
- An equal share
Session: Midterm Peer Evaluation
- Held about two weeks into the tP coding phase
Important questions included in the evaluation:
Some of these questions (e.g., contribution to DG) are omitted from the midterm peer evaluation but are in the final peer evaluation (they are given here for your reference)
Q The team members' contribution to the User Guide is,
Q The team members' contribution to the Developer Guide is,
Q The team members' contribution to the team-based tasks is,
Q The team members' contribution to the product implementation (excluding UG, DG, and team-based tasks) is,
Q The team members' conduct in the project and during tutorials,
- Evaluated based on the following criteria, on a scale
Poor/Below Average/Average/Good/Excellent:
Peer Evaluation Criteria: Professional Conduct
- Professional Communication :
- Communicates sufficiently and professionally. e.g. Does not use offensive language or excessive slang in project communications.
- Responds to communication from team members in a timely manner (e.g. within 24 hours).
- Punctuality: Does not cause others to waste time or slow down project progress by frequent tardiness.
- Dependability: Promises what can be done, and delivers what was promised.
- Effort: Puts in sufficient effort to, and tries their best to keep up with the course/project pace. Seeks help from others when necessary.
- Quality: Does not deliver work products that seem to be below the student's competence level i.e. tries their best to make the work product as high quality as possible within her competency level.
- Meticulousness:
- Rarely overlooks submission requirements.
- Rarely misses compulsory course activities such as pre-course survey.
- Teamwork: How willing are you to act as part of a team, contribute to team-level tasks, adhere to team decisions, etc. Honors all collectively agreed-upon commitments e.g., weekly project meetings.
Q The competency of the team member demonstrated in the project and during the tutorials,
- Considered only for bonus marks, A+ grades, and tutor recruitment
- Evaluated based on the following criteria, on a scale
Poor/Below Average/Average/Good/Excellent:
Peer Evaluation Criteria: Competency
- Technical Competency: Able to gain competency in all the required tools and techniques.
- Mentoring skills: Helps others when possible. Able to mentor others well.
- Communication skills: Able to communicate (written and spoken) well. Takes initiative in discussions.
Q [Optional] Any ANONYMOUS feedback you want to give the classmates you reviewed above?
Q [Optional] Any CONFIDENTIAL comments about any team members?
Session: Final Peer Evaluation
- Held at the end of the tP.
- This peer evaluation is compulsory. Not only it will count for weekly participation, those who don't submit will not get a chance to rebut peer evaluations received.
- This session includes all questions from the Midterm Peer Evaluation:
- In addition, it contains these additional questions:
Q Do you agree with the contributions claimed by team members, as stated in their PPP?
You may omit this question for members who have not submitted the PPP.
Q Rank team members based on their ability/potential to lead a software project team (rank 1 is strongest)
Session: Responses to Peer Evaluations
- This is a chance for you to submit your objections to the ratings you received in the Final Peer Evaluation.
How peer evaluations are used
- Peer evaluations are rarely used directly to calculate marks. They are mostly used to flag cases that need further investigation.
- When investigating such cases to decide if and how much the marks should be adjusted based on peer evaluations, we consider factors such as the following:
- Is there a consensus among team members? We do not want an extreme input from one team member to unduly affect the outcome. In many cases, we discard the highest and lowest ratings received before calculating the average.
- Do the other data points (e.g., LoC written, tutor feedback, commit history) corroborates the peer evaluations?
- In some cases, a lower contribution rating given be peers (even if corroborated by other data) might not affect marks at all e.g., if the lower contribution still meets the bar for earning full marks for that component.
Guidelines for giving peer feedback
Giving constructive feedback to others is a valuable skill for software engineers. It is also an intended learning outcome of this course. Half-hearted/trivial feedback will not earn participation marks.
Here are some things to keep in mind:
- Assume you are giving feedback to a colleague, not a friend. Keep the tone of your feedback reasonably professional. Do not use offensive language or slang.
- The feedback should be honest and consistent. Giving positive qualitative feedback (e.g.
Thanks for all the hard work!and negative ratings (e.g.Equal share - 40%) to the same team member is not being honest. - State your expectations early. All too often students give positive/neutral feedback early (hoping that the team member will improve later) and trash the team member in the final evaluation (because the he/she did not improve as expected). However, this could be confusing to the recipient. It is better to give negative feedback early so that the team member gets a clear signal that he/she needs to improve.
Guidelines for interpreting contribution ratings
When you receive results of a peer evaluation question about contribution, use it mainly to compare the team view to your own view.
- Example 1:
Your view (of your own contribution) : E+10% i.e., 10% more than an equal share
Team view (of your own contribution): E+8%
Conclusion: The team's view is quite similar to yours. - Example 2:
Your view (of your own contribution) : E+15% i.e., 10% more than an equal share
Team view (of your own contribution): E+3%
Conclusion: The team's thinks you did significantly less than you claimed you did.