This study examines the effectiveness of an online peer review forum (forum). The study allows comparisons to be made between different forum scenarios. The forum was introduced, over a series of semesters, in a first-year accounting screencast assignment: from no forum, to an optional forum, to a compulsory forum. Students indicated that the compulsory forum, underpinned by more structured guidelines for providing feedback, was more beneficial in facilitating improvement in their assignment quality and learning outcomes. We observed improved student performance where a forum was made available. This result is stronger where the use of the forum was optional relative to where it was compulsory. We surmise that this was caused by a higher proportion of exemplar screencasts being posted by more motivated and confident students in the optional forum, creating higher perceived expectations across the total student population. Our findings suggest that more structured feedback through the forum does not necessarily lead to higher performance, even though students value more structured constructive and critical comments as part of their learning experience. These findings highlight the importance of carefully considering forum design and assessment guidelines when embarking on peer review learning initiatives.