{"coupon":"","discount":"","doc_name":"Psychology manuscript on peer rating","doc_id":68551,"editorId":"4948","email":"violette_c@yahoo.com","message":"<h1><strong>Title<\/strong><\/h1>\r\n<p>A Modification to Peer Rating System to Recognize Rating Skill as A Learning Outcome<\/p>\r\n<h1><strong>Abstract<\/strong><\/h1>\r\n<p>Peer raters vary in their ability to judge their classmates&rsquo; performance on an assignment and yet their opinions are given equal weight by the existing peer rating system. We recognize rating skills as a quantitative training requirement in selected disciplines and regard unchecked rater performance a missed learning opportunity. A modification to the peer rating system is therefore needed to discern the quality of contribution made by each rater to provide individualized feedback to improve their rating skills. A total of 69 students in five advanced research classes followed the procedure of the existing peer rating system by completing reading assignments, writing reflections online, engaging in class discussions, and rating their peers&rsquo; reflections using a grading rubric. Unique to our modified system, peer ratings were compared with each other and also with the instructor ratings to derive individualized indices of reliability and validity. These technical indicators enabled tailored feedback to each rater mid-semester and the result was an improvement in the quality of peer ratings at the end of the semester, at a significance level of .051 according to an omnibus test across the five classes. Instructors can rely on numerical analysis to hold raters responsible and teach rating skills. The students likewise get to acquire a new skillset of coding open-ended essays which may be transferable to job settings.<\/p>\r\n<p>&nbsp;<\/p>\r\n<h1><strong>Introduction<\/strong><\/h1>\r\n<p>Peer assessments are becoming a necessity in an education environment that is characterized by fiscal constraints and large student\/teacher ratios (Spatar, Penna, Mills, Kugija &amp; Cooke, 2015). Peer assessment is therefore seen as a pragmatic tool &ndash; it saves time for tutors and instructors (Sadler &amp; Good, 2006) and holds the promise of substituting instructor-generated scores with weighed peer-generated scores (Nepal, 2012; Spatar et al., 2015). Pragmatism, particularly one that serves the instructor, does little to incentivize students to participate in rating exercises (Loddington et al., 2009; Neus, 2011). Students prefer to devote their energy and effort to educational activities directly associated with assessment and expect their rating efforts count towards a part of their grades (Reader, 2016). However, instructors rarely assess the quality of peer ratings. They would run a simple tally of the number of times students missed rating assignments, which gauges quantity rather than quality. Alternatively, they would evaluate the overall contribution by a group of raters, which targets group responsibility rather than individual accountability (Johnston &amp; Miles, 2004; Zhang &amp; Ohland, 2009).<\/p>\r\n<p>The gravitation towards pragmatism has obscured the original curriculum objectives of peer assessment. A renewed emphasis on student learning may be a powerful incentive for students to engage in peer rating exercises. The origin of student involvement in assessment started with self-assessment (Falchikov &amp; Bond, 1989). The motivation to introduce self-assessment in a course ranged from a basic curiosity of whether students could assess their own work to reasons explicitly related to education objectives. Self-assessment could prepare students for life-long learning when graduates have to assess their performance and progress without the help of an instructor. A spectrum of discipline areas has since further empowered students in the assessment process who would judge not only their own work but also the work of their peers. In a meta-analysis by Falchikov and Goldfinch (2000), the stated objective of peer assessment was to promote active learning and collaborative learning (Piaget, 1971; Vygotsky, 1962). Collaborative learning is touted as team work with the real-world benefits of communication, problem-solving, leadership and self-management (Johnston &amp; Miles, 2004; Rafiq &amp; Fullerton, 1996; Spatar et al., 2015).<\/p>\r\n<p>The present paper moves beyond the benefits on collaborative learning and teamwork, and report on an unexamined curriculum benefit in selected disciplines. For the social sciences and business, quantitative literacy and data processing skills are curricular goals because graduates are expected to know how to distill systematic information from open-ended interviews, field observations notes, or customer satisfaction surveys (Aiken, West, &amp; Millsap, 2009; Bandalos and Kopp, 2012; Lejeune, 2001). Each discipline may have its own version of learning outcome but it resembles the goals outlined by the American Psychological Association (APA, 2013) &ndash; students ought to &ldquo;collect, analyze, interpret, and report data&rdquo; (sec. 2.4 A) and &ldquo;design and adopt high quality measurement strategies that enhance reliability and validity&rdquo; (sec. 2.4 E). Peer rating exercises provide hands-on learning opportunities for students to gain a skillset to process data. Students learn to operationalize evaluative criteria and render judgments in a valid and reliable manner (Hooper &amp; Cowell, 2014).<\/p>\r\n<p>If rating skills have an explicit place in the curriculum, its training should be intentional and its outcome should be assessed (James, 2014). Therefore, we modify the existing system to provide a numerical tool to assess the quality of ratings from each rater. Individual accountability is an essential feature in our modified system and it serves the curriculum goal of evaluating and improving rating skills. We show that as few as two rounds of analyses in a semester would make rating skills an intentional pursuit among students.<\/p>\r\n<p>&nbsp;<\/p>","price_select":"2","temp_doc_name":"upload_578fe14ce059d9.14424820.docx","word_count":"799","subtotal":15.98,"total_amt":15.98,"firstName":"Violet","lastName":"Cheung","lastfour":"7158","exp":"0418","address":"5220 Anza Street","city":"San Francisco","zip":"94121","state":"CA","country":"United States","transactionReference":"8537904276"}