Loading...
Please wait, while we are loading the content...
Similar Documents
The PRESTO Project Relay: Open, Asynchronous Learning in Virtual Peer Groups
| Content Provider | Semantic Scholar |
|---|---|
| Author | Daalen, C. Els Van Bots, Pieter W. G. Dopper, Sofia |
| Copyright Year | 2019 |
| Abstract | Academic education entails that students engage in open-ended assignments. Global education entails that students interact in projects with peers world-wide. The logistic challenge of offering academic project work to a large, heterogeneous student body, while keeping instructor workload manageable, is daunting. Project relays on the PRESTO software platform meet this challenge. In a project relay, students work in a virtual peer group on assignments that are organized in successive steps, where each step is peer reviewed. Unlike a regular peer review, the reviewing students revise the work they have reviewed. They then add the next step to it, and pass the improved-and-extended work on to another student. The PRESTO software fully automates the relay workflow, facilitates defining, monitoring, and grading projects, and has been adapted for use in LTI-compliant MOOCs. Since 2013, over 20 project relays have run in several courses at Delft University of Technology. Evaluations show that a project relay realizes the intended learning outcomes, but can at times be stressful for students. This prompts ideas for further pedagogical and technical improvements. Introduction Courses that aim at attaining Bloom’s application level of learning typically require repeated practice from students. For some subjects, exercises can be devised that have a single correct result, but to achieve deep learning, students must work on open-ended assignments (Ke & Xie, 2013). Providing feedback on, and assessing, open-ended assignments (e.g., writing an essay or reporting on a problem analysis) requires interpretation and judgement, which makes courses very teacher-intensive. Reducing teacher load by conducting the work in project groups introduces other problems, such as free rider behaviour (Brooks & Ammons, 2003). Faced with the challenge to provide students with sufficient individual practice and feedback, while at the same time keeping the teaching burden manageable, we have developed the project relay method and its supporting software (PRESTO). The project relay was originally developed for teaching first-year students the basics of constructing quantitative models. Since then, it has been adapted to support any type of project work that can be partitioned into consecutive steps, e.g., writing an essay (literature review → thesis statement → introduction → body of information → conclusion) or conducting a life cycle analysis (goal and scope → inventory analysis → impact assessment → The PRESTO Project Relay: Open, Asynchronous Learning in Virtual Peer Groups Pieter W. G. Bots et al. Connecting through Educational Technology – EDEN Annual Conference Proceedings, 2019, Bruges 451 ISBN 978-615-5511-27-1 interpretation). The higher level skills required for these activities cannot be learned by only studying a textbook, but require learning by doing. The objective of this paper is to explain how the relay system can be used by students and instructors. In order to do this, we first introduce the relay method and present the key features of the PRESTO software. We then briefly report some of our experiences using project relays in several courses, argue that it affords open, asynchronous learning in virtual groups across educational institutions, and outline our plans for future development. The project relay method The assignments students carry out in a relay are divided up into a number of consecutive steps. In all but the first step, students receive – similar to the relay baton in athletics – completed preceding steps from an anonymous predecessor. As each step will concern a different case or topic, students cannot continue their own work on the previous step. Knowing that they have to perform their primary task on this new case gives students a natural incentive to immerse themselves in their predecessor’s work. Having to review and – if necessary – improve their predecessor’s work before extending it with their “own” step makes that students gain a deeper understanding of their predecessor’s work, allowing them to provide more substantive feedback to their predecessor. Moreover, having to “take ownership” of their predecessor’s work makes that students receive additional practice in conducting the previous steps. Figure 1 shows an example of a relay process in which students are asked to develop a quantitative model, given a written description of a system. The assignment is divided into 6 steps that follow on from each other, starting with a research question and ending in conclusions that can be drawn from running the computational model with different input values. In Step 1, students start from scratch, and submit work that is strictly their own. In each following step, students have to perform four tasks: 1. study the work which has been submitted by the predecessor, 2. provide constructive feedback and assess the work of the predecessor on a 5-point scale, 3. improve the work, 4. extend the work by adding their “own” step (primary task). The result of the final step is then reviewed and assessed by two different successors. The PRESTO Project Relay: Open, Asynchronous Learning in Virtual Peer Groups Pieter W. G. Bots et al. 452 Connecting through Educational Technology – EDEN Annual Conference Proceedings, 2019, Bruges ISBN 978-615-5511-27-1 Figure 1. Schematic diagram of students going through the steps of a project relay When starting with the first step, a student is assigned a randomly selected case. The result of the first step then goes to another student who has done Step 1 on another case. This student reviews and improves the work, and adds Step 2 to it. This work then goes to another student who just did Step 2 – also on another case. This student then reviews and improves the work, and adds Step 3. In Figure 1, the dotted arrows show how, in this process, an instance of case A is passed on from student to student. The horizontal lines reflect students performing – asynchronously and at their own pace – consecutive steps on different cases. At the end of the relay, a student has worked on all steps for 6 different cases, and will have iterated several times through earlier steps because giving feedback and improving the work always relates to all previous steps. For a video explanation of the procedure, see https://youtu.be/SGmDstdElk. It does not matter if a student drops out after a step, as this student will then not receive a case for the next step. Moreover, the relay system does not require a fixed schedule of who evaluates which work, since the work will be sent to the next person who is ready for an assessment. This allows students to do the steps of the relay at their own pace, as long as they finish the whole relay before the end date. A downside of the asynchronous process is that students receive feedback on a step relatively late, because successors can still modify their review and assessment until they upload their own work. Their predecessor then gets to see the feedback (typically structured according to a rubric), and is invited to respond and appraise the feedback. In addition to the feedback, students can also view the work submitted by their successor to see how their own work has been modified and extended. To ensure that students put in serious effort, we have introduced some specific incentives. The students know that after every step their work is assessed on a 5-point scale by peers, and – more importantly – that it will count towards their final grade following a differential scoring system. This means that students earn points if they score higher than (or maintain the high The PRESTO Project Relay: Open, Asynchronous Learning in Virtual Peer Groups Pieter W. G. Bots et al. Connecting through Educational Technology – EDEN Annual Conference Proceedings, 2019, Bruges 453 ISBN 978-615-5511-27-1 score of) their predecessor, and lose points if they score lower (or fail to improve on a low score). Although this motivates students to do their work well, it has as a perverse effect that it may lead students to underrate their predecessor’s work. The PRESTO system therefore invites students to appeal if they feel that they have been unfairly assessed. When they appeal, a referee (typically an instructor or a qualified student) will judge the peer assessment, change it if needed, and impose penalty points on students rating too harshly or raising unfounded appeals. Even with these incentives, it may happen that students receive work from a predecessor that is below par and would require a disproportionate amount of time to improve. For these situations, we allow students to reject such work. They then immediately receive the work of a new predecessor. Since the students must justify rejection with a thorough review, to which their predecessor can still appeal, this effectively prohibits rejection merely to get better work to build on. The PRESTO software The PRESTO software manages the workflow of a project relay, and facilitates defining, monitoring, and grading projects. It has been programmed in Python on the open source platforms Django and Semantic UI, and will be available under an open source license. The software discerns four user roles: student, instructor, developer and administrator. Students can enrol in courses, are led through relay steps and invited to respond to reviews for ongoing relays, and can view past relays in their personal history. Instructors can develop and share case sets (for use in different relays), create and operate relays for their courses, monitor progress, and grade students. Developers can design and share templates that define step sequences and specify what a student has to do within every step. Administrators can create courses, and assign instructor and developer roles. Student interface The student interface leads students through the tasks that have to be performed in each of the different steps: commit to “the rules of the game”, read first assi |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | https://repository.tudelft.nl/islandora/object/uuid:4ac9d439-b138-4157-af5d-cc6818773c05/datastream/OBJ/download |
| Alternate Webpage(s) | https://pure.tudelft.nl/portal/files/56932454/The_PRESTO_Project_Relay.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
