Loading...
Please wait, while we are loading the content...
Similar Documents
Extra sensory perception - extending the power of QM Perception
| Content Provider | Semantic Scholar |
|---|---|
| Author | McCabe, Michael Daugherty, Brian |
| Copyright Year | 1999 |
| Abstract | Initial experience of using Question Mark Perception for Web-based assessment was presented at the 2 Annual CAA Conference (McCabe, 1998). At that time Perception had been released as a commercial product for barely a month and had been used for formative assessment only. Confidence gained during the 1997/8 academic year has led to the extension of its use for summative assessment during 1998/9. Formal exams within the Faculty of Technology have been successfully conducted in mathematics, astronomy and computer science using Perception, in addition to the routine use of Question Mark Designer. Besides the obvious benefits of greater accessibility to Perception tests, it has been possible to develop variants of existing question types, e.g. multi-numeric, and to exploit the more open system with custom questions. Perception Version 2, scheduled for release in July 1999, includes many new features and improvements. Introduction Both Question Mark Designer and Question Mark Perception tests are used routinely in the School of Computer Science and Mathematics. QM Designer has been used increasingly for both formative and summative assessment over the past five years. A typical first year unit now requires a student to take five computer marked tests at regular intervals during a semester. Although Designer allows tests to be developed rapidly with a variety of standard question types, it is relatively inflexible in this respect Perception (McCabe, 1998) may be regarded as the on-line equivalent of QM Designer and QM Guardian, incorporating facilities for administration, security and scheduling as well as authoring, delivery and reporting. It was initially used for formative assessment and has been used for summative assessment during the past 1998/9 academic year. The obvious benefit of using Perception is its greater accessibility, with straightforward access to tests via a Web browser. Less obvious is its greater flexibility. Perception allows variants of existing question types to be authored relatively easily and enables scores for partially correct answers (Beevers,1999) to be awarded more reliably. Furthermore it has become possible to incorporate new customised question types within tests. Since many subject areas have specialist requirements, the ability to add to standard question types is of great benefit. Culwin (1998) and Whittingham (1998) both provide good examples of new question types. Conversion from Designer to Perception The ability to import tests into Perception either from plain text files or existing Designer (QDL) libraries has enabled tests to be set up quickly. Figure 1 shows an assessment Web page, which accesses a large databank of 3000 multiple choice questions converted from text files. Both open self-assessment tests and secure coursework tests are delivered from the same page. The storage of questions within hierarchical topics, cf. Windows folders, allows different sets of questions to be selected from the same databank easily. Test reporting is also linked directly to the page, allowing rapid checking of results by both staff and students (Figure 2) Figure 1 Large-Scale Question Bank Conversion and Delivery Figure 2 On-line Test Results Figure 3 Conversion of Designer Tests (left) to Perception (right) In the PPARC Astrowise Project for the public understanding of science, a bank of 200 questions on astronomical topics, including the 1999 total solar eclipse, was developed using QM Designer. Conversion of the question bank using Perception enabled the same tests to be delivered both locally and on-line (Figure 3). Other tests have been submitted electronically in plain text, QML text or Designer library format, converted and made available for on-line delivery. Hence lecturers without access to Perception Question Manager can still develop their own tests, even though the use of the QM wizards and dialogues is probably the quickest route for authoring new questions. Once published to the server, completed tests can be embedded seamlessly within other on-line courseware material. Numerical Questions Two commonly cited limitations of Designer numeric questions (e.g. Lawson, 1998) are its inabilities a.) to handle more than one numeric answer in a single question and b.) to use randomly generated numeric parameters in questions and answers (e.g. Thelwell 1998, 1999). Figure 4 shows how questions with more than one numerical answer can be developed. By using question outcomes, each answer can be scored independently or specific answer combinations can be scored separately. By including superscripts, the mantissa and exponent in scientific notation, e.g. 3.3 x 10 , can be entered and marked independently. Subscripts can be used for chemical formulae (cf. Whittingham 1998). Figure 4 Multiple Numeric Answers A calculator, spreadsheet or more sophisticated software application may also be used to support the answering of numeric questions. In Figure 5 the Maple computer algebra system is being used to plot a graph in order to find the correct answer. Figure 5 Numeric Question with Software Support Perception wizards and component dialogue boxes provide a user-friendly means of authoring rigidly defined questions. It is harder to include the automatic generation of random numerical parameters, where the values need to be generated differently each time the test is run. Several different approaches have been tried using Javascript and Java within standard Perception questions. Javascript Randomised MCQ The original approach was to insert Javascript into the Question Mark-up Language (QML) used by Perception. The only question types, which are readily randomised in this way, are the multiple-choice and multiple-response type questions. The radio button (or check boxes) associated with the correct answer(s) are fixed, but they can be shuffled when presented on screen. By using Javascript within Perception, random numbers are generated for the question stem, the correct answer(s) is calculated and suitable random distracters generated. A similar approach could be used within Perception to select a sample from many different text distracters. Unfortunately, it was found that test feedback did not function exactly as desired, since randomisation caused a question to be displayed with different numerical parameters from the one answered. The awarding of marks is nevertheless consistent with the original question, so a fixed feedback message has to be used instead. Furthermore, there is a potential security problem, because an intelligent student could ascertain the correct answer purely by viewing the source code! It was impossible use Javascript to generate a random number and get Perception to accept it as the answer to a standard numeric question, since the numeric answer has to be specified as a constant beforehand. Javascript Randomised Numeric Question A work-around for implementing a numeric question in Perception using Javascript is to display it within a multiple-choice question. Depending on the numeric answer entered, the user is asked to select one of several radio buttons, so that Perception can process the multiple-choice question in the normal way. The method works, but is clearly unsatisfactory. Figure 6 A Randomised Numeric Question using Javascript Javascript Randomised and Staged Questions Perception allows the passage through questions to be modified depending on the responses to previous questions. A question may require two or more staged answers. Correct answers allow a student to proceed rapidly to later stages of the question, whereas an incorrect answer results in the display of intermediate stages and hints needed to derive the correct answer, with lower marks then awarded for the correct answer. An immediate problem in trying to achieve staged questions is that each time a jump is made to a different stage in the question, the original randomised variables are lost. The only way found to get around this problem was to place Perception inside a frameset and keep all the variables in a separate window from Perception. Although somewhat inelegant, it works! It was also found that when more than one numerictype question was displayed, the Javascript variables interfered with each other, so it was again necessary to use the frameset procedure to get round the problem. Overall the question-setting procedure using Javascript can only be regarded as moderately user-friendly. Java Randomised Numeric Question A more satisfactory way of extending Perception, and randomised questions in particular, is to incorporate custom Java applets. Question Mark Computing supply a basic template for the Java applet, but the programming has to be done by the question developer. A trial method developed at Portsmouth allows user-friendly setting-up of random numeric questions. Questions all use the same Java code or class, and need three (or four) data items to be inserted by the user, which are then sent into the Java applet as parameters. These four items are: 1. The random numerical parameters to be included in the question 2. A formula for the correct numeric answer 3. The error permitted in the answer 4. The optional width of the text box for the answer, with a default size in the applet if this item is not entered Random numbers in the question are entered in a standard format, e.g. $p$2.0$6.5$ creates a variable p and assigns it a random value between 2.0 and 6.5 inclusive. The answer is displayed using postfix notation, e.g. qba-/d+ specifies an answer of the form d + [ ( a b ) / q ] . The notation has been modified to include standard unary operators, e.g. cos (x) is entered as x$cos$ . The resulting numeric parameters may then be embedded within the text of the question. The authoring and display of a simple question is shown in Figure 7. Since Perception does not handle it automatically, the applet code ensures that, when required, the correct feedback is given, including the correc |
| File Format | PDF HTM / HTML |
| Alternate Webpage(s) | http://www.lboro.ac.uk/service/ltd/flicaa/conf99/PDFs/mccabe.pdf |
| Alternate Webpage(s) | https://dspace.lboro.ac.uk/dspace-jspui/bitstream/2134/1769/1/mccabe99.pdf |
| Language | English |
| Access Restriction | Open |
| Content Type | Text |
| Resource Type | Article |
