1. Introduction

The Multiple Choice Scoring (MCS) system is a package designed for processing multiple choice test data. It can handle tests with up to 120 questions. Each question can have a maximum of 10 choices. The system makes use of a specially designed multiple choice answer sheet (see Appendix I) which is machine readable. These answer sheets are sold to University staff members at cost, $0.5 per sheet. Users may obtain them from our General Office (Room 223, Run Run Shaw Building).

The system comprises of two subsystems - data capture and data analysis, running on a PC/AT in the Special Facilities Room (Room 107, Run Run Shaw Building). Users can reserve the system for use in advance by filling their names onto a particular session of the reservation form posted on RR-107's notice board.

2. Data Capture

An OpScan 5 Optical Mark Reader (OMR) connected to the PC/AT is used for direct capturing of data from the multiple choice answer sheets. An OMR is a device which can detect the presence of oval-marks in specific areas of a pre-designed form. Data read by the OpScan are stored in a file with the following format:

Columns Contents

1 - 40 Header Information

41 - 47 Candidate Number/University Number

49 -168 Answers String (1 column for each question)

The header information stores system specification of the OpScan, this information will be ignored in data analysis.

A candidate number is used to identify each answer sheet. This part must be filled. The OpScan will reject those answer sheets without a candidate number filled. You can use up to 7 digits or less as the candidate number.

The answers string contains 10 columns (A to J) for the ten choices. Please note that each question allows ONLY 1 answer.

The output file is a plain ACSII text file. This file can be input into the data analysis module of MCS to perform scoring and produce some simple statistics. If users have their own specific data analysis requirement, they can apply other software or design their own program and use this file as input. In Appendix III, a sample output file is shown.

Since the answer sheets are to be read by the OpScan for direct data recording, they should not be folded or stapled. To avoid mis-reading, they should also be completed neatly and the answers must be filled completely. Please refer to the instructions on the sample answer sheet in Appendix I about how to mark the answer sheets for perfect result.

3. Data Analysis

The data analysis subsystem consists of the setup of scoring parameters and the analysis of test data. The analysis program takes the test data file and a scoring parameter file as input to produce one marks analysis report and three student scores lists of different sequences - input sequence, candidate number sequence and rank sequence.

4. Running the MCS system

Before invoking the system, change directory to OMR on the C drive:

C:\> CDOMR (then press the Enter key)

The MCS system can be invoked by the command 'OMR':

C:\OMR> OMR

A menu of options will then be displayed. You can either perform data capture or data analysis.

Multiple Choice Scoring System (OPSCAN)

1. Data Capture

2. Data Analysis

3. Exit

choice (1/2/3)

5. Data Capture Procedure

You can refer to Appendix II which illustrates the parts of the OpScan 5.

(a) Power on the scanner

The system will remind you to switch on the OMR with the message

Press the power switch on the right side of the scanner back and you will see the following message displayed in the operator panel of the OpScan 5:

After waiting for about 20 seconds, you will see:

Not Ready ---------------------------START

The 'Not Ready' message indicates that the input hopper is lowered and waiting for you to load forms.

(b) Load forms

Before loading the forms, make sure they are all oriented the same way and are stacked neatly. Load the answer sheets into the input hopper and use the magnetic sheet guide to keep them aligned and closest to the front of the scanner. The input hopper can hold approximately 300 sheets. There is a symbol stuck on the scanner to show you the direction in which the answer sheets are to be placed.

(c) Start scanning

After loading the forms, press any key to go into the following menu:

          0 - Exit to the Operating System
          1 - ScanTools Document System
          2 - Multiple Choice Answer Sheet
          3 - Install/Change Configuration

Use the down arrow key to choose '2 - Multiple Choice Answer Sheet' and press Enter. Then you will see the following screen.

          0 - Return to previous menu
          1 - Scan data documents
          2 - ScanTools Editor 
          3 - Utilities

Choose '1 - Scan data documents'. The system will ask you to input the file name to store the data as follows:

          Enter data file name.....:   MIDTERM [.dat]
          Enter edit profile name:   MCS [.$ep]
          Enter score profile name: [.$sp]

                 Change standard options? (Y/N):

Input the data file name eg. MIDTERM and press Enter. When it prompts you for the edit profile name, you need not change anything and just press Enter, the scanning will automatically start.

(d) Handle problem answer sheet

If there is an error on the answer sheet, say missing the candidate number or marking more than 1 answer for the same question, you will see the following messages on the screen:

Case 1 - Missing Candidate Number
                Edit Failure
        The current document has failed edit.  The edits were:
        Candno          May not be blank

Case 2 - Marked more than 1 answer
                Edit  failure
        The current document has failed edit.  The edits were:
        Qn10            No Multiples

The scanner will stop at this point. You should take out that problem answer sheet from the output tray. You can correct it and put it back to the input hopper and restart the scanner by pressing any key, or you can press any key to continue scanning first and gather all problem sheets to be corrected and re-scan them later after finishing the first batch.

(e) Finish scanning

After finishing the whole batch of answer sheets, the following screen will come up:

        Input hopper empty      Last document:9200011
        Scanner not ready       Expecting.....:Start of document
        
        Add sheets to hopper.  Press any key to resume scanning.

You can load the second batch of answer sheets if there are more than 300 sheets, or if there are some corrected answer sheets to be re-scanned. Press any key to start the scanning after loading another batch of forms.

When you finish scanning all your answer sheets, press the Esc key twice to exit to the Scantool main menu. Use the up and down arrow keys to choose '0 - Exit to Operating System'. You can then proceed to analyse your data or exit the system.

6. Data Analysis Procedure

Before analyzing the test data, a scoring parameter file which contains the marking information should be setup first. To prepare the scoring parameter file, you can select the 'Setup marking Parameters' option which is a simple form filling procedure. The system will then prompt the user to enter the following information:

Report title,

Number of questions in the test,

Last Choice of each question (A-J),

Answer of each question, and

Marks allocation scheme.

You can also modify a previously prepared parameter file by entering the corresponding file name when the system prompts for the parameter file name.

After setting up the parameter file, the test data can be analyzed by selecting the 'Analyze Marking Data' option. The user will be prompted for the name of the parameter file, data file and output file. The output file will contain the analysis results which include marks analysis reports and scores lists in original sequence, candidate number sequence and rank sequence (see Appendix IV).

To obtain a hardcopy printout, exit from the MCS system and return to the DOS environment, then enter the command :

COPY filename PRN where filename = analysis output file

Before entering the above command, please make sure the printer is switched on with paper loaded.

7. Other Applications on OpScan

Besides the Multiple Choice Test Scoring, there should be many other applications using the Optical Mark Reader which can save you a lot of effort on data entry and ensuring accuracy of the data. In our University, some departments use this equipment to capture data for their course evaluation surveys and other types of questionnaires. If you find this equipment can help you in your daily office work or research, you are welcome to consult our Centre staff through the Help Desk Service (Run Run Shaw Building, Rm 109).

Appendix 1 - Sample Multiple Answer Sheet

Appendix 2 - Parts of OpScan 5

Appendix 3 - Sample Output File

Appendix 4 - Sample Analysis Report

Appendix 5 - Marks and Items Analysis Report