Consortium: Australian Council for Educational Research
*Full room set up, including switching on computers and loading test delivery system onto each computer, can be quite time-consuming, especially if computers undergo automatic software updates upon being switched on. If full room set-up is required, please allow for more than fifteen minutes.
17. Conduct CBA Practice Session
20.The following section includes a script for spoken instructions. The only text to be read to the students is in shaded boxes and must be read word-for-word. The un-shaded text is background information and instructions for you, the CBA TA, to follow.
Test Administration Script
A. Introducing CBA
21.ALLOW the students to enter the room and instruct them to sit at their computers.
22.ASK the students to place a pencil and a calculator on their desks, and then PROVIDE note paper to each of them
23.DISTRIBUTE the CBA Login Forms to students. Instruct students not to type anything until told to do so.
You have already completed a test booklet and a questionnaire for the PISA. Now you are going to take part in a section of the PISA, called the “Computer-Based Assessment”. In this study, we would like to find out how good students of your age are at
Please do the best you can because your answers will help influence future educational policies in your country.
Now, please switch off any mobile phones and any other electronic devices.
26.GIVE students a moment to do so.
28.IF any students are not at the login screen, investigate and rectify.
30.DO NOT admit any more students to the session.
B. Beginning the CBA Practice Test
To make sure that all students doing this test around the world get exactly the same instructions, I will be reading them to you.
Now look at your Login form that I handed out. You will see that it has a Student ID and a Password. Enter the Student ID into the Student ID boxes on your screen. Be careful not to make a mistake.
Now enter your Password into the Password boxes on the screen and press the Login button.
32.CHECK that all students have successfully entered their login. The next screen that the students will see appears below.
You are now going to complete a Practice Test which shows you how to use the test system. You have about
Be sure to read each question carefully, and follow the instructions. The questions are in the bottom part of the screen.
Start the Practice Test now by clicking the link.
33.CHECK that all students are now seeing the screen shown below.
Click ‘Yes’ to begin the test.
34.An example of how the practice test appears is shown in the screen shot below. Some explanation of the functionality is also provided. Further information about the help page, the formulae and the calculator is given after the screenshot.
Buttons for copy and paste – copy button only active in some units and paste only becomes active after some text has been highlighted and the copy button has been clicked
Buttons for calculator and formulae – only appear in maths items
Timing bar indicates how much of the available time has passed – green filled section is time passed
Click to go to next question – popup asking for confirmation that you want to go to next question will appear
Click to go to help page
Questions numbers in test – the item currently being viewed is bordered by a white square
36.The help page explains the functionality of the test. To get back to the test from the help page, click on one of the Go back to test links on the right of the page.
37.The calculator and formulae buttons are available for student use only for maths units. When the calculator or formulae button is clicked, the calculator or formulae page will appear. Click on the ‘X’ in the top corner to close these widgets
38.Check that all students have managed to find the
39.The practice test is complete after the student clicks
If you have not done so already, please now close the HELP page by clicking on one of the links saying “Go back to test”. Then click the ‘Next’ arrow, and you should see a link saying “click here to begin test”. Please do NOT click on this button yet. Wait until I give you some more instructions.
41.CHECK that all students have successfully closed the HELP page and that all screens are displaying a message and a link to start the CBA session.
18. Conduct CBA Session and complete Session Attendance Form
C. Starting the 40-minute CBA session
You have 40 minutes to complete the CBA Test. You are not all doing the same test. There are 18 to
As you saw in the Practice Test, there is a progress bar and a time bar. The progress bar shows the questions, and the time bar shows how much time has passed and how much time remains for the test session. These are to help you to manage your time during the test, but don’t get too worried about them. Just complete as much as you can in the time. If you are finding a task very tough, move on to the next one, but remember you can NOT return to a task once you have moved on to the next task.
As you saw in the Practice Test, you can open the Help page by pressing the “question mark” button at the bottom right of the screen. If you have difficulties in using the test system, use the Help page. If you need more help, raise your hand and let me know.
Also, raise your hand and let me know if something in the system seems not to be working. Allow time for the system to respond to your mouse clicks and keyboard presses – sometimes there may be a short delay.
Please begin the CBA Test now by clicking on the link, then clicking ‘Yes’ in the confirmation popup that appears.
44.USE a watch or clock to monitor the time of the session.
45.The timing for the CBA session is monitored by the test delivery system. After the allotted time of 40 minutes has elapsed the test will time out.
46.MONITOR the students. Move around the room, so you can observe if any students are having difficulties understanding how to proceed.
47.RECORD session attendance in the Session Attendance Form for CBA (“Attendance” column).
If a student is absent from the CBA session (NOT due to a technical problem), record “0”.
If a student is present at the CBA session, record “1”.
If a student leaves the assessment room for more than 10 minutes (NOT due to a techcnial problem), record “2” and explain
If a student leaves the assessment room for more than 10 minutes (due to a technical problem), record “3” and explain
If a student is absent from the CBA session due to a technical problem, record “4”.
If a student is absent from the CBA session due to lack of computers, record “5”.
48.In the event of a computer or software crash, please restart the computer and load the test delivery software again by running the ‘PISAMenu.exe’ file and clicking on ‘PISA Assessment’ on the control panel. Then enter the national version encryption password and, when the student login screen is reached, ask the student to enter his/her student ID and password again. The test should return to the item the student was viewing when the crash occurred, and the timing bar should be set at the point it was at when the student first moved to the item he/she was viewing when the crash occurred. The student should then be allowed to continue the assessment either until he/she finishes or the CBA assessment times out. If the crash recovery does not return the student to the point at which the crash occurred, then the feature has not been successful, and the student should be given “3” (partially present due to technical failure) on the Session Attendance form for CBA.
49.As the end time approaches, watch for students who appear to have finished.
50.After the allotted 40 minutes has elapsed, any student who has not finished the assessment will see a dialogue box alerting him/her that the test has timed out. When ‘OK’ is clicked in this dialogue box, the screen shown below under paragraph 55 will appear. This screen will also appear when a student clicks the ‘Next’ arrow on the final question in the assessment. The test session is finished when all student computers show this screen and 40 minutes have passed on the watch/clock you have been using to monitor the timing of the session.
53.DISMISS the students according to the policy of the school.
54.DISCARD any students’ work on paper in a secure manner.
56.COLLECT the CBA delivery USB from each computer by first pressing ALT TAB to access the PISA 2012
20. Complete CBA Session Report Form
58.Please complete a separate Computer-Based Assessment Session Report Form
21. Transfer student responses to
Note to NPMs
This section on transferring the student responses to ACER describes the process if data transfer is to be the responsibility of the Test Administrator. If you plan to centralise the data transfer activities at the National Centre, you will need to describe the data management, backup and transfer processes accordingly.
Please remove this box upon completion of this section.
Note to NPMs (ADDED 28th FEBRUARY 2012)
This section was removed from the TA manual after the release of version 2.0 of the data collection tool on February 24th 2012. If you plan to include instructions for using the data collection tool in your national version of the CBA TA manual, please refer to the instructions that are available on the FTP site on the ‘cbasource’ directory (the same location where version 2.0 of the tool was made available to countries). The instructions available on the FTP site supersede those instructions that were in this section in release version 3 (‘TA_Manaual_CBA_MS12_3.docx’) of this TA manual
Please remove this box upon completion of this section.