Features not to be tested

5.Features not to be tested

• 
  Database. Web application will perform the integrity check and all data retrieved will not be altered so database will not be tested as a standalone feature.
  
• 
  Large number of users as high-load stress. High-load is only possible when the product is released to public because the emulated test result in lab is not convincible.
  
• 
  Network delays. It’s not possible to control the network delay as in realistic environment so it will not be covered in this acceptance test.
  

6.Approach

• 
  Unit test. Developers are responsible for unit test as white-box testing. The implementation of each module and individual component will be verified separately.
  
• 
  Integration test. After the unit test is passed above the defined quality threshold, testers will execute the integration test cases. After all the modules are integrated, it’s crucial to test the product as a black-box. End-to-end scenarios will be tested to ensure the communication functionality.
  
• 
  Regression test. After developers fix the bug in one feature, regression test will be executed by testers to ensure that the other functions are not affected.
  
• 
  Field test. Firstly, untrained end users recreate one or more existing (but narrow) mass observation events in the MOb system. A number of observers will be invited to help with evaluation. After that, post event questionnaires will be used to collect quantitative usage data as well as qualitative data and further improvement will be taken into consideration.
  
• 
  Positive and negative testing design technique. This approach will be combined with unit test and integration test. Test cases are designed in sunny day scenarios, which ensure that all functional requirements are satisfied. What’s more, rainy day test cases will also be covered to show how the system reacts with invalid operations.
  

6.1Approach to configuration and installation

7.Item pass/fail criteria

• 
  Preconditions are met
  
• 
  Inputs are carried out as specified
  
• 
  The result works as what specified in output => Pass
  
• 
  The system doesn't work or not the same as output specification => Fail
  

7.1Installation and Configuration

7.2Documentation problems

Test plan document should describe test cases clearly without ambiguity, which helps tester to execute the test cases and validate the requirements.

User manual document should be finished completely before filed test, or untrained end user will have trouble in using the MOb product correctly.

8.Suspension criteria and resumption requirements

9.Environmental needs

9.1Hardware

• 
  Laptop or Desktop PC
  
• 
  Mobile with Android platform
  

9.2Software

• 
  PC Operating System:
  

• 
  Windows XP (32-bit)
  
• 
  Windows Vista (32- or 64-bit)
  
• 
  Windows 7 (32- or 64-bit)
  

• 
  Browsers: (screen resolutions of 1024x768px or more)
  

• 
  Opera 10 or later
  
• 
  Microsoft Internet Explorer 7 or later
  
• 
  Mozilla Firefox 3 or later
  
• 
  Apple Safari 4 or later
  
• 
  Google Chrome 4.0 or later
  
• 
  Opera Mini 10+ as mobile web browser
  

• 
  Mobile Platform: Android 2.0/2.1 (Eclair) Based on Linux Kernel 2.6.29 or later
  
• 
  Eclipse 3.4 (Ganymede) or 3.5 (Galileo) with ADT Plugin
  

9.3Other

• 
  Internet connection for PC
  
• 
  WiFi/3G/GPRS connection for mobile
  

10.Test procedure

10.1Test case specifications

10.1.1Web Application - WEBAPP

10.1.1.1Admin creates user - WEBAPP-001

Administrator add previously inexistent user to the database.

Positive & Negative

None
Input definition:

1. 
   Administrator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Administrator selects “Create New Account” from “Admin Panel”.
   
3. 
   Administrator fills in the required profile to create a new user, username, password, confirm password, first name, last name and roles.
   
4. 
   After filling in the required information, press “Save”.
   

1. 
   When all the required profiles are filled and the user doesn’t exist before, click “View Accounts” to check that the user is successfully added.
   
2. 
   When administrator does not fill in enough profiles, display an error message.
   
3. 
   When the new user already exists, display an error message.
   
4. 
   When administrator doesn’t click “Save” and click other pages, the filled data will be lost and the new user will not be added.
   

This is the basic case that starts the MOb application.

10.1.1.2Admin edits user - WEBAPP-002

Administrator edits user’s profile.

Positive & Negative

There is at least one existing user in the database.

1. 
   Administrator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Administrator selects “View Accounts” from “Admin Panel”.
   
3. 
   Administrator selects the user to be edited from the list and update the profiles.
   
4. 
   After updating, press “Save”.
   

1. 
   When the required fields are filled/updated completely, click “View Accounts” to check the details of the user is successfully updated.
   
2. 
   When the necessary fields are not complete, display an error message.
   
3. 
   When administrator close the edit window, the filled data will be lost and the user will not be updated.
   

None

10.1.1.3Admin views user - WEBAPP-003

Administrator views user’s profile details.

Positive

There is at least one existing user in the database.

1. 
   Administrator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Administrator selects “View Accounts” from “Admin Panel”.
   
3. 
   Administrator selects the user to be viewed from the list.
   
4. 
   After viewing, close the pop up window.
   

1. 
   Administrator can view the user’s profile details successfully.
   

None

10.1.1.4Admin deletes user - WEBAPP-004

Administrator deletes an existing user.

Positive & Negative

There is at least one existing user in the database.

1. 
   Administrator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Administrator selects “View Accounts” from “Admin Panel”.
   
3. 
   Administrator selects the user to be deleted from the list and click the delete button. The user selected could be a initiator, observer, consumer or guest user (inactive user).
   
4. 
   Administrator confirms the action by clicking “Delete”.
   

1. 
   Click “View Accounts” to check the account is successfully deleted from the list.
   
2. 
   When administrator clicks “Cancel”, the user will not be deleted.
   
3. 
   When administrator closes the pop up window, the user will not be deleted.
   

None

10.1.1.5Initiator creates an OE - WEBAPP-005

Initiator creates an observation event and makes it available to the observers.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator clicks the “Create observation event” from “Initiator Panel”.
   
3. 
   Initiator fills in the observation event details, title, description, start date, end date, click “Next Step”.
   
4. 
   Initiator selects interface components, note, image, audio, video, radio poll, checkbox poll.
   
5. 
   Initiator selects group(s) as initiators/consumers.
   
6. 
   Press “Save” button.
   

1. 
   Click “View own observation events” from “Initiator Options” and check the OE is successfully created.
   
2. 
   When initiator logins with an invalid username or password, display an error message.
   
3. 
   When initiator does not fill in the key fields, display a warning message.
   
4. 
   When initiator presses “Cancel”, the entered data of the OE will be lost and no OE will be created.
   

This case covers requirements INI-1, INI-7 and INI-8.

10.1.1.6Initiator starts an OE - WEBAPP-006

Initiator starts an existing observation event.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database, and the observer belongs to a specific group.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator creates a new OE, defines the start date and selects the group.
   
3. 
   When the start date meets, observer(s) in the group login to the mobile application.
   

1. 
   Observer gets a new OE notification when login to the mobile application. The OE is successfully started.
   
2. 
   When initiator logins with an invalid username or password, display an error message.
   
3. 
   When initiator does not fill in the key fields, display a warning message.
   

None

10.1.1.7Initiator terminates an OE - WEBAPP-007

Initiator terminates an existing observation event.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database., and the observer belongs to a specific group.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator creates a new OE, defines the start date, the end date and selects the group.
   
3. 
   When the end date meets, observer(s) in the group login to the mobile application.
   

1. 
   Observer cannot upload any observations when the end date meets. The OE is successfully terminated.
   
2. 
   When initiator logins with an invalid username or password, display an error message.
   

None

10.1.1.8Initiator edits an OE - WEBAPP-008

Initiator edits an existing observation event.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least an observation event created by the initiator in the database.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator clicks “View own observation events” from “Initiator Panel”.
   
3. 
   Initiator selects one observation event from the list.
   
4. 
   Press “Edit” button.
   
5. 
   Initiator edits the details of observation event.
   
6. 
   Initiator presses “Save” button in each step.
   

1. 
   Click “View own observation events” from “Initiator Options” and click the updated OE name to check that the OE details are successfully updated.
   
2. 
   When initiator logins with an invalid username or password, display an error message.
   
3. 
   When the key fields are not complete, display a warning message.
   
4. 
   When initiator presses “Next Step” or “Exit”, the edited data of the OE will be lost and the OE will not be updated.
   

None

10.1.1.9Initiator deletes an OE - WEBAPP-009

Initiator deletes an existing observation event.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least an observation event created by the initiator in the database.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator clicks “View own observation events” from “Initiator Panel”.
   
3. 
   Initiator selects one observation event from the list.
   
4. 
   Press “Delete” button.
   
5. 
   Initiator presses “Delete” button on the pop up message.
   

1. 
   Click “View own observation events” from “Initiator Options” to check the OE is successfully deleted. And the observation list is refreshed.
   
2. 
   When initiator logins with an invalid username or password, display an error message.
   
3. 
   When initiator presses “Cancel” button or close the pop up window, the OE will not be deleted.
   

None

10.1.1.10Initiator views an OE - WEBAPP-010

Initiator views an existing observation event.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least an observation event created by the initiator in the database.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator clicks “View own observation events” from “Initiator Panel”.
   
3. 
   Initiator selects one observation event from the list and clicks the name.
   

1. 
   A list of the observation events owned by initiator will be displayed.
   
2. 
   When initiator clicks on the OE name, the OE details will be displayed.
   
3. 
   When initiator logins with an invalid username or password, display an error message.
   

None

10.1.1.11Guest registers - WEBAPP-011

Guest registers on the web application.

Positive & Negative

1. 
   Guest navigates to http://161.53.67.212/.
   
2. 
   Press “Register” button.
   
3. 
   Guest fills in the user detail form, username, password, confirm password, first name and last name.
   
4. 
   Guest selects the role as an initiator, observer, consumer or a combination of them.
   

1. 
   Administer login and view the accounts to check the guest is successfully registered as a new user of the application.
   
2. 
   When guest doesn’t fill all the key fields, display an error message.
   
3. 
   When guest registers as an existing user, display an error message.
   
4. 
   When guest switches to another panel, all the input data will be lost and the guest will not be a registered user.
   

None

10.1.2Mobile Android Application - MBAPP

10.1.2.1Observer views available OE - MBAPP-001

Observer views available observation events.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database.
   
3. 
   There is at least an observation event created by the initiator and available to the observer.
   
4. 
   Observer has internet access.
   

1. 
   Observer logins the MOb mobile application.
   

1. 
   A list of the observation events owned by observer will be displayed.
   
2. 
   When observer logins with an invalid username or password, display an error message.
   

This is the basic case for observers to make observation.

10.1.2.2Observer accepts/declines OE - MBAPP-002

Observer accepts/declines an observation event.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database.
   
3. 
   There is at least an observation event created by the initiator and available to the observer.
   
4. 
   Initiator starts the observation event and an invitation is sent to observer.
   
5. 
   Observer has internet access.
   

1. 
   Observer logins the MOb mobile application.
   
2. 
   Observer clicks on the “new notification”.
   
3. 
   Observer views the observation event details and click “Accept” or “Decline”.
   

1. 
   Observer can accept/decline the observation event successfully.
   
2. 
   When observer logins with an invalid username or password, display an error message.
   

None

10.1.2.3Observer makes observation - MBAPP-003

Observer makes observation with different interfaces.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database.
   
3. 
   There is at least an observation event created by the initiator and available to the observer.
   
4. 
   Initiator starts the observation event and an invitation is sent to observer.
   
5. 
   Observer accepts the invitation.
   
6. 
   Observer has internet access.
   

1. 
   Observer logins the MOb mobile application.
   
2. 
   Observer clicks the OE from the list.
   
3. 
   Observer makes observation by writing notes, taking pictures, polling in checkbox/radio, recording audio or recording video as required.
   
4. 
   Observer presses “Save” button.
   

1. 
   Observer can make the observation to server successfully.
   
2. 
   When observer logins with an invalid username or password, display an error message.
   

None

10.1.2.4Observer sends observation - MBAPP-004

Observer sends observation.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database.
   
3. 
   There is at least a user created by administrator acting as the role of consumer in database.
   
4. 
   There is at least an observation event created by the initiator and available to the observer.
   
5. 
   Initiator starts the observation event and an invitation is sent to observer.
   
6. 
   Observer accepts the invitation.
   
7. 
   Observer has internet access.
   

1. 
   Observer logins the MOb mobile application.
   
2. 
   Observer clicks the OE from the list.
   
3. 
   After making observation, observer presses “Send”.
   
4. 
   Observer presses “Back” button to return the main screen.
   

1. 
   Consumer login to the web application and view the send data to check observer sends the observation successfully.
   
2. 
   When observer press “Close”, the observation will not be sent.
   
3. 
   When observer logins with an invalid username or password, display an error message.
   

None

10.1.2.5Observer offline functions - MBAPP-005

Observer makes observation without internet connection.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least a user created by administrator acting as the role of observer in database.
   
3. 
   There is at least a user created by administrator acting as the role of consumer in database.
   
4. 
   There is at least an observation event created by the initiator and available to the observer.
   
5. 
   Initiator starts the observation event and an invitation is sent to observer.
   
6. 
   Observer accepts the invitation.
   
7. 
   Observer doesn’t have internet access.
   

1. 
   Observer uses the MOb mobile application by clicking “Skip”.
   
2. 
   Observer makes observation to accepted OEs.
   
3. 
   Observer clicks “Save” to save the observation results.
   
4. 
   Observer presses “Back” button to return the main screen.
   
5. 
   Connect to internet. Observer login MOb mobile application and upload the offline observations.
   

1. 
   Consumer login to the web application and view the send data to check observer sends the observation successfully.
   

None

10.1.3Mobile Web Application - MBWEB

10.1.3.1Observer views available OE - MBWEB-001

10.1.3.2Observer accepts/declines OE - MBWEB-002

10.1.3.3Observer makes observation - MBWEB-003

10.1.3.4Observer sends observation - MBWEB-004

10.1.4End-to-End Scenario - E2E

10.1.4.1Set up an OE and run the event - E2E-001

Initiator creates a mass observation event and makes it available to a group of observers. Then observers make observation by the means required and send the observation data to web server. The observation data will be collected for further review and analysis.

Positive

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least an observation event created by the initiator in the database.
   
3. 
   Observers have internet access.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator chooses a set of people as observers by name/email, by region, or by other discernible attributes (or even by direct permission) as groups.
   
3. 
   Initiator chooses the means for observer to observe, writing notes, taking pictures, polling checkbox/radio, recording audio and recording video.
   
4. 
   Initiator creates an observation event following WEBAPP-005 & WEBAPP-006.
   
5. 
   Observers login in the mobile Android application or use Opera Mini to browse to http://tiny.cc/404pn
   
6. 
   Observers get a notification of “new observation notification” and accept it.
   
7. 
   Observers make observation and send the observation result to web server.
   
8. 
   Consumers login in the website.
   
9. 
   The observation results of the observation event are displayed for consumers. Consumers can filter the result by fields.
   

1. 
   Initiator, observers and consumers can login the system successfully with valid username and password.
   
2. 
   The process of running the observation event is successful without error.
   
3. 
   The observation data can be reviewed and analyzed by the consumers.
   

This is a basic end-to-end scenario for integration test, which consists some of the test cases above. The communication between mobile and web is verified in this test case. When observers don’t have internet access, the observation data can also be sent to web server by cable connection with the PC.

10.1.5Non-functional Scenario - NFR

10.1.5.1Authentication - NFR-001

When observation requires to be “signed” by an observer, system will generate a key for the observer selected. And only observation data with the key matched will be valid. This allows the initiator to determine if the observation has come from an invited observer.

Positive & Negative

1. 
   There is at least a user created by administrator acting as the role of initiator in database.
   
2. 
   There is at least an observation event created by the initiator in the database.
   

1. 
   Initiator navigates to http://161.53.67.212/ and login the MOb web application.
   
2. 
   Initiator creates an observation event and makes it available to the invited observers.
   
3. 
   Invited observers make observation and send to web server.
   

1. 
   The observation data sent by invited observers will be accepted by web server. However, the observation data sent by uninvited observers will be rejected by the web server and not display on the website for consumers.
   

None

10.1.5.2Load Website - NFR-002

Load a web application must be less than 5 seconds.

Positive

1. 
   Navigate to http://161.53.67.212/.
   
2. 
   Repeat the step 1 for times and record the loading time.
   

1. 
   Calculate the average loading time. The website should be loaded in less than 5 seconds.
   

None

10.1.5.3Search database - NFR-003

Search the database and print the results must be less than 10 seconds.

Positive

1. 
   Initiator navigates to http://161.53.67.212/ and login in the web application.
   
2. 
   Initiator views the observation event and the observation list is displayed.
   
3. 
   Repeat step 2 and record the searching time.
   

1. 
   Calculate the average searching time. The result should be displayed in less than 10 seconds.
   

None

10.1.5.4System Recovery - NFR-004

If the system goes down, all data from the database must be preserved.

Negative

1. 
   Run the event as specified in E2E-001.
   
2. 
   In the step 1 process, disconnect the network cable and recovery the internet connection.
   

1. 
   The system recovery successfully and all the data are preserved in database.
   

It’s hard to trigger a system crash, and it’s simulated by plugging out the network cable in this case.

10.2Test plan

11.Responsibilities

11.1Developers

• 
  Unit test when implement their owned models
  
• 
  Fix the bug and make sure the code change does not affect other functions
  
• 
  Deliver a new testable version to tester
  

11.2Testers

• 
  Write acceptance test plan
  
• 
  Execute test cases and record test results
  
• 
  Inform developers with the bugs as early as possible
  
• 
  Summarize the results as Test Report
  

11.3User representative

• 
  Clarify the suspicious requirements
  
• 
  Comment on the test plan and test cases
  

12.Risks and contingencies

13.Approvals

Name	Title	Date yyyy-mm-dd	Signature