Computer Vision Controller

4.6 Computer Vision Controller

4.6.1 Video Image Processing

The video image processing will be divided in three subsystems to facilitate the implementation. The three subsystems are Movement Detection, Acquire Target, and Target Tracking. The input will be coming from the webcam or thermal camera depending on the time or light conditions. A series of frames will be stored in memory which will be utilized by the OpenCV algorithms for the necessary computations. A subsystem that will always be running will be the Movement Detection subsystem to determine if any movement has occurred in the current room where the vehicle is stationed at. Only when a movement is detected is when the other two subsystems will be activated. Once activated, the Acquire Target subsystem will perform a window based search in order to find the target that triggered the subsystem. Once the target has been found, it is sent to the Target Tracking subsystem which will utilize it as input. From here, another window based search will be performed and the top left corner of the target will be stored as an object. Once the target’s position is stored in an object, its motion can be tracked in any direction. Each time the position changes, the motor controllers will be notified in order to move the vehicle and keep the target in the vehicles view. Below is a diagram of how all the mentioned subsystems function:

4.6.2 Cameras

The project will make use of the Logitech C300 as a Webcam and the Tamarisk 320 as the Thermal camera. Depending on the time, day or night, each camera will be toggled. During the day, the Webcam will be utilized since it will provide a better view. During the night, the Tamarisk 320 will be activated in order to show a detailed heat signature. For the position of the cameras, various positions will be tried and tested to see which one provides the best result during the testing phase.

4.6.3 Video Streaming

The input of the video streaming will be the Logitech C300 Webcam. Since OpenCV will always be running, the camera will always be on but the streaming subsystem is only activated once a user opens the mobile application and a request is made. This will allow us to conserve resources such as processing power and battery power. Once a request is made, the streaming subsystem activates the input plugin which starts saving image frames from the camera to the internal storage of the Beagle Bone board. Once there’s a specific amount of frames stored, the output plugin is activated. The output plugin will then either stream the images in a video form or output a single frame depending on the user’s command. A state diagram of the video streaming subsystem is shown below depicting all the possibilities.

Figure 4.6.3 Video Streaming Diagram

4.7 Mobile Application

Description- The mobile application plays a crucial part in the project. It will allow the user to view a live video feed of what the robot is seeing as well as have the option to control the robot. Since the application is going to be able to stream a video feed and is done via the internet, it’s necessary to apply security measures. The video feed will be available via the internet by opening a port in a wireless router/modem. Since the mobile application will be compressed as an application package file (APK), it is safe to define the port that will be used in the application’s program code. In order to protect the privacy of the user, once the application is launched the user will have the option to create an account along with a username and password. Once the account is created, the credentials will be stored in the internal storage of the device using Cryptography as a form of encryption. The encryption will be done by using Android’s built-in functions that can generate a key for the encryption as well as the decryption. Each time the application is launched, the user will be asked to login. As a result, if anyone else takes unauthorized possession of the phone, it will not be possible to open the application. In addition, once the application becomes a background process, meaning that is not the currently active application, it will be terminated immediately. As a result, the user will have to login again.
Design- The application will consist of three different sections, the registration/login section, the registration section, and the control section. The registration/login section will be where the user will have the ability to either register for an account or login with an existing account. If the user already has an existing account, the user will input their username and password. From here, the username and password will be tested to see if they match with the username and password the user used for registration. Below is a diagram of the graphical user interface (GUI) of how the registration/login section will look like:


Figure 4.7-1 Login Form
If the user doesn’t have an existing account, the user can choose to register an account. Once the user picks the option to register for an account, the user will be directed to the registration section. From the registration section, the user will be asked the following: First Name, Username, Password, E-mail, and Cell Phone number. The first name will be utilized to create a customized welcome message in the login/registration section as well as the control section. The username and password will be the credentials that user has to use in order to gain access to the application’s functionality. Lastly, the e-mail and cell phone number will be utilized in order to alert the user whenever the robot has detected any movement. Below is a diagram of the graphical user interface (GUI) on how the registration section will look like:


Figure 4.7-2 Registration Form
If the user logged in successfully, the user will be presented with the control section. In the control section the user has the ability to view the live video feed transmitted via the internet as well having the ability to save a snapshot of the video feed. In addition, the user will have the ability to control the robot by using the virtual joystick in the application. In case there’s an emergency and the user requires calling 9-1-1, a button will be included in the control section where once activated the application will make the call to 9-1-1 for the user. Below is a diagram of the graphical user interface on how the control section will look like:


Figure 4.7-3 Control GUI
State Diagrams- The mobile application has various section and each serve a different purpose. Below are detailed state diagrams for each section depicting all the possibilities that the user may encounter.
Registration/Login Section-



Figure 4.7-4 Mobile Application Registration/Login State Diagram
Register Section-



Figure 4.7-5 Mobile Application Registration State Diagram
Control Section-



Figure 4.7-6 Mobile Application Control State Diagram
Class Diagram- In Android, each section or screen is referred to an Activity and each Activity is a JAVA class. In order to go from Activity A to Activity B an intent is required. An intent is an “intention” to perform an action, in this case, to switch between activities. The mobile application will consist of three activities, the first activity will be where the user has the option to register or login with their existing account. The second activity will be the registration activity which will allow the user to create an account. The third activity will be the control activity where the user has the ability to control and send commands to the robot. Below are the class diagrams along with the necessary methods and functions:


Figure 4.7-7 Mobile Application Class Diagram
The Main Activity will present the user with the graphical user interface (GUI). From the Main Activity depending the user’s choice, intent will be made to either show the Registration Activity or the Control Activity.

Directory: seniordesign
seniordesign -> Executive Summary 4
seniordesign -> Hvac control and Feedback System 0 Group 2 Steve Jones Mathew Arcuri Elroy Ashtian, Jr
seniordesign -> Group 5 Spr-Sum 2011
seniordesign -> Senior Design II paper
seniordesign -> Remote Touchscreen-Controlled Defense Turret Senior Design Documentation Courtney Mann, Brad Clymer, Szu-yu Huang Group 11
seniordesign -> Flight Performance Data Logging System
seniordesign -> Amphibious vehicle
seniordesign -> Ehvac: Wireless Modular Multi-Zone hvac controller Group b javier Arias Ryan Kastovich Genaro Moore Michael Trampler
seniordesign -> Table of Contents Executive Summary 2

Download 0.53 Mb.

Share with your friends: