Psu ece robotics 2 Class Project Tony Muilenburg, Dawson Green

Robot firmware code

Robot control code

The user interface is split up into two tabs. The first allows the user select the Bluetooth device that is to be used, and configure it. While some laptops come with integrated Bluetooth, the laptop used for this did not, so a USB dongle was added which added it. After selecting the Bluetooth port, the connect button will enable communication with the NXT equipped robot that has been flashed with the robotc firmware created for this project. The computer vision code uses a text file to pass location data to this robot control code. The path to the file the computer vision code generates should be added to the textbox using the find file button which pops up a file browse dialog window. The “Test File” button should then be used to ensure that the path is valid and file permissions are ok (permission to read the file is granted).

The second tab has buttons to allow for manual or automatic control. In manual mode, the robot can be turned, or orders can be given for it to go forward or backward. A speed control is available, as well as an emergency stop (see below).

In automatic mode, the code will read the coordinate from the file the computer vision code generated, then a heading and distance will be generated from the robot to a blue dot, and commands will be send to have the robot turn and drive to the blue dot. The coordinates are written and read many times per second to ensure that the robot does not drive too far, or get off course.

Computer vision code

The top four windows show the threshold image for a particular color. The colors red, blue, green and pink are tracked, though red is not currently in use (see the next steps section for more on this). In looking at the green and pink threshold windows, it is clear that these colors exist in the camera feed. All other colors are filtered out by specifying filter in HSV format (follow this link for a description of HSV). Along the bottom, there are four windows each of which have sliders that can be used to adjust the H, S, and V values and filter out all but the desired color to be tracked. The “camera feed” window in the middle of the lower portion of the screen shows what the camera sees along with superimposed circles that correspond to areas being tracked.

Next steps

• 
  Put together a third option for scripted control in addition to manual and automatic control of the baby Schrodinger cat.
  
• 
  Enable tracking and control of other robots, including Schrodinger cat, Einstein, Newton, Bohr, Marie Curie, etc.
  
• 
  Enable finer granularity for the fuzzy logic speed control
  
• 
  Incorporate depth control using a Kinect camera
  
• 
  Train and use a Haar cascade classifier to recognize robots, and robot orientation (color would no longer be needed for tracking).
  

CatBot Control Introduction

This document describes the breakdown of files used in the CatbotControl program for the baby Schrodinger Cat robot (referred to as catbot throughout). Each section of the project is broken down in turn and notes on the usage and setup configuration for it are discussed. Be aware that full understanding of the code will likely only come from reading it and experimenting with it. However, this should hopefully help expedite that process along. If any objects or styles of code are unfamiliar to you when you come across them, please refer to the Microsoft MSDN for details about Visual C++ and the Common Language Runtime (CLR).
NOTE: This document assumes you are using Visual Studio Professional 2013. If you are using a different IDE to compile this project, directions to edit settings may be different. Please use your best judgement to interact with your IDE UI and update this document if future classes will use a different program to interact with this code.

Main Execution File (main.cpp)

The main routine is cleverly hidden in the main.cpp file. This file is fairly barebones and is a useful format for how to invoke a GUI in Visual C++. The file does contain a few notes on project settings, so make sure these are configured correctly before compiling the code. To edit them, right click the project in the solution explorer and select “Properties”. This opens the master set of project properties for modification.

Compilation Warnings

Any time the code produces are warning for C4101, you are free to ignore it. This warning is generated by the compiler letting you know that a local variable isn’t actually used in its function and most often caused by try/catch blocks. If in doubt, check the line of code throwing the warning. There will be a comment if it was expected and should be ignored.

Namespaces

When possible, this code declares and uses namespaces to shorten down conventions. However, for accuracy, this document will often expand out the namespace of an object so it’s clear what object is being discussed. Namespaces are separated by the double colon indicator “::”, with the last keyword on the line being the actual object in question. Please check Google if you need more information on how these work.

Global Definitions File (definitions.h)

The global definition file is included in all other header files. It is used as a centralized hub for all file includes, all #defines and all the global variables in this namespace. Note that #defines aren’t restricted by Visual C++ to a particular namespace, so care should be exercised about naming conventions for them.

#define values

Visual C++ doesn’t provide a way to define string constants when using CLR. While most numerical values have been locked in as globals, we have not found a good way around using #defines to drop in strings to path files yet. Please update this section if you find a way.

Use of enums

If you’re not familiar with enums, they provide a good way to turn keywords into integers. Each enum defaults to numbering its elements from 0 to length-1. While you can manually order these and change the numbering system, this isn’t encouraged. It is expressly prohibited for the BotID enum, as the TOTAL_BOTS value would then be inaccurate. Please refer to Google if you need more information on how these work.

Use of ref structs & ref classes

The really cool thing about Visual C++ is that it has a garbage collector available to it that’s similar to other garbage collected languages. However, this causes a split between “managed” code and “native” code. Managed objects can always be identified by the hat (“^”) character at the end of their declaration (“Object^”). If all references to an object are deleted by destructor or go out of scope of the code, then the objects destructor will be automatically invoked. You can create your own managed objects simply by putting the keyword “ref” before the declaration of any struct or class you create. For the most part, this project uses them to make life easier and more automagical.

Port Manager Object (PortManager.h & PortManager.cpp)

This object provides a useful abstraction for interacting with the catbot and any future robots you choose to tie into this project. At its core, it is designed to manage all of the serial ports (COM ports) used to communicate with robots controlled by this application. From the outside, a user simply passes the appropriate BotID and this object will handle sending or receiving data to the right robot. While only catbot is currently established in the code right now, stubs are provided to show how the functions can be expanded to control a full array of robots. NOTE: This object can only interact with robots registered to a Windows COM port. If you have another communication system, you’ll need a new class.

Port Available and Robot Ready functions

While the NXT provides the Bluetooth Serial Port Profile (SPP) that allows it to be controlled over a COM port when paired to the computer, the BT receiver on it is fairly slow. It’s so slow, in fact, that testing it for availability is a prohibitively time expensive operation. Therefore, this code will only check if ports are currently owned by this program and configured for this program’s robots before trying to interact with them. If the port gets hijacked by another process, this code won’t realize it and will probably start doing weird things.

Sending BT Messages to an NXT

While the catbot has its own code to interpret the messages to the NXT and translate them into motions, the NXT itself requires messages to be formatted in specific ways. Each communication is referred to as a “telegram” and a master list can be found in the supplementary documentation for RobotC. For this project, each message can be broken down and understood in the following way:
array<Byte>^ BT_Msg = gcnew array < Byte > {0x07, 0x00, 0x80, 0x09, 0x00, 0x03, msg, msgMag, 0x00};

1. 
   Variable type (“array^”) – Managed array of 8 bit bytes gets automatically disposed of after sending.
   
2. 
   Message Length (“0x07, 0x00”) – Each message must start with the length of the actual payload, LSB first. This format indicates that there are 7 bytes being sent not including the two bytes declaring the length.
   
3. 
   Command Type (“0x80”) – This command doesn’t need a response from the NXT.
   
4. 
   Command (“0x09”) – Our code is writing a message to the system
   
5. 
   Message Mailbox (“0x00”) – The message will be in “mailbox1” for the RobotC code to read from.
   
6. 
   Message Size (“0x03”) – Number of bytes of actual message data (i.e. length of the remainder of the array)
   
7. 
   Message Contents (“msg, msgMag, 0x00”) – The RobotC code specifies that the robot command and command speed are bytes 1 and 2 of the message. The NXT requires all messages to terminate with a null character, so “0x00” must be tacked on to the end of any message and included in the payload calculations.
   

Please refer to “Appendix 2 – LEGO MINDSTORMS NXT Direct Commands.pdf” for details of how to format and send other messages to the NXT.

SerialPort object

Most of the structure of this code is making use of functions and properties provided by the System::IO::Ports::SerialPort object. MSDN has excellent documentation about the properties of this object and you’re encouraged to check it out if you need more info on an object function call.

Catbot Auto Control object (CatbotAutoCtrl.h & CatbotAutoCtrl.cpp)

Just as the PortManager helped abstract the interface layer for communicating with the robot, the Auto Control object is designed to abstract the process of reading data from the OpenCV program and determining what to do with that data. Because each robot will consume data differently, this object is only targeted to the catbot. However, it should provide a good template if you want other robots to consume OpenCV code in a similar fashion.

FilePath variable

While the object used to read data from the file in readData() can theoretically support relative paths to the file, it is our experience that this can cause some odd read/write issues with the file. These were fixed by using the absolute path of the variable. If you can ever figure out why, please update this document with the answers.

Reading OpenCV data

The OpenCV program takes measurements of position of the necessary markers and provides those in a valid text file. The GUI can find this file in the system and test that it is able to be read, but reading that file requires a set of keys. Back in definitions.h, there’s a series of keywords that corresponds to the data relevant to catbot from OpenCV. Note that ORDERING IS IMPORTANT for the moment, as each keyword is mapped to a specific variable in the data reading. The order the keywords appear in the text file doesn’t matter (or if unexpected keywords are there), but the order in the definitions.h file does.
Each keyword expects an integer to be read from the textfile, separated by any of the valid delimiters found in definitions.h. Currently, a space (‘ ‘) is used. This program will use position data to calculate angles, so the more accurate those integers can be, the better. Values are currently scaled to the number of pixels viewed by the webcam.

Calculating Catbot Motion

The algorithm works basically by calculating differences between the target and the current position/heading of the catbot. Angles are constrained to being between -180 and +180 degrees, with conversions being provided to transform radians into degrees. The process is that, if the catbot is “close enough”, it will stop. If it is in the right general direction, it will drive forward. If it still needs to turn, then it will turn in the “fastest” direction. Catbot motion resulting from this calculation are then packaged and returned to the caller. NOTE: the calculation will return a null pointer if the read OpenCV data was invalid or if the calculation causes a repeat in the motion being sent to the catbot. The caller is expected to NOT send data if it gets a null pointer back.

Application UI (AppUI.h, AppUI.cpp & AppUI_Events.cpp)

We now come to the main application UI, where everything needed for the UI and everything else that couldn’t be abstracted out has been crammed in. Much of the header file was created automatically by the Visual Studio GUI creator interface. As such, you only need to refer to the top and bottom of the file to find declarations made by our team. Also, code is split into UI events and “other” functions. AppUI.cpp contains all the non-event function definitions.

Functions for the Setup Tab

These functions are mostly presented as is. They are primarily utilitarian and necessary to initialize data needed for the actual control of the catbot. All functions here will be executed on the main GUI thread, so we have tried to minimize slowdowns. In order to improve abstraction of the methods (and allow greater extension of the code), helper functions have been created to gather and package data necessary to pass to other functions. Similar GUI gather functions should be of a similar format to catbotGetSettings(), as this allows utility functions further downstream to update the GUI and modify the system via indirection.

Manual Controls

Once setup is complete, manual control can be enabled under the control tab. Logic for the motion buttons (except for stop) is to allow user to ramp up motor speed by repeatedly clicking on the same motion command (start at 10% power and increase by 10% each click). Each time a new motion is started, it defaults to starting at 10% motor power. There is also a slider for manual speed control. Updating the slider causes a thread to get started which reads the slider and waits to send the command until it has been stationary for about 100 ms. This is again to accommodate the NXT’s BT receiver only being able to handle new data once every 30 ms. On a stop or emergency stop, the speed bar is set to 0 and disabled.
Also, enabling manual control will cause automatic control to disengage.

Automatic Control

Enabling automatic control will disable manual control and cause a new thread to be spawned. This thread used to actually control and manage the Catbot Auto Control object. It is designed so that it is frequently sleeping, then waking up to check if it should do work. The sleep/wake-up model allows a volatile bool variable to be checked frequently. If this variable is ever set to false, the thread will assume it should be ended and will exit the controlling function. Otherwise, it will run through each of the 4 states and direct traffic between the auto controller and the port manager to make sure messages get where they need to go.

Download 1.73 Mb.

Share with your friends: