Augmented Reality Control of the Telerobot 2003

Introduction 8

Concept 8

Programming Language 9

Operational Equipment 9

Time delay/bandwidth limitations 9

User Manual 12

Block modelling 14

Module Definitions Detail 21

1. 
   Image capture hardware and set up 23
   

1. 
   Initial Specification 23
   
2. 
   Implementation 25
   
3. 
   Final Specification 29
   
1. 
   Image Capture Software 30
   
   1. 
      Initial Specification 30
      
   2. 
      Design/Implementation 30
      
   3. 
      Testing 32
      
      1. 
         Zooming 2x 32
         
      2. 
         Mirroring 33
         
   4. 
      Final Specification 34
      
   5. 
      Documentation 35
      
2. 
   Camera/Image Calibration 38
   
   1. 
      Procedures 38
      
   2. 
      Module Specification 43
      
   3. 
      Implementation issues 46
      
   4. 
      Test Specification 47
      
3. 
   Interfacing Image Calibration 48
   
   1. 
      Initial Specification 48
      
   2. 
      Design Process 49
      
   3. 
      Testing 52
      
   4. 
      Final Specification 54
      
   5. 
      Documentation 55
      
      1. 
         Prism transformation interaction 57
         
4. 
   GUI 61
   
   1. 
      Specification 61
      
   2. 
      Design 61
      
      1. 
         The model 61
         
      2. 
         Manipulations Sequence 62
         
   3. 
      Testing 65
      
5. 
   AR graphics 66
   
   1. 
      Specification 66
      
   2. 
      Design and implementation 66
      
      1. 
         3D prism representation 66
         
      2. 
         2D prism representation 68
         
      3. 
         3D prism manipulation 69
         
         1. 
            Translation 69
            
         2. 
            Scaling 70
            
         3. 
            Rotations 70
            
      4. 
         3D to 2D mapping equations 71
         
      5. 
         Drawing the prism 72
         
      6. 
         Greying the back of prism 74
         
      7. 
         Control points 76
         
         1. 
            Assigning control points 76
            
         2. 
            Drawing control points 79
            
      8. 
         Translation lines 81
         
      9. 
         Prism calibration 81
         
   3. 
      Testing 85
      
      1. 
         Control points 85
         
      2. 
         Eye point 88
         
   4. 
      Final Specification 90
      
   5. 
      Documentation 90
      
6. 
   Cursors 100
   
   1. 
      Initial specification 100
      
   2. 
      Design and Implementation 100
      
      1. 
         Cursor types 100
         
         1. 
            Select cursor 100
            
         2. 
            Rotate cursor 100
            
         3. 
            X & Y position cursors 101
            
         4. 
            Direction manipulation 101
            
   3. 
      Testing 103
      
   4. 
      Future Work 104
      
   5. 
      Final Specification 104
      
7. 
   Interaction State Machine 105
   
   1. 
      Specification 105
      
   2. 
      Design and implementation 105
      
      1. 
         State 105
         
      2. 
         Input 107
         
   3. 
      Testing 111
      
8. 
   Event logging 112
   
   1. 
      Initial Specification 112
      
   2. 
      Design and Implementation 112
      
      1. 
         Basic operation 112
         
      2. 
         Wrapping the buffer 113
         
      3. 
         Additional functionality 113
         
      4. 
         To string vis 114
         
   3. 
      Testing 114
      
   4. 
      Future work 119
      
   5. 
      Final Specification 119
      
   6. 
      Documentation 119
      
9. 
   ABB Robot Control 121
   
   1. 
      Module specification 121
      
   2. 
      Implementation issues 123
      
   3. 
      Test specification 124
      

Appendix A: Trouble Shooting 127

Appendix B: Module Specification Summary 130

Appendix C: MATLAB Scripts 151

Appendix D: References 164

Introduction

Objectives

In the effort of facilitating an augmented reality interface, the team would have to:

• 
  Incorporate the concept of augmented reality into the user’s operating environment through a control prism.
  
• 
  Create a graphical user interface by incorporating additional display and control features.
  
• 
  Enhance the image capture, both in hardware and software, of the actual operating environment along with camera calibration.
  

Concept

This idea furthers the work of Harold Friz (1998) who suggested a similar approach to this as an extension to his development of USHER, an AR tool used to specify the robot’s end effector position and orientation. The potential of object domain commands far exceeds that of Cartesian space commands and robot domain control such as USHER, especially within static systems, because of its inherent intuitiveness. Moreover, object commands allow the use of increased computer task load thus reducing the operator’s task load. The operator will have the prospective ability to delegate tasks to a computer system such as determining suitable robot positioning and orientation, all robot movements, path planning, object avoidance and placement schemes.

Programming language

The ARIOM interface is programmed using LabVIEW so that it is easily implemented within the actual UWA Telerobot control interface which is currently being reprogrammed in this language. Since a rudimentary level of graphics is required for AR based on bandwidth restrictions, LabVIEW is easily able to handle such tasks. It also allows excellent hardware connectivity allowing the interfacing with digital cameras and any input devices used in the future. LabVIEW can also be used on all computer platforms and has good internet facilities.

Operational equipment

The AR interface has been designed for use with a personal computer (PC) with typical hardware; mouse for input and monitor for output. This eliminates the need to purchase expensive specialised hardware. PC speakers could also be effectively utilised as an output mechanism, but this is outside the scope of this thesis which is to explore visual mechanisms.

Time delay/bandwidth limitations

With regard to remote-control two of the most important issues with regard to the communication link are reliability of the connection and the transmission delay. The transmission time is proportional to the transmission distance and the speed of transmission through the medium. What results during most forms of remote operation is information feedback that can be seconds old.

The standard bandwidth is 56kbps (kilobits per second); however, the use of asymmetric digital subscriber line (ADSL) is becoming more common. ADSL provides a maximum bandwidth of 2Mbps (megabits per second) downstream. For a single monoscopic image of size 640x480 pixels and jpeg format means that the time taken to receive this image would be as follows.

Amount of data = 640x480x8bits

= 2400 kbits

ADSL bandwidth @100% = 2048 kbps

ADSL bandwidth @90% = 1843.2 kbps

Standard bandwidth @100% = 56 kbps

ADSL transmission time @ 100% = 1.2 seconds

ADSL transmissin time @ 90% = 1.3 seconds

Standard transmission time @ 100% = 43 seconds

As can be seen an ADSL would allow a the images to be displayed with a delay of less than 2 seconds but a delay of more than 40 seconds using a standard 56kbps connection makes this not feasible at the current point in time. A standard connection would be much more suited by a 320x240 pixel image.

Amount of data = 320x240x8bits

= 600 kbits

Standard transmission time @ 100% = 10.7 seconds

At this image size the transmission time would be expected to be less than twenty seconds and therefore much more acceptable. If the ARIOM system was implemented in as an online control mechanism the option of either image size would optimise the program. For a ADSL the use of multiple monoscopic views is feasible and therefore would encourage an increase in the allowable perception of the environment and a subsequent increase in operator performance (see ---).

User Manual

This section details the use of the Augmented Reality Interface. This interface allows manipulation of the ABB robot set up in G21 of the Civil and Mechanical Engineering Building. This program is currently incomplete and has limited functionality.

Calibration

Upon opening the program, a dialog box will open asking if you wish to run calibration. It is highly recommended that calibration be run to improve accuracy and usability of the Augmented Reality Interface.



Figure 1

If No is selected, then the values will be set to the default values and the program will enter the Control Interface. If Yes is selected, an interface will appear as below.

Figure 2

This interface requires that the coordinates of the each position under the Workspace section be set to each corner of the grid area. The XY Cursor Position must then be set to the top right corner of the bottom left square. When this is completed, the Done button can then be pressed. If Cancel is pressed at any time, the values will revert to the default values.

After completing the calibration, the Control Interface appears as shown in figure 3.

Directory: Information
Information -> Management and Marketing
Information -> Computer Science 101 October 13, 2016
Information -> Student athlete handbook
Information -> Table of Contents Page
Information -> Policies and Procedures 40 – Eligibility
Information -> Procedures for Implementing and Managing the All-University Policy on Mobile Communication and Information Devices (Mobile cids) Policy University of Kansas Medical Center
Information -> Policies and Procedures 40 – Eligibility
Information -> Cacc director’s council may 24-25 2010 12 noon caesar’s, Atlantic City, nj

Download 6.79 Mb.

Share with your friends: