Augmented Reality Control of the Telerobot 2003


Module 8: Interaction State Machine



Download 6.79 Mb.
Page16/17
Date26.04.2018
Size6.79 Mb.
#46776
1   ...   9   10   11   12   13   14   15   16   17

Module 8: Interaction State Machine

8.1 Specification


The Interaction State Machine is the heart of the user interaction-response within the AR interface. This module will be responsible for determining the relationship between user input and the outputted graphics. Input into the state machine includes state input from the graphical user interface (GUI) and calibration parameters from the Image Calibration Module. It also outputs the state of the models to the Drawing Module so that they can be drawn and their geometry, position and orientation to the Robot Control Module so that the robot can manipulate them.

8.2 Design and implementation


The design of the state machine was dependent on the desired output from a combination of an input with the current state of a model. In order to achieve this a LabView control structure describing the state of a model and the input to the state machine had to be planned.

8.2.1 State

The chosen representation for the state was a series of binary substates:



error/

Rectangular Prism



regular/

Highlighted



not selected/

Selected


mouse up/

Mouse Down



The meaning of each binary substate is explained in the table below.

Sub state

Description

Error (e)

Indicates that the state of the model is not valid

Rectangular Prism (RP)

Indicates that the state of the model is valid

Regular (r)

Indicates that the model is in a ‘static state’ and is not currently active (see figure –a)

Highlighted (H)

Indicates that the model is in a ‘static state’ but has changed colour to point out that a mouse click will change it to a manipulable state.

not selected (n)

Indicates that the model has not yet been selected and therefore remains in either the regular or Highlighted state

Selected (S)

Indicates that the model has been selected for manipulation

mouse up (mu)

Indicates that the model, although selected, is not currently being altered

Mouse Down (MD)

Indicates that the model is selected and is currently being altered in either position, orientation or dimension

The state control also includes an integer from zero to seven which specifies the action that is occurring if in a Selected and Mouse Down mode. The complete state definition which makes the action field valid is RP_H_S_MD . The meaning of each of these actions is as described in the table below.

Action number

Description

0

No action

1

Width is being altered

2

Length is being altered

3

Height is being altered

4

Orientation angle is being altered

5

Position in the x and y direction is being altered

6

Position in the x direction is being altered

7

Position in the y direction is being altered

8.2.2 Input

The input to the state machine is represented as indicated in the following table.



Element

Representation

Description

Mouse state

Boolean

Is the mouse down over an interactive point

Mouse position

Integer from 0 to 3

What is the mouse over

  • Nothing (0)

  • Rectangular prism (1)

  • Rectangular prism control point (2)

  • Another rectangular prism (3)

Control point

Integer from 0 to 7

What control point is the mouse over. These values correspond to the action type of the model state mentioned previously

As can be seen in the figure below, the state machine itself operates in an expected fashion. More specifically an input and current state are input into the machine and a new state is outputted.



Figure 52

The state transitions resulting from each unique pair of inputs and states were determined by considering the desired outcomes. These state transitions are summarized in the table below. All blacked out rows correspond to transitions that are not possible and therefore are logged as errors.



Input

 

Current state

 

Next state

 

 

mu

empty

 

RP

r

n

mu

n

 

RP

r

n

mu

n

mu

RP

 

RP

r

n

mu

n

 

RP

H

n

mu

n

mu

RP_CP

 

RP

r

n

mu

n

 

RP

r

n

mu

n

mu

otherRP

 

RP

r

n

mu

n

 

RP

r

n

mu

n

MD

empty

 

RP

r

n

mu

n

 

RP

r

n

mu

n

MD

RP

 

RP

r

n

mu

n

 

RP

H

n

mu

n

MD

RP_CP

 

RP

r

n

mu

n

 

RP

r

n

mu

n

MD

otherRP

 

RP

r

n

mu

n

 

RP

r

n

mu

n

 

 

 

 

 

 

 

 

 

 

 

 

 

 

mu

empty

 

RP

r

n

MD

n

 

RP

r

n

MD

n

mu

RP

 

RP

r

n

MD

n

 

RP

r

n

MD

n

mu

RP_CP

 

RP

r

n

MD

n

 

RP

r

n

MD

n

mu

otherRP

 

RP

r

n

MD

n

 

RP

r

n

MD

n

MD

empty

 

RP

r

n

MD

n

 

RP

r

n

MD

n

MD

RP

 

RP

r

n

MD

n

 

RP

r

n

MD

n

MD

RP_CP

 

RP

r

n

MD

n

 

RP

r

n

MD

n

MD

otherRP

 

RP

r

n

MD

n

 

RP

r

n

MD

n

 

 

 

 

 

 

 

 

 

 

 

 

 

 

mu

empty

 

RP

H

n

mu

n

 

RP

r

n

mu

n

mu

RP

 

RP

H

n

mu

n

 

RP

H

n

mu

n

mu

RP_CP

 

RP

H

n

mu

n

 

RP

H

n

mu

n

mu

otherRP

 

RP

H

n

mu

n

 

RP

r

n

mu

n

MD

empty

 

RP

H

n

mu

n

 

RP

r

n

mu

n

MD

RP

 

RP

H

n

mu

n

 

RP

H

S

mu

n

MD

RP_CP

 

RP

H

n

mu

n

 

RP

H

n

mu

n

MD

otherRP

 

RP

H

n

mu

n

 

RP

r

n

mu

n

 

 

 

 

 

 

 

 

 

 

 

 

 

 

mu

empty

 

RP

H

n

MD

n

 

RP

H

n

MD

n

mu

RP

 

RP

H

n

MD

n

 

RP

H

n

MD

n

mu

RP_CP

 

RP

H

n

MD

n

 

RP

H

n

MD

n

mu

otherRP

 

RP

H

n

MD

n

 

RP

H

n

MD

n

MD

empty

 

RP

H

n

MD

n

 

RP

H

n

MD

n

MD

RP

 

RP

H

n

MD

n

 

RP

H

n

MD

n

MD

RP_CP

 

RP

H

n

MD

n

 

RP

H

n

MD

n

MD

otherRP

 

RP

H

n

MD

n

 

RP

H

n

MD

n

 

 

 

 

 

 

 

 

 

 

 

 

 

 

mu

empty

 

RP

H

S

mu

0-7

 

RP

H

S

mu

n

mu

RP

 

RP

H

S

mu

0-7

 

RP

H

S

mu

n

mu

RP_CP

 

RP

H

S

mu

0-7

 

RP

H

S

mu

1-7

mu

otherRP

 

RP

H

S

mu

0-7

 

RP

H

S

mu

n

MD

empty

 

RP

H

S

mu

0-7

 

RP

H

S

mu

n

MD

RP

 

RP

H

S

mu

0-7

 

RP

H

S

mu

n

MD

RP_CP

 

RP

H

S

mu

0-7

 

RP

H

S

mu

1-7

MD

otherRP

 

RP

H

S

mu

0-7

 

RP

r

n

mu

n

 

 

 

 

 

 

 

 

 

 

 

 

 

 

mu

empty

 

RP

H

S

MD

1-7

 

RP

H

S

mu

n

mu

RP

 

RP

H

S

MD

1-7

 

RP

H

S

mu

n

mu

RP_CP

 

RP

H

S

MD

1-7

 

RP

H

S

MD

1-7

mu

otherRP

 

RP

H

S

MD

1-7

 

RP

H

S

mu

n

MD

empty

 

RP

H

S

MD

1-7

 

RP

H

S

MD

1-7

MD

RP

 

RP

H

S

MD

1-7

 

RP

H

S

MD

1-7

MD

RP_CP

 

RP

H

S

MD

1-7

 

RP

H

S

MD

1-7

MD

otherRP

 

RP

H

S

MD

1-7

 

RP

H

S

mu

n

 

 

 

 

 

 

 

 

 

 

 

 

 

 

8.3 Testing


The state machine was tested by the running of the ‘state transition.vi’ and checking off the expected results in the table above.

Module 9: Event Logging

9.1 Initial Specification


Any change in the system should be logged so that if errors occur they can be reproduced to trace the source of the error and fix them, also so that user interactions with the system can be observed in an empirical way.

9.2 Design and Implementation


9.2.1 Basic Operation

The operations in the specification can all be performed using a simple circular buffer, operating as a stack. In a stack the first element on is the last element off, conceptually one can consider a stack of plates, we can either add a plate to the top of the pile, or remove a plate from the top of the pile. The difference is that we require some form of wrap around. These operations can all be performed in labVIEW using two variables, the first is an array of fixed length that allows us to store elements, and the second is an index into that array that stores the next location to write to.

Since we wanted to store many variables of different types we decided that the one unifying variable type was the string, the array of elements was therefore to become and array of strings. The index is simply an index into an array and so should be of type integer.

The basic operations of the data logger, as described by the specification, is to read and write elements (Strings), thus we also need to define a “mode” that the buffer is to operate in. This mode is inputted into the buffer as a string, there also needs to be a “data in” input and a “data out” output, also of type string. The modes used are



  1. “read”: This is when we wish to pull an element off the stack, the element appears at the output “data out” and is removed irretrievably.

  2. “write”: This is when we want to add an element to the stack, the element added is whatever appears at the “data in” terminal.

  3. “initialise”: This is necessary before performing any “read” and “write” operations, as it initialises the array with a set number of elements and also sets the internal index variable to refer to the start of the buffer. This mode can also be used as a reset, when we wish to clear the buffer for any reason.

If any mode, other than those above, appears at the “mode” input then an error appears at the output and no operation is performed.

The buffer is of limited size, and is internally set to hold 200 elements.



9.2.2 Wrapping the Buffer

For this buffer to be useful it needs to behave in a cyclic manner, so that when the array becomes full the index wraps around and references the oldest element, this is done using the modulus, or modulo function, shortened to ‘mod’.



If we give the variables that we have been referring to names then we can define the wrapping formulas. Call the array “a” and the index into the array “index”, then to step back in the array:

And to step forward:





9.2.3 Additional Functionality

Now that we have the basic functionality of the circular buffer we can build additional functionality around it. We created a shell to encapsulate the circular buffer that acted as an error handler to deal with spelling mistakes on the “mode” input.

We also wanted to be able to access the stack in its entirety without effecting the operation of it. This was to incorporated another mode, a “file dump” mode that outputs whatever is in the stack and writes it to a text file. To do this we added an additional output to the basic circular buffer (BCB) that outputs the array. We also needed access to the internal index of the BCB for reasons that will become apparent.

The array that the BCB now outputs is not useful to us when writing to a file, this is because the first element in the array is not the most recent element. The most recent element is located at the position before (including wrap around) what the now outputted index is referring to. To write to a file in a useful form, we require the most recent element to appear at the beginning of the file. In order to achieve this we have to split the array at the index and shuffle the two split arrays around a little, we then have an array with the most recent string appearing as the first element and older strings appear in chronological order after that. This array is called “stack chorological” and is an output of the shell encapsulating the BCB.

The shell still has the same “data out” and “mode” terminals that behave the same as the BCB does.

The shell has effectively two “data in” terminals; these are labelled “variable name” and “variable value”, and are of type string. These two inputs effectively form the “data in” entry to the BCB with some additional information attached. The format for the string actually stored is:



Variable Name: “name” Value: “val”

Where “name” and “val” are the inputs to the circular buffer. So for example:



Variable Name: 2D prism Array Size Value: 1

9.2.4 To String vis

As the circular buffer required strings inputs we had to write numerous toString vis that converted the information that we were currently storing in .ctl files into a suitable and meaningful string representation.


9.3 Testing


Although this is not an exhaustive test it does indicate the wrap around effect and overwriting of old elements, as well as the ordering of “stack chronological”. To save space and time the size of the stack was reduced to 5 elements, to improve clarity the time stamp added is just abbreviated to , the “Variable Name:” has been abbreviated to “VN” and “Value:” has been shortened to “Val”

Variable Name

Variable Value

Mode

Data Out

Stack Chronological

Expected

Do we care

no

initialise

VN: Do we care Val: no

“”

“”

“”

“”

“”



Yes

First

1

write

VN: First Val: 1

VN: First Val: 1

“”

“”

“”

“”



Yes

Second

2

write

VN: Second Val: 2

VN: Second Val: 2

VN: First Val: 1

“”

“”

“”



Yes

Third

3

write

VN: Third Val: 3

VN: Third Val: 3

VN: Second Val: 2

VN: First Val: 1

“”

“”



Yes

Fourth

4

write

VN: Fourth Val: 4

VN: Fourth Val: 4

VN: Third Val: 3

VN: Second Val: 2

VN: First Val: 1

“”



Yes

Last One

Check

write

VN: Last One Val: Check

VN: Last One Val: Check

VN: Fourth Val: 4

VN: Third Val: 3

VN: Second Val: 2

VN: First Val: 1



Yes

Over write

one

write

VN: Over write Val: one

VN: Over write Val: one

VN: Last One Val: Check

VN: Fourth Val: 4

VN: Third Val: 3

VN: Second Val: 2



Yes

Over write

two

write

VN: Over write Val: two

VN: Over write Val: one

VN: Over write Val: two

VN: Last One Val: Check

VN: Fourth Val: 4

VN: Third Val: 3



Yes


9.4 Future Work


Our file writing implementation is not ideal, the user is always prompted for a location to save the log to. A Boolean button on the main GUI could be added that writes the log to file upon pushing of the button.

9.5 Final Specification


The final specification of this module, while no functionality has been removed from the initial specification, is decidedly different from that initial specification. The added functionality includes error handling, file writing, and the presence of a time stamp on the stored variable. These functional components were added as it was deemed necessary for implementation into the final overall programme.

9.6 Documentation




Figure 53

This vi is the shell that encapsulates the BCB and builds the additional functionality, including error handling, into the BCB.



Inputs

Type

Description

Variable Name

String

The name to attach to the data value, used only in “write” mode

Variable Value

String

The data value as a string to store, used only in “write” mode

Mode

String

The mode that the buffer operates in, can be “read”, “write” or “initialise”.



Outputs

Type

Description

Stack chronological

String array

An ordered array so that the most recent element to be written to the stack is the first element in the array, and older elements appear in chronological order after that

Data out

String

In “write” mode data out contains what is being stored in the stack, for “initialise mode” then data out contains what would be stored on the stack, given Variable Name and Variable Value if the mode was “write” however no write action is performed. If the mode is “read” then data out contains the most recent element of the stack, and that element is then shifted to the bottom of the stack.





Download 6.79 Mb.

Share with your friends:
1   ...   9   10   11   12   13   14   15   16   17




The database is protected by copyright ©ininet.org 2024
send message

    Main page