3D Augmented-Reality [AR] applications are currently being developed in a multitude of locations around the Boeing enterprise, by organizations connected to many different functions and programs. Because of the vast diversity of these, it is impossible to identify a single standardized workflow, process pipeline or software tool complement to accomplish this type of development effort. Rather, this document defines general processes for characterizing and designing projects that develop 3D visual applications.
This standard documents the processes and techniques required to develop 3D Augmented-Reality visualizations for industrial and aerospace applications. In general, the use of a program’s 3D design geometry data as a starting point is assumed, although in some cases, AR applications can be created from purely notional model data.
The processes and workflows described herein represent the best practices in use by the Boeing AR/VR Center of Excellence, as of the first quarter of 2017. These practices are in a continuing state of evolution and refinement, as computing hardware, software, networking and display technologies all continue their respective growth and development.
Virtual Reality [VR]
A technology that replaces a user's view of the real world with a computer-generated image, thus providing an entirely synthetic experience.
Augmented Reality [AR]
A technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view.
3D AR Application
An executable software program that displays holographic 3D geometry and 2D graphics, images or text as augmentations to a user’s view of the real world.
A 3D representation of any environment that an AR or VR user can be virtually transported to. Environments can be natural or synthetic, at any scale from microscopic to astronomical.
A 3D geometry model of an object that can be placed, animated, assembled or deployed in an AR application.
A subset of Environment models, Facility models are 3D representations of interior or exterior space used to house industrial production systems.
A 3D geometric model of a product or production system element, produced at authoritative levels of detail and accuracy. CAD models are used to provide model-based definition [MBD] of any product being manufactured
A 3D model of a product or production system element, produced at representational levels of detail and accuracy. Visualization models should convey the overall visual essence and appearance of their subject and be able to function as required by the visual application, in the same manner as the corresponding original CAD model
12.4Visualization Project Resources
The development of visual products is enabled and driven by the acquisition and development of 2D and 3D assets. Effective use of existing project resources plays a crucial part in meeting cost and schedule targets. In addition, a good working awareness of such resources is essential in developing low-risk project plans.
12.4.1Data Repositories for 3D Models
The Boeing Solid Model Library [BSML] has been established to support detail CAD design efforts with a wide variety of standard parts and components:
BSML provides authoritative models at very high levels of detail and accuracy. This is vitally important in the development of product models for manufacturing. However, for most AR applications, much lower part resolutions are more appropriate. Discussions are currently underway to expand BSML, to include 3D models that are suitable for incorporation in AR applications.
To be truly useful, a 3D model repository should incorporate the following functionality:
When required in an AR application, realistic or authoritative terrain data can be obtained from the Boeing Geospatial Intelligence Repository [BGIR]:
BGIR was established to support Boeing’s diverse simulation community, and offers terrain elevation and corresponding satellite image data for the majority [if not the entirety] of the Earth’s surface, often at varying levels of resolution. BGIR is provided through a community-of-practice called the Synthetic Natural Environment Working Group [SNEWG]:
BGIR is currently administered by Robert Kramer, whose Insite link is as follows:
The use of image-based textures in AR applications significantly leverages runtime performance. Texture is often used to replace complex 3D geometry to greatly reduce model size. In addition, textures can incorporate bump, lighting and shadow effects that greatly enhance realism.
At this time, there is no central, enterprise-level repository for texture images. This data is generally stored or otherwise identified with the 3D geometry model that incorporates the images.
12.4.4Script/Macro Code Repositories
Much of the underlying code in an AR application is re-usable. For this reason, it is advantageous to maintain common repositories that can be accessed by developers as they collaborate. Boeing has been a leader in the use of GIT
12.4.5Incorporation of Purchased Elements
Quite often, 3D models can be purchased from commercial vendors on a cost-effective basis. However, the terms and conditions associated with these purchased models must be observed to avoid legal exposure.
12.5AR Visualization Project Development Processes
The early phases of project development are centered upon understanding, characterizing and allocating the Customer’s requirements. Usually the value proposition associated with creating the visualization needs to be clearly communicated to the Customer before funding can be approved. This makes initial discussions crucial.
12.5.1Current Product Life Cycle Status
The product life cycle status of a Customer’s program will greatly inform the design of any visualization project developed for them. Life cycle status determines the top-level objectives for the visualization. In general, this falls into two categories:
Providing an advance look at anticipated future development – as a way of securing funding for the Customer’s program or project
Providing a current or retrospective look at ongoing work – as a way of reporting status and major accomplishments, thereby justifying previous funding and possibly securing follow-on funding.
12.5.2Customer Themes & Messages
These are the principal ‘selling points’ that need to be communicated by the visualization. For a development program, they are usually related directly to its Most Important Requirements (MIR’s). Examples of these might be:
Compatibility with other fielded systems or infrastructure
Novel or innovative concept of operations (ConOps)
Use cases provide more specific direction as to what functionality needs to be highlighted in a visualization. For example, if a product or system needs only to be browsed in a static state, the visualization effort will be significantly less than for complex animations depicting assembly, articulation or deployment of components of that same product.
Use cases also drive the design of the model curation
12.5.4Assessment of Available Source Data
The scope and quality of the data available from the customer has a significant effect on the design of a visualization project. Examples of source data include:
As such, it also drives the project’s schedule and budgetary estimates. However, during initial project discussions it may not be possible to fully assess what is available, and what would need to be created or developed using project funding.
Because of this, it is often useful to plan for an initial project phase to more fully investigate and assess the Customer’s source data before committing to budget and schedule for the entire project.
12.5.5Milestones and Deliverables
All engineering effort is executed to support program or project milestones. Completion of any interim or final deliverable products needs to conform to these milestones. This potentially constrains the scope and approach of a development effort as much, or even more than the budget available.
12.5.6Desired Level of Interactivity
User-interactivity requirements directly inform AR application design, and consequently, development cost and schedule. All AR applications by definition embody a baseline functionality:
Superposition of CG imagery over real environment
User-controlled view navigation
The potential for added AR functionality is only beginning to be explored. Some additional features that are relevant to industrial use are:
Guided/scripted view waypoints, providing a ‘guided tour’
Interactive user positioning/manipulation/assembly/installation of scene objects
Interactive user tele-operation of scene objects, or their mechanisms or components
User access to metadata that is associated with scene objects
Incorporating additional AR functionality involves software code and/or script development, the nature of which relates to the GIDE being used for application development
12.6Visualization Project Management
To minimize cost and schedule risk and maximize the value provided to a customer, conventional project management techniques can be readily tailored to suit AR development.
Visualization customers are often unclear on what exactly they want in a finished project. Once the Customer sees early versions of a project in development, it becomes easier reviews are useful to stimulate thoughts, ideas and brainstorming, often resulting in additional AR project scope
12.6.2Organization of Supplied Project Data
Issues associated with data history and configuration control can often arise, as Customer-supplied data evolves and is updated. A systematic and consistent approach to managing and storing project data reduces schedule and budgetary risk and facilitates collaborative efforts by multiple team members.
For initial or moderately-sized projects, or one-off proof-of-concept efforts, project data management can be accomplished using a folder structure that reflects the development pipeline. For example, if an AR project was initiated to develop an application to show commercial airplane passenger cabin layout options, the project folder structure might look like this:
This example shows a simple, server-hosted project folder structure for a development effort that uses CAD source models in the CATIA format to develop applications in the Unity Game Engine/Integrated Development Environment. The numerical folder prefixes reflect the sequence of the development pipeline and processes.
An agreed-upon project data structure, when hosted on a mutually-accessible server, provides for team collaboration and easy access to project status. If appropriately descriptive file-naming conventions are applied, this is an adequate project data-management solution.
However, some projects may be very large and/or complex, particularly as they are scaled up to support actual use in a production program. For these types of efforts, it may be appropriate to use one of the commercially-available production-control software tools that are used by the motion-picture and video game industries, such as Shotgun:
These tools generally provide review, asset management and production tracking functionality
12.6.3Status Tracking of 3D Model Decimation/Optimization
For an AR project, the model decimation and optimization process may be complex and involve several steps, using different software tools and methods. In addition, the process may vary for different individual models, especially if they have been acquired from diverse sources.