Changing the Game with Mixed Reality for Construction Planning and Operations

Construction teams are constantly looking for ways to improve safety and productivity on-site. The immersive visualization of Bentley’s SYNCHRO XR application with HoloLens 2 provides a quick and easy way to evaluate the project plan against the actual constraints of a job site. Bentley Systems recently showcased the new capabilities of Microsoft’s HoloLens 2 with a preview application built into the SYNCHRO digital construction environment. Using head-mounted devices, HoloLens 2 allows you to see your surroundings while the holograms also appear to be part of your physical job site.

More intuitive and user-friendly

The interaction patterns for HoloLens 2 have been completely redesigned. The hand tracking has been vastly improved to support more natural gestures that allow users to reach out and touch holograms. Users can grab objects and inspect them in their hand, twisting holograms and scaling them with wrist rotations and with two-handed stretching gestures.

Bentley’s SYNCHRO XR preview application takes full advantage of these new opportunities and makes working with digital twins intuitive. Users can now work collaboratively in a shared holographic mixed reality, using natural hand gestures to plan construction logistics, slide back and forth through a construction timeline, and select and inspect elements. When designing the application, the user research and user experience teams focused on making the interactions easy to understand and favoured tactile interactions that take place within arm’s reach. With the application, users can scale, zoom and rotate models with easy-to-learn hand gestures. Using both hands, users can pick up objects as if they were right in front of them, rotate them freely in space, and stretch them to zoom in. The ‘space-time’ slider allows users to slide backward and forward through construction sequencing with their hand.

Multi-user experience

The interactions suggest that the use of 2D screens and mouse input could be replaced by interactive spatial interactions with 3D and 4D models. This feature becomes especially important when working on holograms using shared experience, which enables an unlimited number of users to participate and interact with a holographic model data. In the shared experience mode, users can even pass holographic 3D objects to each other. Users in remote locations can also participate in the shared experience, fundamentally changing how users collaborate with digital models.

Mixed reality

HoloLens is considered a form of augmented reality, allowing you to see both your surroundings and the holograms, which appear to be part of your physical space. The digital models are therefore aligned with physical space. This situation is different from virtual reality, where the user is fully immersed in a virtual world and detached from physical reality. Augmented reality reinforces the concept of a digital twin that is linked with physical reality. In effect, it changes the very definition and nature of what we consider reality.

Other features of the HoloLens 2 are its ergonomics and increased field of view (FOV), which play a huge role in making the device more comfortable to wear while delivering a more immersive experience. The weight of the device is evenly distributed on the head and can be worn comfortably for 45 minutes. The increased FOV (two times larger) makes the holograms seem like they are part of physical space. With the SYNCHRO XR application, the result is an immersive visualization.

Technical details

SYNCHRO XR works by connecting with a construction digital twin database running in the cloud. At the core of that database is SYNCHRO data, which includes tasks, resources, 3D meshes, user fields, documents, issues, users, companies and statuses. From this database, the application can generate 4D models and the ability to move back and forth through the construction timeline. These features allow construction teams to visualize their planned approach to the job using the holographic computing power of the HoloLens – an advanced mixed reality device. Once the data is loaded to HoloLens, users can align digital assets and information with the physical world and even make updates to the data and synchronize these changes back to the server.

Benefits and potential

The construction industry is looking for ways to improve safety and productivity on-site. The HoloLens provides a natural interface for understanding 4D construction information. Not only is the construction sequencing freed from the constraints of a 2D screen, but the hologram can also be aligned with physical reality. This capability provides a quick and easy way of evaluating the planned situation against the actual constraints of a job site. Our users see this new way of evaluating their construction approach as a way of identifying safety concerns, as well as improving planning and project management by embracing radical transparency.

Once you introduce reality meshes into the equation, other possibilities are available. For example, a reality scan of as-built conditions could be loaded into the HoloLens, thereby providing X-ray vision. Users can see through walls and ceilings to what was actually built by superimposing the scan data with the physical space. From a maintenance and renovation perspective, this feature would be game-changing.

The future of construction

The objective of SYNCHRO XR is to improve the accessibility and usability of digital twin data. The application is a live connection to a construction planning database, which opens the door for new human-machine interfaces. Put simply, the objective is to uncover new ways of working and designing with data. It explains what it means to have digital data become part of our physical space – not as screens, but as 3D objects in space and time.

SYNCHRO XR is more than just an engaging experience. It synchronizes live data with a cloud server that hosts the construction project data, allowing users to interact with the data in new ways. This ability to bring 4D digital construction models into mixed reality is a first in the industry and promises to change how the entire supply chain interacts with digital twins.

To learn more about SYNCHRO and SYNCHRO XR for HoloLens, see here. 

To learn more about the work of Greg Demchak and his team at Bentley Systems, visit www.bentley.com.