Will Augmented Reality enhance the way we train and manufacture our next Gen solutions?
Augmented Reality (AR) is a technology that is beginning to find real world use cases from social media filters to surgical procedures and is rapidly gaining enhanced elements which will aid the operators. Although this technology is improving, it is still very much in its infancy.
According to a report published by AR/VR adviser Digi-Capital, the AR market could grow to between $85 billion and $90 billion revenue by 2022, while the VR market might reach to $10 billion to $15 billion in the same timeframe. The largest AR revenue streams could come from eCommerce, hardware, advertising, appstore revenues (non-games and games) and enterprise business models. Similarly Digi-Capital published a study showing that AR/VR startups raised a record of $3.6 billion in the 12 months to Q1 2018, as the investment market transitions to AR from VR.”
Google has Google Glass, and has invested $1.4 billion into Magic Leap, while Facebook paid $2 billion for Oculus. Apple released its augmented reality ARKit to developers and Microsoft has Microsoft Hololens. Today the world's leading companies are pouring significant investment into AR, and AR is slowly gaining a practical use case.
In this article, I am going to briefly explore various examples of how AR is being used and how it may evolve in the future. Unfortunately I cannot cover all aspects of AR in one concise article, but hopefully the feature will provide you with enough data in order to educate you more on this growing tech.
Users of Snapchat are familiar with Snapchat filters, which superimpose special effects over live images such as eyes blinking, or raising eyebrows to a user(s) mouth being open. The snapchat user’s face is mapped using the phone’s camera and it adjusts accordingly based on an algorithm.
Pokemon Go was a runaway success, earning 96% of all AR software sales in 2016, which blended the real world with a virtual one. The game accesses the phone’s GPS and clock and a Pokemon ‘appears’ on the user's device and gamers need to catch the Pokemon!
Tim Cook, Apple’s CEO said that Apple was “high on AR in the long run, and the company continues to invest a lot in this…AR can be huge,” while Mark Zuckerberg of Facebook, and Sundar Pichai, of Google and Satya Nadella from Microsoft all hailed Pokemon as a significant win for AR.
Although the above two examples are of entertainment value, large corporations and various industries are looking at the applicational use of technology and applying it to their sectors.
Valerie Riffaud-Cangelosi, from Epson said in an article by EurekaMagazine, “Deploying AR will enable more efficient processes by enhancing the reality of the user, so they’ll be able, for example, to maintain an engine or a complex electrical board in an intuitive and easy way. They’ll be able to see inside the device and act on the information there and then.”
So What is Augmented Reality?
Augmented Reality (AR) is a super imposed computer-generated image or layer of information on a user's view of the real world, providing a composite view.
AR is not limited to visuals only, and can be enhanced with sounds or touch & haptic feedback in order to give the operator a more immersive experience.
Virtual Reality (VR) is a different technology where similarly a user wears a headset, and experiences an entirely “computer generated environment”, whereas AR uses a real world environment and simply overlays information on top of it.
An example of simple applications of AR can be text-notifications, to more complex visual instructions such as open heart surgery, or an audio based experience using built-in GPS coordinates and a host of other tech.
How Does Augmented Reality Work?
To use AR, the operator needs a display unit. This unit, can be a mobile phone, a tablet, a monitor, specially designed glasses (Google Glass) or a heads-up display (HUD) unit which will then layer the ‘augmented reality’ visual onto the real world environment.
Key components required for Augmented Realty
The following components are typically found in an AR environment:
Sensors and cameras
The use of sensors and cameras on an AR device is simply to provide a real-world view of the surrounding environment and the devices take this information and formulate a digital model to determine an output which is then displayed. Sensors determine the user's location, and depth, while cameras process the surrounding physical environment and objects.
On all AR devices, a miniature projector is essential to turn any surface into an interactive environment. Using the information gathered by the sensors, and the camera, the AR then computes, and projects its data for the user onto the appropriate display.
AR utilizes mini-computers that require processing power, and have an array of components which include a CPU, a GPU, flash memory, Bluetooth/Wifi (connectivity), GPS (location), accelerometers (which measures speed), gyroscopes (measuring or maintaining orientation and angular velocity), magnetometer (a compass) and RAM. Using all this technology, the AR devices are becoming increasingly more efficient in providing a truly immersive experience. One key technological challenge, especially for mobile AR, is miniaturising the components into a more compact, sleek and lightweight format which people can wear throughout the day. AR headsets must also remain cool, which brings additional power and thermal constraints.
Radhika Arora, previously of ON Semiconductor, argued that “For a true mobile AR system, there is still not enough computing power to create stereo 3D augmented reality graphics. For image sensors, getting the form factor more compact will be key as well as improving their performance in varying light conditions.”
According to a Deloitte survey, 88 percent of mid-market companies (firms with annual revenues of between $100 million and $1 billion) are already using some form of Virtual or Augmented Reality as part of their business.
Applicational Use of AR in Manufacturing
Google Glass has been deployed by AGCO in its bid to improve efficiency, quality and safety. Today the company is piloting 100 pairs of Google Glass to assist with quality control and other key assembly processes such as accessing manuals and assembly instructions all managed on an Internet of Things (IoT) infrastructure. According to the publication;
This cutting-edge technology has been very successful, already increasing productivity and improving product quality.
- 30% reduction in inspection time, with the elimination of paperwork and manual uploading
- 25% reduction in production time on low volume, complex assemblies
- 50% learning curve reduction for new hires
In most manufacturing plants, workers and robotics follow a predetermined function, and defined assembly instruction. Ash Eldritch, CEO and co-founder of Vital Enterprises has said that, factory workers follow a set of pdf instructions, which are hard to work with, and may be out of date. His company “takes those instructions and make them glanceable in your field of view at all times, hands-free and voice-controlled.”
Ash also goes on to say that his company, “breaks down the work instructions along with associated technical drawings and even video from the last person who did the procedure and put all that onto the [AR] glasses. That means you can keep your hands on your task and you don’t need to walk over to a work station to check something.”
While complex assembly lines have been using AR for many years now, according to Milan Kocic, a director at Hexagon Manufacturing Intelligence he has described how in the aerospace industry, for example, AR is assisting the operators during the assembly process, by showing the next placement of components, in the right chronological order.
Maintenance & Field Support
For field technicians who travel to job sites to fix equipment, AR is an incredible asset that helps increase the skill and efficiency for remote teams.
For example, as a worker wearing smart glasses examines a system in a remote location, diagnostic information appearing in his field of vision indicates the system is malfunctioning. If the worker can’t fix the problem himself, skilled technicians in another location would be able to transmit detailed digital instructions for repairing the malfunction and, then, walk him through the repair process quickly and efficiently.
In another example written by Harvard, the video below, shows a side-by-side time-lapse comparison of a GE technician wiring a wind turbine’s control box using the company’s current process, and then doing the same task while guided by line-of-sight instructions overlaid on the job by an AR headset. The device improved the worker’s performance by 34% on first use.
Caterpillar has also successfully completed its pilot scheme similar to the example above by GE using AR for maintenance tasks via the use of an app. The app provides a sequential set of instructions which must be followed verbatim and aided by visual cues. This enables novice mechanics the ability to learn the correct maintenance procedures faster and avoid possible critical errors.
Some companies have already started to deploy AR for Quality Assurance (QA). Car manufacturer, Porsche has deployed “Porsche Production 4.0” which utilises robotics, and a host new technologies to aide and improve its effiencies. As part of its program, the company is using AR to obtain precise design information about its components, and with the aide of tablets or other visual platforms, the company can instantly gauge the dimensions accurately by using optic measuring, each component can be fully digitised and AI and other softwares can asses the tolerance levels and if interferences may occur. According to SlashGear’s article, the eventual plan for Porsche, is to link the cameras on the production floor, and the HUDs directly to Porsche’s cloud-based parts database in order to enable a real-time deep analytical engine of its parts and the assembled processes.
Another company using “Merged Reality” is Airbus which has deployed Smart Augmented Reality Tool (SART). In 2011, Airbus launched its AR solution called MiRA/SART “to standardise quality across its global network of production facilities}. SMART utilises digital mockups during production, and allows assembly workers access to 3D models of aircrafts, or the actual components. According to Airbus, “the system is fully integrated into the information system, and optimises working times and quality management and control on the assembly line. The deployment of this solution for inspections of Airbus A380 fuselages has reduced inspection times on system supports from three weeks to three days."
Design & Construction
Builders and designers are taking advantage of Building Information Modelling (BIM) tools, and their hope is to replace traditional handheld devices with AR units. In September 2017, the Soluis Group launched its project with Crossrail’s Liverpool Station for its Augmented Worker System (AWE) pilot scheme. According to the report, the project involved:
“the use of AR to track the progress of installation of pre-fabricated super-structure components using an AR application developed by Bechtel Infrastructure. Construction sites are busy, dirty, dangerous places, and they typically suffer from poor or no data connections which provides a challenge with getting the AR content to the AR display at the point of use. Some AR systems do have local storage capability allowing the user to cache the AR content prior to accessing the site and this can then be accessed via Bluetooth or a wired connection when required. As with all technology implementations cost is always a consideration. Many of the AR solutions available today a prohibitively expensive for widespread use. This limits the practical use of AR systems.”
As mentioned above in the quality assurance, and the maintenance and field support section, AR glasses are empowering workers, who may not have developed the necessary skills. Using AR as a training guide, with step-by-step instructions will empower the workforce and improve on efficiencies.
AR is being used to train nurses, and they use a tablet which has a simulation program whereby they can access past/real life patient scenarios, which allow them to develop the correct coping and daily skills required for the job. Another and extremely useful use of the tech is ‘live streaming of surgeries’ for medical educational purposes coupled with patient monitoring and a pain management skills.
AR systems have a lot of strengths, but capturing and storing valuable employee insights is one that should be noted. A recent US government survey found that the percentage of ‘the line-worker workforce expected to retire within the next five to ten years could approach 50% in some organisations’. Using AR, those skills, or job functions and actions can be be stored in the cloud, and any worker can utilise those skills by following the built-in guides, gleaned from those retired line-workers, thus continuing a line of succession.
According to a study by New Jersey Institute of Technology (NJIT), AR could improve the education system, with primary and secondary schools introducing new ways to teach students. For example, Popar Toys is a mobile app which has interactive collections of books covering topics such as animals, dinosaurs, the human anatomy, interactive world maps, planets and/or sea life. Other notable tools cited in the study which are being used by teachers are Daqri, Chromville and Two Guys and Some iPads.
AR has great potential in assisting educators to teach students, schools, and “several studies published in the last four years have shown that it enhances learning and classroom interactions, helps students maintain positive attitude to learning, enhances collaboration, and motivates students to do better.”
Resource: See NJIT’s Educational Infographic.
At CES 2017, Volkswagen showed its AR featuring a heads-up-display device. The HUD allows the driver to focus on the road, while layering information about the cars speed, the surrounding environment, warning signals and indicator arrows on the windscreen of the car. The next evolution of HUD for auto manufacturers, is the inclusion of the exterior view of traffic conditions, with virtual information, or augmentation.
In an article by Photonics, Mark Boyadjis, an analyst with IHS has said, “Augmented reality is — hands-down — the most exciting new advance in HUD innovation. The biggest opportunity for augmented reality systems is for safety, but imagine a HUD enabling a virtual arrow to be laid upon the road itself as you drive, helping you to navigate more naturally. Other options could be illumination of the road lines or pedestrians at night or in poor visibility conditions.”
Applicational Use of AR in Healthcare
According to a new report by Grand View Research, the global augmented reality & virtual reality in healthcare market is expected to reach USD 5.1 billion by 2025. The market is driven by applications in surgical simulations, diagnostic imaging, patient care management, rehabilitation, and health management. From Dentists, to surgeons, the use of AR, composited with real-time data, acquired from CT or MRI scans can then be superimposed using stereoscopic projections. This allows for the surgeons to access data on complicated surgeries that require precision, which should result in reduced complications and a more manageable trauma.
AR is forecasted to enable an increased collaboration between AR/VR and IoT developers. If Virtual reality (VR) is about visualisation and Internet of Things (IoT) is about connectivity, AR is blending the digital capabilities with the physical world.
AR headsets will become sleeker and could evolve into contact lenses. In some extreme cases it has been envisioned that AR devices could be implanted directly into a user's nervous system, which is extreme to say the least.
Adam Kerin, previously Qualcomm’s senior manager of marketing said in an article in newelectronics, ”AR is still very new, with limited market penetration, but as the technology advances and the form factor decreases, we can expect to see it evolving into a seamless experience that users will interact with daily”
Today, however, augmented reality is already being used and deployed in various industries, and the technology is slowly enhancing the processes of those industries and providing valuable data, and improvised efficiencies.