The role of augmented reality in training and machinery maintenance

Augmented reality (AR) has huge potential to deliver more efficient vessel operations and improve maritime training. Demetris Frosynis and Nikos Anaxagora, part of the University of Frederick Students’ Research for Augmented Reality in the Maritime Industry explain how AR is being used to help tackle operational and training challenges.

One of the biggest challenges that the industry is facing, and this ‘challenge’ will spread, is the familiarisation of staff with new equipment. There is a need for training in the use of this new equipment, especially in machines. This challenge is something that Augmented Reality (AR) can address.

Living through this industrial revolution, the maritime sector has already begun to implement digital solutions. Big data analytics of ships, augmented and virtual reality, cloud technologies, machine learning and cognitive artificial intelligence are some relevant examples of the technologies now implemented by shipping companies.

AR technology is becoming one of the main digital tools of Industry 4.0. Demetris Frosynis and Nikos Anaxagora recently carried out research to investigate the potential use of AR in the maritime industry. The research methodology used was qualitative, based on literature reviews and interviews. These consisted of indepth analysis with twenty-four people from the shipping industry, and from this, a thematic analysis of findings was produced. AR in shipping is currently at a very early stage. Today, AR is used in the navigation of ships, mainly by companies from Scandinavia and as a pilot or prototype model in marine engineering.

AR has great potential to help in the maritime industry. In maintenance, remote assistance, diagnostics, work tasking, work checklists and for training purposes AR can be integrated. Any result obtained through AR equipment can be exported as a report in PDF or video. It has great potential to be of use in all operations. In addition, AR improves situational awareness, for example:
- Onboard ships the crew can easily recognise pipelines or wiring and speci-fy what each one is for.
- Maintenance assistance for machinery with steps and tasks.
- Check-list control and inspections (no need to be typed, the report can be verbally reported).
- Officers can use it for daily tasks (again, no need to be typed, the report can be verbally reported).
-Video recording.
-Video reporting with statistical analysis (shared in real time from ship to land).
- Remote assistance from the offices with IT and Technical Departments.
- Training in advanced technology equipment using pre-recorded instructions.
- Bridge systems transferred and integrated in the AR headset.

One of the biggest advantages of AR is that the system allows all the information to appear on many screens in front of the user’s eyes. For example, when a vessel is in a situation with bad weather and low visibility, the officer on watch relies on his radar to look around the vessel to see what other vessels are doing. With AR, the officer on watch can look outside the window and directly get the information of vessels around him. In this way, the officer can see where the other vessels are and what their speeds and courses are. This increases the officer’s awareness of what is surrounding his vessel.

Another challenge that the shipping industry faces is the familiarisation of staff with new technologically advanced equipment, which is often unknown to operators and to seafarers. There is a need for training for the use of this new equipment, especially the engineers. Deck and engine officers say that ‘equipment is changing all the time’. There are too many manufactur ers. Shipping companies cannot control the supply of equipment available in the market and in addition, changing rules and regulations coming from the IMO are mandatory. Furthermore, new manufacturing companies appear in the market and supply new equipment. Competency with this equipment must be internalised and familiarised with the crew onboard. This challenge is something that AR can address.

The concern that seafarers may be overwhelmed by digitalisation is a fact, but the industry’s real needs are digital maritime services that will provide additional motivation which is necessary and easily assimilated into onboard training. The most important thing in technology is that technology, at the end of the day, is not being used. A lot of software and hardware exists, companies invest in technology, but ultimately people do not use it. Technology providers need to revise, see the usability of all these tools and understand how easy or difficult it is to use them. Users need good training before they can make use of any technological tool.

The shipping industry has a traditional, more conservative approach to the operation of ships. New technologies have always been regarded with scepticism, and training is more attractive to them when it follows a ‘hands on’ method of training.

On one hand, shipowners are financially orientated and driven by cost, and every new investment is made according only to finance and the requirements of current legislation. AR technology is very new and very expensive. On the other hand, there is a great opportunity here. AR technology can solve other, more serious problems. The research showed that shipowners are looking to reduce the cost of training on new equipment from providers and manufacturers for shipboard operations. There is a big gap in this. Engineers need to have a tool that enables training on new equipment, controls the maintenance and has remote assistance with the provider on land. This tool can be the AR headset. AR technology can guide them and deliver the training as the engineer looks at the equipment by adding the digital information, completing the checklist and any inspection. Few companies are using it, but the demand for this solution based on the research will be very high. As AR technology is becoming increasingly well-known, there will be more commercial solutions that will soon be eagerly adopted.

Eventually, the shipping providers will follow. They are currently undertaking research in order to invest in AR technology for maritime navigation systems. An AR headset having maps, radar, alarms, warnings and interacting with the real view out of the bridge window will be a fantastic option regarding the improvement in safety and situational awareness. AR relies on the input of many navigation-al systems that will create a new generation of bridge development. AR is not just a PC or virtual reality content; AR combines natural with digital experience along with the interaction between the objects of both worlds made possible in real time. Perhaps no other technology has the potential to revolutionise so many tasks. We are, with AR, approaching a totally new era.

AR can transform the shipping industry, and there are already a few obvious applications for it in training with the creation of real-world videos and modules. Applications, too, can be seen in maintenance by providing engine diagnostic information and remote support to engineers at sea, and in operations where AR technology can provide improved situational awareness and decision-making support.






  LMB-BML 2007 Webmaster & designer: Cmdt. André Jehaes - email