Augmenting Human Operators for the Era of Automated Industry – A/HOPE/AI

This project aims at defining and demonstrating the technical and business feasibility of a set of novel Mobile Human-Machine Interaction (MHMI) services for supporting the ongoing digitalization and automation efforts in manufacturing.

The idea is to identify the core elements of a novel platform enabling the transmission, cloud processing and visualization of IoT data that is critical to manufacturing operations.  The applications enabled by the envisioned platform will allow human operators to understand the states, behaviors and intentions of autonomous machines while they perform specific tasks, will enable safe remote operations and the possibility of human supervision of other humans or machines.

The proposed approach is enabled by the recent advancements in computer vision and Artificial Intelligence (AI), by the secure and cost effective communication capabilities of upcoming 5G networks and by the commoditization of high performing hardware and software technologies that are finally enabling rich and immersive real-time data visualization modalities in VR/MR/AR. Based on the combined expertise in: (1) advanced VR and AR eye-tracking solutions by Tobii, (2) the Ericsson factory for smart manufacturing including a novel cloud-based AR solution, and (3) KTH’s amedia delivery solutions for 5G networks and human-centric service design and assessment, the project will focus on identifying and defining the technology components of a novel platform for enabling human-centric services for Industry 4.0.

The tackles one of the crucial aspects associated with increased automation in the process industry, i.e. creating novel enablers for supporting the human transition from active participation the process execution to process supervision. Different industrial processes will require different degrees of human involvement. Our ambition is to define the core components of a novel platform for enabling industrial applications with a wide range of requirements, ranging from human-to-human to human-to-AI interfaces and data visualization capabilities, while supporting both on-site and remote operations. The proposed Human-to-AI interface is a novel concept allowing human operators to visualize, in real-time, the strategic decisions taken by the AI modules running the automation policies for specific processes and equipment units.

This will enable human operators to achieve an effective supervision of the production cycles, which in turn will lead to increased production and safety for Swedish manufacturers. Novel forms of human-to-human visualizations, including information from eye-trackers embedded in glasses or VR/MR/AR displays, will also bring the additional benefits of supporting continuous training for new operators and effective transfer of knowledge from the more experienced workers, reducing the “skill gap” in the Swedish workforce and introducing novel “softer” work opportunities for experienced workers in the increasingly aging Swedish population. The platform envisioned in this project will also have positive effects across the Swedish ecosystem of SMEs currently operating in the area of industrial software solutions. By opening an SDKs towards the core platform functionalities, different software houses will be able to rapidly develop innovative VR/MR/AR applications and to tailor their services towards various manufacturers.

The project will feature a set of technical explorations, including:

• Definition of core components needed by a novel platform for supporting advanced real-time visualizations of industrial IoT data (Digital Twin).
• Novel VR/MR/AR digital-to-physical interaction modalities that are specific for
  human-centric industrial applications.
• Innovative user input solutions exploiting information from connected eye-trackers.
• Demonstrators showcasing novel services in respect to the (a) Human-to-Human, (b) Human-to-Machine and (c) Human-to-AI communication needs of Industry 4.0.
• A novel experimental methodology to evaluate service design and performances for human-centric industrial applications.
• The establishment of an advanced “sand box” for the definition and evaluation of advanced industrial services in Ericsson’s manufacturing facility in Kista.

For what concerns the business dimension, the project results will include:

• an exploration of the service ecosystem novel business models enabled by the
  proposed platform and its enabled human-centric industrial services.
• An evaluation of increased production efficiency for selected use cases.

The establishment of a consortium of manufacturing companies that will further
refine the proposed solutions into novel and scalable products and services.

Project leader
Konrad Tollmar
[email protected]

Participating organisations
→ KTH
→ Ericsson
→ Tobii