Block One.Key concepts: What is mixed reality remote assist?
|
Introduction.
Within manufacturing and service based industries, there has been growing use by organisations of industrial Mixed Reality (MR) tools for the remote support of their workforce, while at the same time also using these mixed reality tools to provide additional means of Subject Matter Expert (SME) knowledge transfer within the workplace.
Mixed reality remote assist technologies allow remote workers to establish a low-bandwidth, secure video and audio call with an SME when they need help solving a complex problem. The SME, who could be located anywhere in the world, can then see what the remote worker sees, and call upon a range of holographic and interactive resources to support the task in hand.
Industrial mixed reality offers a better way to create and deliver easily consumable task instructions and support by overlaying digital content onto real-world work environments. These digital overlays can include, for example, the use of animated holograms, where the SMEs can drag and drop three-dimensional holograms into the worker’s field of view. These holograms can be animated to show the multi-step processes required for specific repairs or tasks, improving comprehension. Additionally, image capture, annotation techniques, and the use of supporting assets can be used to assist task understanding. SMEs can take photos of the worker’s field of view; these photos can then be annotated on and appear in the remote worker’s field of view as holograms, SMEs can also annotate items live in the worker’s field of view. Microsoft Office files or PDF documents, for example, can be brought into the session, which can also appear in the worker’s real-world environment.
Therefore, it may be argued that the mixed reality remote assist technologies currently being used in industry, are equally well suited to supporting remote education not only in the workplace but also across several other educational sectors. Providing an additional ‘interface’ for educators to use, offering alternate digital futures of participation for both students and teachers. Advantages using this approach in education can be highlighted, albeit simplistically, by substituting the term ‘SME’ in this introduction, with that of ‘teacher’, and ‘worker’, with that of ‘student’!
Mixed reality remote assist technologies allow remote workers to establish a low-bandwidth, secure video and audio call with an SME when they need help solving a complex problem. The SME, who could be located anywhere in the world, can then see what the remote worker sees, and call upon a range of holographic and interactive resources to support the task in hand.
Industrial mixed reality offers a better way to create and deliver easily consumable task instructions and support by overlaying digital content onto real-world work environments. These digital overlays can include, for example, the use of animated holograms, where the SMEs can drag and drop three-dimensional holograms into the worker’s field of view. These holograms can be animated to show the multi-step processes required for specific repairs or tasks, improving comprehension. Additionally, image capture, annotation techniques, and the use of supporting assets can be used to assist task understanding. SMEs can take photos of the worker’s field of view; these photos can then be annotated on and appear in the remote worker’s field of view as holograms, SMEs can also annotate items live in the worker’s field of view. Microsoft Office files or PDF documents, for example, can be brought into the session, which can also appear in the worker’s real-world environment.
Therefore, it may be argued that the mixed reality remote assist technologies currently being used in industry, are equally well suited to supporting remote education not only in the workplace but also across several other educational sectors. Providing an additional ‘interface’ for educators to use, offering alternate digital futures of participation for both students and teachers. Advantages using this approach in education can be highlighted, albeit simplistically, by substituting the term ‘SME’ in this introduction, with that of ‘teacher’, and ‘worker’, with that of ‘student’!
Activity.
For the purpose of the OER we are going to consider mixed reality remote assist technologies based around the use of some existing Microsoft products, including HoloLens 2, Teams, and Dynamics 365 Remote Assist. We will start by watching three short videos which introduce the technologies, and provide a small insight what they can do.
|
Video 1Introducing Microsoft HoloLens 2Microsoft proposes that 'HoloLens 2 offers the most comfortable and immersive mixed reality experience available, coupled with industry-leading solutions that deliver value, enhanced by the reliability, security, and scalability of cloud and AI services from Microsoft'.
|
|
Video 2What can HoloLens 2 do?How companies across differing industries use #HoloLens2 to achieve more.
|
|
Video 3Introducing Dynamics 365 Remote Assist for HoloLens 2 and mobile devicesAn example showing employees who are empowered to work together from anywhere with Dynamics 365 Remote Assist on HoloLens and mobile devices.
|
Having watched the videos, here's a couple of questions for you to consider:
- Do you currently consider the implication of mixed reality remote assist in education achievable'? Why or why not?
- How do we incorporate new technologies like these into the educational system 'without disrupting the system, and without overloading students and teachers'?
Let's further examine this technology in action.
Here we're going to look at a couple of examples of how this technology is currently being used.
(1) Imperial College Healthcare NHS Trust
Kinross, J. (2020). Imperial College Healthcare Trust uses mixed reality in the fight against COVID-19. Available online here.
Kinross, J. (2020). Imperial College Healthcare Trust uses mixed reality in the fight against COVID-19. Available online here.
Dr Kinross outlines the nature of the work underway at Imperial College Healthcare trust to use Microsoft Hololens 2 and Dynamics 365 Remote Assist, to provide hands-free telemedicine capabilities. Holding Teams video calls with colleagues and experts anywhere in the world, while interacting with the caller and patient at the same time, with medical notes and other critical documents placed in the wearer's field of view during the call.
(2) Mercedes-Benz USA
Pietsch, J. (2020). Transforming the service workforce with Dynamics 365 Remote Assist and Hololens 2. Available online here.
Pietsch, J. (2020). Transforming the service workforce with Dynamics 365 Remote Assist and Hololens 2. Available online here.
Mercedes-Benz USA chooses Microsoft HoloLens 2 and Dynamics 365 Remote Assist technologies to improve service technician efficiency, reduce time to problem resolution, reducing the cost and environmental impact of service-related travel and using the opportunity to coach and train technicians for the future.
Does MR remote assist have a place in the 'remote' classroom?
Mixed reality clearly offers unique and unparalleled ways to engage and stimulate students, both in the traditional classroom setting, and during remote learning. Students, when immersed in what they are learning are motivated to understand the subject thoroughly, often requiring less cognitive load to process the information (Andersen et al., 2016), with Merchant at al’s (2014) meta-analysis also indicating a high learning efficiency through the use of immersive reality in Higher Education. Weyhe, Dirk et al., (2018), propose this can be explained through the ‘constructivist view of learning, which claims that human experience and learning are subject to certain construction processes’, learning influenced by sensory, neuronal, cognitive, and social processes.
The more complex learning opportunities provided through mixed reality remote assist use, lead to greater opportunities for the teacher to adapt traditional lesson design and development, in order to facilitate or ‘choreograph’ ‘flows of experiences’. Promoting a more parallel relationship between teacher and student, with the student learning ‘from their own ‘experience’ while adding their values, beliefs, patterns and previous experiences to the new information’ (Neubert et al., 2001).
However, before there is any consideration of what technology to use, or how to use it, there needs to be a clear and fundamental understanding of exactly what the needs are. Try to envisage what this might look like, and its implications for students in your particular educational setting, and how this technology may help enhance learning.
The more complex learning opportunities provided through mixed reality remote assist use, lead to greater opportunities for the teacher to adapt traditional lesson design and development, in order to facilitate or ‘choreograph’ ‘flows of experiences’. Promoting a more parallel relationship between teacher and student, with the student learning ‘from their own ‘experience’ while adding their values, beliefs, patterns and previous experiences to the new information’ (Neubert et al., 2001).
However, before there is any consideration of what technology to use, or how to use it, there needs to be a clear and fundamental understanding of exactly what the needs are. Try to envisage what this might look like, and its implications for students in your particular educational setting, and how this technology may help enhance learning.
Learning should never be led by technology. Teachers teach, technology assists.
Questions to contemplate:
Spend a few minutes on each of the following questions, to consolidate your current understanding of mixed reality remote assist technology as it is currently used in industry, as well as considering the potential opportunities it presents for use in support of remote learning in education.
- What purpose and function will it fulfil?
- How will the technology be used and managed?
- How will these devices fit into your existing information and communications technology resource framework?
- What training will staff, and students require to successfully use mixed reality remote assist?
In the next block, we are going to start thinking more critically about potential mixed reality remote assist projects as they make the move from industry, and into education.
References.
Andersen et al., (2016). Cognitive Load in Mastoidectomy Skills Training: Virtual Reality Simulation and Traditional Dissection Compared. Journal of Surgical Education, 73(1), pp.45–50. Available at: DOI:10.1016/j.surg.2015.09.010
Merchant et al., (2014). Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70(C), pp.29–40. Available at: https://doi.org/10.1016/j.compedu.2013.07.033
Neubert S, Reich K., (2001). Voß R. Lernen als konstruktiver Prozess. In: Hug T, editor. Einführung in Das Wissenschaftliche Arbeiten Wie kommt Wissenschaft zu Wissen? Vol 1. Baltmannsweiler: Schneider Verlag Hohengehren p. 253–65. [Translated]
Weyhe, Dirk et al., (2018). Immersive Anatomy Atlas-Empirical Study Investigating the Usability of a Virtual Reality Environment as a Learning Tool for Anatomy. Frontiers in Surgery, 5, p.73. Available at: DOI: 10.3389/fsurg.2018.00073