Pick to HoloLens; An application for training of operator in MR and/or VR

Virtual Manufacturing

Virtual Manufacturing is a supplier of lean-based production development services. We believe in and are willing to take on the challenges of combining technology, methods and hard work around production development in order to achieve results. This is carried out both in the virtual and physical world, before and after the start of production. In projects, on a daily basis, we fulfil our own and our clients' dreams. We know how to create value together with our clients. Virtual Manufacturing shall be the natural, most innovative partner in modern production development. Competence, quality and long term sustainable commitment are our guiding principles. By using new approaches and perspectives we help leading companies in the manufacturing industry achieve "operational excellence" faster. 


Previous thesis work

A thesis work to develop the pick to HoloLens concept have been carried out during the autumn of 2016. The outcome of the work is an app that can guide an operator to the correct picking position in a kitting station. The app guides the operator to the correct part and at same times as the amount of parts to be picked is visual. When that part or parts are picked the operator gets information where to put the part/parts. There are three methods to get the app to move to the next part. Either the operator uses voice saint “yes, hello, next”, or using “air tap” which means that the operator is tapping his/hers fingers in front of the HoloLens, the last alternative is to use a cord less remote-control. The app is currently designed for a specific experiment performed during the thesis.

In the current version there is no connection to an ERP system and all pick positions are manually configurated. 


Deficiencies in the current setup

 The application GUI design is not created in an intuitive matter. The current design is just design in a simple and functional way. The application is a proof of concept and it is customized for the specific experiment. No handling of spatial mapping errors or anchor points are in concern in this project. 


Desired future state

An application for training of operator in a Mixed Reality (MR) and/or Virtual Reality (VR). The tutorial application should train the operator to assemble a log splitter. The purpose of training in MR/VR is to give the operator enough knowledge about the workflow before the operator starts work in the production.



With tracking hardware and/or software instruct the operator how to assemble step-by-step and verify that the operator have done the correct assemble. Create a connection with either HoloLens or HTC Vive with a tracking system as Nexxonar or similar. 



Knowledge in object originated programming, some knowledge about CAD drawing since Holograms are made easiest with CAD software. 



William Falkenström,  Consultant Assembly & Production flow, william.falkenstrom@virtual.se

Mikael Miettinen, Consultant Assembly & Production flow, mikael.miettinen@virtual.se

Torbjörn Danielsson
Manager assembly & Production flow and Industrial management