Digitalization affects more and more areas. However, in this process, we often forget that it must also be possible to appropriately integrate humans into our new digital infrastructure.

In principle, many devices are suited to this role, including smartphones and tablets. However, in environments where chemicals are used, such as laboratories, their limitations quickly become apparent—they are very difficult to use when wearing gloves. One solution for this problem could be the use of wearables, in particular, smart glasses. For safety reasons, laboratory workers must wear protective glasses anyway, and these could simply be modified to include a data function. The opportunities, possibilities, and risks of the use of smart glasses in the laboratory setting are the subject of current research.

 

1. The Development of Smart Glasses

End of the 1960s: The First Head-Mounted Displays (HMDs)

The idea of being able to superimpose graphical information directly into the human field of vision to create a bridge between the real and virtual worlds has fascinated people for a long time. The first head-mounted display (HMD) was produced over 50 years ago, at the end of the 1960s, by pioneers such as the American computer scientist and internet pioneer Ivan Sutherland. This device could be worn on the head and superimposed graphical content into the user’s field of vision [1].

The device created by Sutherland and his colleagues was called “the sword of Damocles”, however, because its massive weight and bulky head tracking mechanism made it necessary to attach it to the ceiling, making it appear to hang menacingly over the user’s head (see Fig. 1a, b).

 

Figure 1. (a, b) The world’s first head-mounted display (HMD), named the “Sword of Damocles” (1968). (c) One of the first HMDs used in a professional setting, the Integrated Helmet and Display Sighting System (IHADSS) for Apache combat helicopters (1985). (d) Google Glass® (2013) (Loïc Le Meur, CC BY 2.0) [5-7].

 

1980s: Pilots with HMDs

In recent decades, the miniaturization of electronics and optics has allowed for the development of smaller and more powerful HMDs. In the 1980s, HMDs became small enough to be attached to a helmet while the other associated equipment could fit into a backpack [2].

This made it possible to equip pilots of Apache combat helicopters with the “Integrated Helmet and Display Sighting System” beginning in 1985 (see Fig. 1c). This device superimposed the live image from an infrared camera directly into the pilot’s field of vision [3]. This gave them “night vision” and made it possible to see what was directly underneath their helicopter.

 

2013: Google Glass^®

Today, the dimensions of HMDs barely differ from those of normal glasses and they are the basis for modern smart glasses [4]. When Google Glass^® was introduced in 2013, smart glasses technology was made available to the general public for the first time (see Fig. 1d). Other producers followed Google and produced models with similar technical properties.

Although most current models of smart glasses are not successful with consumers, applications have been developed in some professional fields. These include documentation, visualization, and education applications in the healthcare sector, as well as the support of production runs and quality assurance in logistical and manufacturing processes [8,9].

 

2. Interaction with Digitalized Laboratories

Another exciting area of application for smart glasses is the use in chemical and biological laboratories. For years, there have been efforts to achieve paperless, fully digitalized laboratories, in which all instruments are networked with each other and all processes can be monitored and documented digitally [10,11]. However, a fully digitalized laboratory is not a fully automated laboratory. For many experiments, particularly complex ones, humans are still needed in the lab. In consequence, people must be integrated into the digitalized laboratory.

 

Tablets

We, therefore, need interactive media that can act as the interface between humans and the digital laboratory. In most cases, tablets available on the consumer market are currently being used for this. Their costs are relatively low and acceptance is high, since most people use tablets in their private lives.

However, the problem with tablets in the lab is that they must be used with hands, which can easily lead to chemical and biological contamination. This contamination is difficult to remove because most cleaning agents can damage the display or the electronics. In addition, it is an advantage to have both hands free for many laboratory processes.

 

Touchscreen Projectors and Smart Speakers

For this reason, research and trials are currently being carried out to find useful alternatives to tablets for use in laboratories. Porr et al., Leibniz University, Hanover, Germany, have proposed the use of touchscreen projectors in the laboratory. These function like a tablet, but the image is projected onto a table surface and overlayed with an infrared grid that registers touches. This converts the table surface into a control panel, protecting the device itself from contamination [10].

Austerjost et al., also from Hanover, have developed a purely speech-based system for the interaction with devices by using an Amazon Echo smart speaker [12]. However, this concept completely forgoes any visualization, which makes it difficult for people in the laboratory to record complex procedures. In addition, neither touchscreen projectors nor the smart speaker integrate a camera, a device that is becoming increasingly important for documentation in the lab. It is, thus, clear that there is a need for suitable interactive devices in laboratories. This is where smart glasses come into play.

 

Smart Glasses

In most laboratories, wearing protective glasses is mandatory for occupational safety. Why should the glasses not provide a bonus service?

Smart glasses have integrated sensors, such as microphones and cameras (see Fig. 2). These can be used for documenting experiments, as well as for measurements. Pictures captured by the glasses can be directly evaluated by image processing algorithms. In addition, smart glasses have WiFi and Bluetooth interfaces that make it possible to connect external sensors and instruments, as well as allowing embedding and data transfer to a higher-level IT infrastructure. The display of the smart glasses can be used to directly superimpose information, work procedures, or measurement results into the user’s field of vision. This information can be overlayed with live images from the smart glasses’ camera in order to further increase the quality of the information. This type of information representation is known as assisted or augmented reality (see Infobox 1). Most smart glasses also have voice control, which allows a person in the laboratory to be integrated with the digital system while keeping their hands free to work.

 

Figure 2. Smart glasses with safety-glasses function (Vuzix M300).

 

Below, we will introduce some applications for smart glasses that are either already published or stem from current research projects. We will focus on applications of monocular systems, as shown in Fig. 2, because their low weight and high comfort seem to make these most suitable for work in a laboratory. Other systems, such as augmented reality glasses (see Infobox 1) are still too heavy and limit the field of vision too much for safe use in a laboratory.

 

3. Smart Glasses and Digital Operating Procedures

All certified laboratories, like those involved in environmental or food analysis, use standard operating procedures (SOPs). These are highly structured, detailed instructions for work procedures or experiments, often still paper-based. The development of digital infrastructure in laboratories will allow these paper-based SOPs to be replaced with digital SOPs [10]. One illustration of this is the digital SOPs developed by Austerjost et al. for MEBAK methods, which are standard analytical methods used by the Central European Commission for Brewing Analysis (known as MEBAK) to evaluate beer characteristics, including color, nitrogen content, or original gravity.

These digital SOPs even allow laypeople to carry out these assays [11]. The goal of the SOPs is to support people as comprehensively as possible and particularly to help them to avoid errors. The person is, thus, guided through the operating procedure step by step, with all relevant information provided in a structured manner in conjunction with whichever step in the process is being carried out. Optimally, any networked laboratory instruments would automatically be programmed with settings and measurement parameters, and the digital infrastructure would simultaneously document the processes so that the person can concentrate on their actual laboratory task. This reduces the probability of errors but makes the creation of digital SOPs more elaborate than paper-based SOPs (see Infobox 2).

Smart glasses are a powerful tool for the implementation of digital SOPs. Current information can be displayed on the smart glasses’ screen, which allows the person to have everything directly in their field of vision, regardless of where they are in the laboratory. Navigation within the digital SOPs requires only a few commands. Five essential ones are shown in Table 1. If needed, the camera can be used for documentation or scanning QR codes, while both hands remain free for laboratory work at all times.

 

Table 1. Required commands for carrying out a digital SOP.

 

Figure 3 shows excerpts of the visualization steps of a digital SOP for the production of a yeast extract medium, as used for the cultivation of bacteria. To start, the person identifies themselves to the system with a QR code. In the second step, a beaker is placed on the scale. As soon as the person uses the “confirm” voice command to verify that the beaker is on the scale, it is automatically tared. In the next step, a yeast extract container must be retrieved and registered in the system by means of a QR code. Upon registration, all information known about this chemical is automatically stored in the database, such as the name of the chemical, its expiration date, and lot number. If the wrong QR code is scanned, the person gets an error message and must scan again. The SOP only proceeds when a correct QR code is scanned.

 

Figure 3. Excerpt from a digital SOP for the preparation of a yeast extract medium. The number in the upper right corner of each picture indicates the current step of the SOP.

 

In the next step, the yeast extract is weighed. Both the target value and actual weight value are displayed live. As soon as the actual weight is within a predefined tolerance of the target value, the weighing is considered successful, and the system automatically moves to the next step. The weighed mass of the yeast extract is automatically documented by the system.

Next, the yeast extract is transferred to a screw-cap bottle (Schott bottle). This process is repeated for all of the components of the growth medium, until all components are combined. In the last step, the prepared medium is given a QR code label that can be used to access all of the important information. Scanning this label registers the medium in the system and the SOP ends.

A mini-study was carried out to see if such digital SOPs could lead to an increase in efficiency. Participants carried out a MEBAK SOP for the photometric examination of beer. All participants were carrying out the SOP for the first time. One half of them used a paper-based SOP, the other half the digital SOP. Participants using the digital SOP were on average almost 20 % faster [11]. Whether this significant gain in efficiency also holds for other SOPs should be more thoroughly investigated in larger studies.

 

References

[1] I. E. Sutherland, in Proceedings of the December 9–11, 1968, fall joint computer conference, part I on – AFIPS ’68 (Fall, part I), ACM Press, New York, USA, 1968, 757.

[2] S. Mann, Wearable computing: a first step toward personal imaging, Computer (Long. Beach. Calif). 1997, 30, 25–32. https://doi.org/10.1109/2.566147

[3] C. E. Rash, J. S. Martin, The impact of the US Army’s AH-64 helmet-mounted display on future helmet design, 1988.

[4] J. Carmigniani, B. Furht, Handbook of Augmented Reality, Springer New York, New York, NY, 2011, 3–47. ISBN: 978-1-4614-0064-6

[5] ‘The Sword Of Damocles’, 1st Head Mounted Display, vrroom.buzz, 2016. (accessed May 3, 2021)

[6] US Army, Integrated Helmet Display Sight System, wikimedia commons, 2008. (accessed May 3, 2021)

[7] Loïc Le Meur, A Google Glass wearer, wikimedia commons, 2013. (accessed May 3, 2021)

[8] S. Mitrasinovic et al., Clinical and surgical applications of smart glasses, Technol. Heal. Care 2015, 23, 381–401. https://doi.org/10.3233/THC-150910

[9] W. A. Farrell, Learning Becomes Doing: Applying Augmented and Virtual Reality to Improve Performance, Perform. Improv. 2018, 57, 19–28. https://doi.org/10.1002/pfi.21775

[10] M. Porr et al., smartLAB – Interaktives Arbeiten in digitalisierter Laborumgebung, Chem. Ing. Tech. 2019, 91, 285–293. https://doi.org/10.1002/cite.201800090

[11] J. Austerjost et al., A flexible IT infrastructure for the integration of smartglasses into the brewing laboratory as a digital support for standard analysis workflows, Brew. Sci. 2019, 72, 1–9. https://doi.org/10.23763/BrSc18-20austerjost

[12] J.Austerjost et al., Introducing a Virtual Assistant to the Lab: A Voice User Interface for the Intuitive Control of Laboratory Instruments, SLAS Technol. Transl. Life Sci. Innov. 2018, 23, 476–482. https://doi.org/10.1177/2472630318788040

---

The article has been published in German as:

• Smartglasses im Labor,
  Daniel Marquard, Marc Porr, Ferdinand Lange, Jonas Austerjost, Sascha Beutel,
  Chem. unserer Zeit 2020, 55, 86–97.
  https://doi.org/10.1002/ciuz.202000022

and was translated by Caroll Pohl-Ferry.

---

Smart Glasses: Tool or Toy? – Part 2

Useful science apps for smart glasses and challenges of their use in the lab