The opportunities, possibilities, and risks of the use of smart glasses in the laboratory setting are the subject of current research. This time, we will look at useful science apps for smart glasses and the challenges of their use in the lab.

 

4. Smart Glasses and Scientific Apps

One use for smart glasses in the laboratory is running scientific apps. These are smaller, specialized applications that require little or no digital infrastructure.

 

Colony Counter App

One example of this involves counting colony-forming units on agar plates. This is a widely used process for finding the number of living microorganisms in a sample for scientific research and quality assurance. There are instruments that can be used to automatically count the colonies on the agar plates, but these are usually only found in specialized labs. In academic settings, the agar plates are usually counted by hand. This can take several minutes for each plate and is prone to error because often, to save time, only part of each plate is counted and the result extrapolated to get a total.

The Colony Counter app offers an alternative by using images to determine the number of colony-forming units on agar plates [13]. This program uses a client-server architecture. The camera on the smart glasses is used to take a picture, which is not processed by the glasses themselves. Instead, it is transferred to a server where it is evaluated with an image-processing algorithm (see Fig. 4).

 

Figure 4. Flowchart for the Colony Counter app.

 

The results can then be transferred back to the smart glasses and displayed in the user’s field of vision. Subsequently, the user can decide if the result of the evaluation is satisfactory. If not, a new picture can be taken, and the evaluation carried out again. The big advantage of such a client-server architecture is that very computationally intensive tasks like the evaluation of images can be carried out by a powerful server. This vastly increases the speed of image evaluation and spares the battery in the smart glasses.

Similar concepts have been used for the quantification of chlorophyll in the leaves of plants and for the evaluation of immunochromatographic assays. Google Glass^® has been used in both of these cases [14,15].

 

Smart Glasses Used for Plant Classification

Another example of the use of scientific apps is in the study of the toxicity of chemicals to plants, known as ecotoxicology. In this type of study, various concentrations of chemicals are mixed into the soil around plants. The plants are then observed for a period of time to see if the chemicals have an effect on their growth or physiology. The plants are examined according to specific criteria by a researcher.

In the past, results for each plant were entered into a form. However, this approach has two distinct disadvantages: First, the information is only available on paper and must be digitized for statistical evaluation, and second, it is advantageous to have both hands free for the examination and visual evaluation of the plants. One hand can hold the plant while the second can move the leaves around to search for traces of chemical influence, such as growth defects. This is difficult if the researcher must also hold a pen and clipboard, making it nearly impossible for a single person to carry out the task effectively. This was the problem that inspired the development of the smart glasses application known as Plant-Classifier, a program that works without further digital infrastructure, allowing it to be used in any location.

A typical implementation of the Plant-Classifier proceeds like this: The researcher puts on the glasses and logs into the application with a QR code (see Fig. 5). This assigns the subsequent measurements to this user. The researcher then takes the first plant and scans the QR code on its pot. The QR code is linked to all relevant information, such as which study the plant is in, the type of plant, or attributes to be examined. The application automatically loads the correct interface for the evaluation.

The researcher can use voice commands to evaluate the various attributes in the interface. Examples of evaluation levels could include “normal”, “slight”, “average”, and “high”. Each entry is directly shown on the smart glasses’ display. Convenience functions make it possible to set all unchanging attributes to a single value. This speeds up the process of recording the attributes and is especially helpful when all features of the plant are normal.

 

Figure 5. Flowchart for Plant-Classifier. (A) Scan of the QR code to identify oneself to the system. (B) Scan of the plant QR code. (C) The appropriate evaluation layout for the plant and study is automatically loaded. (D) Plant attributes are evaluated using verbal commands. (E) Data are stored in a file on the smart glasses; the sequence from B to E is repeated until all plants have been examined. (F) All data is exported to a PC.

 

Once all of the plant attributes are recorded, the data are saved in a results file in the smart glasses. The user can then scan the QR code for the next plant and repeat the process until all plants have been examined. Afterward, the smart glasses can be connected to a computer and the results files can be transferred for statistical analysis.

The Plant-Classifier application on smart glasses offers the researcher several advantages during the examination. Both hands remain free for the examination of the plants, while all entries for the current plant are always overlayed in the visual field. The QR codes reliably identify the plants, preventing mistakes. The data generated are machine-readable and can easily be transferred for evaluation. Mistakes that may occur when transferring data from paper to the computer are avoided.

 

5. Smart Glasses and Remote Maintenance

Many laboratories use complex and highly specialized instruments. Instruction in the use of these instruments and their maintenance are often very costly because specialists must first often travel across the globe to be on site. This is particularly annoying when the problem is minor and could be solved with a couple of simple maneuvers if the people working in the lab just knew how to do it. For this reason, more efforts are being made to develop remote maintenance. This would allow laboratory workers to fix problems with digital guidance, making technician travel unnecessary.

The simplest option is to provide instructional pictures and videos or augmented reality content for problems that occur frequently. These can easily be provided on tablets or notebook computers [16]. If these are insufficient, a technician can be included by means of a video conference. This is another good application for smart glasses (see Fig. 6).

 

Figure 6. Assistance with remote maintenance. Types of support for remote maintenance using the maintenance of an industrial machine as an example, including 2D schematics and highlighting of relevant areas in the image, 3D models with concrete positioning, and symbols that clarify the instructions (modified according to [17]).

 

Both parties can communicate with each other over the microphone and speaker in the smart glasses. The smart glasses’ camera can provide a video stream to the technician. This shows the technician exactly what the instrument user is seeing (personal view). The same video stream is also superimposed on the visual field of the smart glasses. Depending on the software being used, the technician can include notation or 3D elements in the video stream to provide better guidance for the person performing the maintenance [17].

The advantages of the use of smart glasses in this case are that the video perspective corresponds to the visual field of the person on site, that the person on site receives all of the information from the technician directly in their field of vision, and above all, that both hands are free to carry out the actual service.

Because it is of great financial interest, the field of remote maintenance has already seen a great deal of development. There are various firms that offer services and technology using smart glasses, including the German company Oculavis [18].

 

6. Challenges in the Use of Smart Glasses

If smart glasses have such great potential for use in laboratories, why are they still so rarely found in a lab? Smart glasses are a relatively new technology with which people have had little experience. This means that legal aspects must be clarified, technical limitations overcome, and potential health effects considered.

 

Validation Required

In combination with the right digital infrastructure, smart glasses offer many advantages. These are particularly apparent in regulated fields that work according to GMP (good manufacturing practice) or GLP (good laboratory practice), such as the pharmaceutical industry. In these cases, extensive documentation can be taken over by the digital system. However, before new technology like smart glasses can be implemented in such fields, it must be validated, which can also be very complicated and expensive, depending on the complexity of the application.

 

The Possibility of Monitoring Workers

Another legal problem is the potential ability to monitor workers. Like smartphones, nearly all smart glasses contain directional and acceleration sensors that can theoretically be used to determine what a person is doing at a given time [19]. This problem is compounded when smart glasses are also worn as eye protection. In this case, they are not to be removed, so unlike a smartphone, they can make it impossible for a worker to actively protect his or her privacy. The camera poses a similar problem. With smart glasses, the camera normally shows what a worker is looking at, making it a very efficient surveillance tool to abuse.

The privacy of third parties can also be compromised if photographs or videos are recorded with the smart glasses [20]. These concerns should be addressed and, if possible, controlled technically before the introduction of smart glasses into the workplace. This is particularly important for increasing the acceptance of the new technology by workers.

 

Outdated Operating Systems and Battery Capacity

A further limitation is in the smart glasses themselves. In many cases, smart glasses are delivered with outdated operating systems. For example, when introduced in 2019, the Vuzix Blade was delivered with Android 5, an operating system developed in 2014. It was four generations behind Android 9, the latest OS at the time. This makes the development of software more complicated because it often requires the use of outdated software libraries. Because of the low processing power of smart glasses, computationally intensive tasks have to be outsourced to servers. This requires more complex infrastructure to be in place wherever the application is going to be used.

The internal batteries in many smart glasses have very low capacities, so auxiliary batteries are required. These may be fastened to the protective glasses themselves, which increases the weight. Alternatively, extra batteries can be worn on the body, but this may limit the freedom of movement of the head. Older models of smart glass also had low-resolution screens, which made it difficult to display complex content.

 

Health Hazard?

The third important aspect is the question of whether the use of smart glasses represents a health hazard for workers. The increased weight carried on the head because of the smart glasses can lead to elevated stress to the neck and shoulder musculature. The eyes may also be strained by smart glasses because the eye must constantly adjust its focus between “near” (to see the information on the smart glasses) and “far” (for normal vision). Physicians refer to this as accommodation. In addition, when using the non-transparent displays incorporated into many smart glasses today, the field of vision is always limited, even when no information is shown on the display.

The degree to which these influences affect people was investigated in a study by the German Federal Institute for Occupational Safety and Health (known as baua) in 2016 [21]. In the study, the strain of working with smart glasses was compared to that of working with tablets. Participants worked for up to four hours with either smart glasses or a tablet. It was shown that the use of smart glasses has no effect on the vision of the study participants. No impairment of the accommodation or field of vision was observed.

The study found a very small additional load on the neck and shoulder musculature. The subjective impression of pain by the study participants was significantly higher than the measurable strain. Mental strain was also studied. At the start, this was both objectively and subjectively higher with smart glasses than with a tablet. After a period of familiarization, the objective strain of using smart glasses was the same as for the tablet, but it felt significantly higher. Whether long-term use of smart glasses could lead to adverse health effects needs further study.

It should also be noted that the baua study was carried out with the first generation of commercially available smart glasses. There have been some developments since that time that have significantly improved both the comfort and technical performance of the devices.

For example, Table 2 displays various models of smart glasses offered by the Vuzix company. In each generation from M100 through M300 to M400, the hardware, processor, working memory, camera, and display were all improved. Higher display resolution, in particular, contributes to an improved user experience. In addition, certain design decisions significantly increase the comfort of wearing the devices. From the model M300 onward, an additional battery was built into the side opposite the display, which increases the overall weight of the device, but increases comfort because of the balanced weight distribution.

 

Table 2. Generations of Vuzix smart glasses [23,24].

 

At first glance, the Vuzix Blade looks like a somewhat chunky pair of normal glasses. The display is integrated into the glass, so that an inactive display does not impact the field of vision. With a total weight of 90 g, this device is relatively light. However, in comparison to the M400, which was released at nearly the same time, the hardware components of the Blade are relatively weak. In the opinion of the authors, this model is still not suitable for daily use, though development is heading in the right direction. Aside from Vuzix, several other companies are developing exciting concepts for new smart glasses. One example of this is the American company North, which is using technology that directly projects images into the eye, so that no display is required [22].

 

7. Conclusion

It is evident that some legal and medical questions must be answered, and further technical developments are needed with regard to smart glasses before they can be implemented in laboratories on a daily basis. For now, smart glasses are a tool for special tasks, such as remote maintenance or individual experiments with an app like Plant-Classifier, which only require them to be worn for a few hours and not every day.

However, smart glasses are improving greatly with every generation and new technologies are always being developed that could revolutionize the field of smart glasses. Whether smart glasses will completely replace classic safety glasses is hard to predict and depends on technical developments and acceptance by laboratory workers. If smart glasses gain prevalence among consumers, as was unsuccessfully tried with Google Glass^® in 2013, this process will certainly accelerate.

It is also conceivable that another technology will render smart glasses unnecessary, such as a combination of smart speakers and mobile displays. Smart glasses clearly have extraordinary potential for application in the laboratory and are currently a valuable tool for special tasks.

 

8. Summary

Safety glasses are required in the lab, so why not give them added value? The advantage of smart glasses is that the wearer sees digital information directly in his or her field of vision, can navigate these through verbal commands, and still has both hands free for work. This strength is particularly important in carrying out digital workflows, using digital apps, and remote maintenance.

Currently, smart glasses are used for special laboratory applications that are limited in duration. Before the extensive use of smart glasses becomes possible, legal and health-related questions must be answered. For all-day use in the lab, the technology and comfort of smart glasses still need improvement. In the fully digitalized laboratories of the future, smart glasses could become a very valuable and powerful tool for daily lab work.

 

References

[13] J. Austerjost et al., A smart device application for the automated determination of E. coli colonies on agar plates, Eng. Life Sci. 2017, 17, 959–966. https://doi.org/10.1002/elsc.201700056

[14] B. Cortazar et al., Quantification of plant chlorophyll content using Google Glass, Lab Chip 2015, 15, 1708–1716. https://doi.org/10.1039/C4LC01279H

[15] S. Feng et al., Immunochromatographic Diagnostic Test Analysis Using Google Glass, ACS Nano 2014, 8, 3069–3079. https://doi.org/10.1021/nn500614k

[16] S. Webel et al., An augmented reality training platform for assembly and maintenance skills, Rob. Auton. Syst. 2013, 61, 398–403. https://doi.org/10.1016/j.robot.2012.09.013

[17] S. Aromaa et al., Use of wearable and augmented reality technologies in industrial maintenance work, Proceedings of the 20th International Academic Mindtrek Conference on – AcademicMindtrek ’16 2016, 235–242. https://doi.org/10.1145/2994310.2994321

[18] Oculavis GmBH, oculavis.de. (accessed May 31, 2021)

[19] F. Attal et al., Physical Human Activity Recognition Using Wearable Sensors, Sensors 2015, 15, 31314–31338. https://doi.org/10.3390/s151229858

[20] Andreas Kotsios, Privacy in an augmented reality, A. Int. J. Law Inf. Technol. 2015, 23, 157–185. https://doi.org/10.1093/ijlit/eav003

[21] Bundesanstalt für Arbeitsschutz und Arbeitsmedizin (baua), Head Mounted Displays – Arbeitshilfen der Zukunft. Bedingungen für den sicheren und ergonomischen Einsatz monokularer Systeme (in German), 2016. https://doi.org/10.21934/baua:praxis20160809

[22] North, www.bynorth.com. (accessed May 31, 2021)

[23] Vuzix, Compare Vuzix Smart Glasses. (archived version, accessed May 31, 2021)

[24] Vuzix, Vuzix M100 Smart Glasses, 2016. (accessed May 31, 2021)

 

The Authors

Daniel Marquard, born in 1988, studied life science in Hannover, Germany, and Cambridge, UK, obtaining his master’s degree at Leibniz University Hannover in 2013. He received his doctoral degree in 2017 from the Institute for Technical Chemistry for the development of image processing algorithms for biotechnological processes. He is currently a postdoctoral researcher working in the areas of laboratory digitalization, standardized instrument communication, data management, digital workflows, and interactive media for the digitalized laboratory.

Marc Porr, born in 1992, studied life science in Hannover and received his master’s degree in 2017 from Leibniz University Hannover. He is currently carrying out doctoral research at the Institute for Technical Chemistry at Leibniz University Hannover in the area of laboratory digitalization. In addition to the development and implementation of system architectures, he works on standardized instrument communication, data management systems, digital workflows, and user interaction media for digitalized laboratories.

Ferdinand Lange, born in 1990, studied computer science in Hannover and received his master’s degree in 2018 at Leibniz University Hannover. Since then, he has been a research assistant at the Institute for Technical Chemistry at Leibniz University Hannover and is a leader in the development and implementation of software architectures for laboratory digitalization projects.

Jonas Austerjost, born in 1989, studied life science in Hannover and at Stanford University, CA, USA, receiving his master’s degree in 2014 at Leibniz University Hanover. Subsequently, he worked as a research assistant at the Institute for Technical Chemistry at Leibniz University Hannover, receiving his doctorate in 2019 for the development of innovative automation solutions for the chemical and biotechnological laboratory environment. He currently is a researcher in the Corporate Research Division of Sartorius AG in Göttingen, Germany, working on the development of digital tools for the support and optimization of laboratory workflow.

Sascha Beutel, born in 1971, studied chemistry at Leibniz University Hannover. He received his doctorate there in 2000 at the Institute for Technical Chemistry. In addition, he became a Professor in the field of technical chemistry in 2018. He heads a research group working in the area of bioprocess engineering for the cultivation of prokaryotes, enzyme technology processes, methods for the preparation of biotechnological products, and new sensor components. The team also carries out research on the adaptation of new technologies, such as 3D printing, smart glasses, or voice interaction for the laboratory.

 

Contact Information

PD Dr. Sascha Beutel
Leibniz-Universität Hannover Institut für Technische Chemie #
Callinstraße 5
D-30167 Hannover
E-Mail: [email protected]

---

The article has been published in German as:

• Smartglasses im Labor,
  Daniel Marquard, Marc Porr, Ferdinand Lange, Jonas Austerjost, Sascha Beutel,
  Chem. unserer Zeit 2020, 55, 86–97.
  https://doi.org/10.1002/ciuz.202000022

and was translated by Caroll Pohl-Ferry.

---

Smart Glasses: Tool or Toy? – Part 1

Latest research into laboratory applications for smart glasses