Knowledge Management enhances Augmented Reality and drives Field Service performance

Knowledge is power, this insight of the English philosopher Francis Bacon from 1597 is even more valid today. In earlier times, knowledge was usually kept to oneself to gain a personal advantage. Today, it is the sharing of knowledge that leads companies to success. Knowledge is the most valuable asset of a company, especially in times of increasing digitization.  And the upshot is that knowledge must be managed. Like employees, processes, changes, etc., knowledge needs to be developed and controlled as an essential resource within the company. I have been dealing with this for a long time. The trigger was many years ago an interview with a manager from Mercedes-Benz, the automobile company, who said in passing, “if Mercedes knew what Mercedes knows …“, and left the end of the sentence open, implying that “then we would have a huge competitive advantage“. Especially in service, this is often, even more, the case.

Um eine deutsche Version dieses Artikels zu lesen, klicken Sie bitte hier

Knowledge is regularly “hidden” in the various IT systems and applications and thus not easy to access and is not always consistent. And a lot of knowledge is (only) in the heads of employees and is often lost in turnover because there is usually not enough time available either for the training of the successor or even for the passing on of information. As Dorothy Leonhard and Walter Swap wrote in the Harvard Business Review, back in 2004, about “Deep Smarts”: “When veterans leave, it is painful to lose the capabilities partly because growing them in the first place is extraordinarily challenging. Deep smarts are experience based. They can’t be produced overnight or imported readily, as many companies pursuing a new strategy have discovered to their dismay.”

In service, this is a particularly noticeable problem. On the one hand, this is because the product portfolio that service must handle is significantly larger than the current offer of a company due to long product lifetimes and ever faster changing series. On the other hand, it is precisely in service that knowledge must be immediately available, in a distributed fashion, to achieve quick solutions and to ensure customer satisfaction.

In service, knowledge about the product plays a decisive role. The root cause of a problem has to be found and it is important to know how to rectify it quickly, preferably in advance, so that the intervention does not fail due to the lack of essential tools or spare parts. Frequently occurring equipment failures do not play a significant role, here virtually everyone knows from daily experience what to do. The many rare failures are the real problem. Although common “trial and error” eventually resolves the problem, it is not the fastest, nor the most cost-effective approach, nor does it lead to satisfied customers.

Many companies make do with databases and wikis to consolidate and share knowledge. Often, however, this fails because of the associated high cost to create, sustain and, mainly, continuously update them, as well as due to the lack of user-friendliness.

Today’s systems work differently. They can network existing knowledge, process it intelligently, continuously expand it and provide it in a targeted manner. Existing information stored in different systems is merged. This can be structured data such as parts lists and unstructured information such as service tickets or service reports. Unstructured knowledge – text or prose – is tapped via text mining tools and integrated with the structured data. Large amounts of data can then be condensed and used intelligently. Urgently needed information is provided easily and quickly.

Networking also makes it possible to recognize contexts, to analyze causes of failures and to create transparency. By using the system and verifying or falsifying system results, users continuously enrich it with expert knowledge. The current case may already be the solution for the next user.

Expertise, previously only available to humans, is structured and prepared in such a way that automated processes can work with it. Networking of information, recognition of interrelationships and text mining techniques are all elements of artificial intelligence (AI). However, in this case, it is always possible to understand what the system has learned and how it comes to its conclusions.

A classic example for the system is finding similar cases. That is, if an engineer is looking for the cause of a failure, the system looks for solutions and offers them based on comparable constellations. The source for this could be, for example, the targeted evaluation of completed service cases (e.g. service tickets). By choosing one of the proffered solutions based on the engineer’s own understanding, receiving the associated repair instructions, and confirming them as successful (or not successful, as the case may be) after the repair – the system has learned through this interaction.

Furthermore, such a system can generate new knowledge from existing information. By visualizing and recognizing patterns, correlations can be identified, and appropriate measures initiated. For example, as part of a maintenance action or repair, the system can recommend the maintenance or repair of other elements to avoid subsequent failures that have arisen in similar constellations.

A concrete example: A field engineer must solve a problem on site and needs support. In addition to basic information about the machine, which is already known, e.g. by generating a ticket or scanning a QR code, the engineer provides the system with information about the failure symptoms. This input can be made via a predefined structure or via free text. In the latter case, the vocabulary used is checked. Entries for an affected component can be compared with the Bill of Materials. When entering text, for example, the word “hum” is queried, whether it refers to “sound development” and whether vibrations are also occurring. Based on this current information, as well as the history of the specific machine and comparable machines, as already mentioned above, similar cases are identified, and the corresponding failure causes are suggested to the engineer.

If the system cannot provide a solution for the current case, it offers communication with a suitable expert. This is where Augmented Reality (AR) technology comes in, offering fascinating possibilities. Therefore, a short excursion:

AR technology makes it possible to supplement a real object, e.g. a machine or a component, with additional digital content. For example, in addition to the video live image on a tablet, smartphone or smart glasses, information and instructions for solving a problem can be displayed. These may be created by an expert remotely or they may be rendered as step by step instructions by the knowledge management system.

This makes AR technology a promising field of application in service. The engineer in our example is safely guided through the repair process with AR support. Since both field engineer and expert share the live image, the expert can intervene at any time to point out dangers and prevent errors in the repair process. I have seen myself how an engineer at a machine, who was connected by telephone with an expert, exchanged a wrong board since the two obviously spoke of different things and the expert saw a different image of the machine in front of him. And even frequently used tools such as video calls do not offer the benefits of targeted AR communication. Indicating a concrete object by means of AR symbols and icons, which also adhere to the object during movement, offers immense advantages, information and data can be precisely rendered, confusion excluded, and problems of noisy environments and even language barriers can be overcome.

Back to our example case: The field engineer could solve the problem with the help of the remote expert. The solution, the individual steps necessary are now available in the form of AR annotations and can be subsequently edited and saved. This is another advantage of the AR system: The repair process gets documented and can be used again for similar cases. So, if the engineer encounters this problem again in the future, the system will offer the solution and the technician can reuse the annotations of the first repair without having to consult the expert. In addition, the solution is also available to all other engineers. This saves significant time and effort. Over time, the database fills with instructions for troubleshooting and repair. The knowledge management system ensures that the appropriate instruction for each case is quickly found and rendered.

As a side effect of this process, the current configuration of the machine is continuously documented and updated. Many systems change over time and are adapted to new situations without documentation. Therefore, the expert at the remote desk sometimes has a picture of a machine that does not anymore correspond to reality.

If the visit of an expert on site is necessary, it can be carried more efficiently with the information available beforehand. Required spare parts and tools can be organized in advance. Time and costly travel are reduced, as is machine downtime.

In summary, the combined use of AR and service-focused knowledge management systems accelerates the service process and saves time and money for both the service provider and the user of machines and systems. The service provider saves search and travel times and can develop digital service and support offerings using either own or customer maintenance personnel directly. Agreements regarding service levels and availability can be extended and more easily adhered to. And the customer, the user, gets the tremendous advantage of less plant downtime.

Experience has shown that introducing digital support through combined knowledge management applications and augmented reality is easier than it first appears. If you choose a Pareto approach, the first good results can be achieved within about three months. In addition to the technology, a good preparation, an adaptation of the processes and the early inclusion of the employees involved are important.

Many companies provide their customers with free telephone support even after the warranty period. This requires resources that are not paid directly by the customers. Utilizing the new technological possibilities offers the opportunity to generate additional revenues with new offerings and at the same time increase customer satisfaction.

This article first appeared on 17. August 2018 in Service in Industry Hub

Related Posts

Share the Post:

Service in Industry

Deep dive into the industrial service business.

Join our community to receive analysis, insight, news and more.
We will never share your data

Service in Industry

Deep dive into the industrial service business.

Join our community to receive analysis, insight, news and more.
We will never share your data

Service Innovation for value-driven opportunities:

Facilitated by Professor Mairi McIntyre from the University of Warwick, the workshop explored service innovation processes that help us understand what makes our customers successful.

In particular, the Customer Value Iceberg principle goes beyond the typical Total Cost of Ownership view of the equipment world and explores how that equipment impacts the success of the business. It forces us to consider not only direct costs associated with usage of the equipment such but also indirect costs such as working capital and risks.

As an example, we looked at how MAN Truck UK used this method to develop services that went beyond the prevailing repairs, parts and maintenance to methods (through telematics and clever analytics) to monitor and improve the performance and  fuel consumption of their trucks. This approach helped grow their business by an order of magnitude over a number of years.

Mining Service Management Data to improve performance

We then took a deep dive into how Endress + Hauser have developed applications that can mine Service Management data to improve service performance:  

Thomas Fricke (Service Manager) and Enrico De Stasio (Head of Corporate Quality & Lean) facilitated a 3 hour discussion on their journey from idea to a real working application integrated into their Service processes. These were the key learning points that emerged:


In 2018 the Senior leadership concluded that to stay competitive they needed to do far more to consolidate their global service data into a “data lake’ that could be used to improve their own service processes and bring more value to customers. As a company they had already seen the value of organising data as over the past 20 years for every new system they already had a “digital twin” which held electronically all the data for that system in an organised fashion. Initially, it was basic Bill of Material data, but has since grown in sophistication. So a good start but they needed to go further, and the leadership team committed resources to do this.

  • The first try: The project initially focused on collecting and organising data from its global service operations into a data lake.  This first phase required the development of infrastructure, processes and applications that could analyse service report data and turn it into actionable intelligence. The initial goal was to make internal processes more efficient, and so improve the customer experience. E+H looked for patterns in the reports of service engineers that could:
    • Be used to improve the performance of Service through processes and individuals
    • Be used by other groups such as engineering to improve and enhance product quality.
  • Outcome: Eventhough progress was made in many areas, nevertheless, even using advanced statistical methods, they could not extract or deliver the value they had hoped   for from the data. They needed to look at something different.
  • Leveraging AI technologies: The Endress+Hauser team knew they needed to look for patterns in large data sets. They had the knowledge that self-learning technologies that are frequently termed as AI, could potentially help solve this problem. They teamed up with a local university and created a project to develop a ‘Proof of Concept’. This helped the project gain traction as the potential of the application they had created started to emerge. It was not an easy journey and required “courage to trust the outcomes, see them fail and then learn from the process”. However after about 18 months they were able to integrate the application into their normal working processes where every day they scan the service reports from around the world in different languages to identify common patterns in product problems, or anomalies in the local service team activities. This information is fed back to the appropriate service teams for action. The application also acts as a central hub where anyone in the organisation can access and interrogate service report data to improve performance and develop new value propositions.
  • Improvement:  The project does not stop there. It is now embedded in the service operations and used as a basic tool for continuous improvement. In effect, this has shifted the whole organization to be more aware of the value of their data.

Utilizing AI in B2B services

Regarding AI, our task was to uncover some of the myths and benefits for service businesses and the first task was to agree on what we really mean by AI among the participants. It took time, but we discovered that there are really two interpretations which makes the term rather confusing. The first is a generic term used by visionaries and AI professionals to describe a world of intelligent machines and applications. Important at a social & macroeconomic level, but perhaps not so useful for business operations -at least at a practical level. The second is an umbrella term for a group of technologies that are good at finding patterns in large data sets (machine learning, neural networks, big data, computer vision), that can interface with human beings (Natural Language Processing) and that mimic human intelligence through being based on self-learning algorithms. Understanding this second definition and how these technologies can be used to overcome real business challenges is where the immediate value of AI sits for today’s businesses. It was also clear that the implication of integrating these technologies into business processes will require leaders to look at the change management challenges for their teams and customers.

To understand options for moving ahead at a practical level we first looked briefly at Husky through an interview with CIO Jean-Christophe Wiltz to CIOnet where we learned that i) real business needs should tailored drive technology implementation, and ii) that before getting to AI technologies, there is a need to build the appropriate infrastructure in terms of database and data collection, and, most importantly, the need to be prepared to continually adapt this infrastructure as the business needs change.

Add Your Heading Text Here

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.