Cognitive Systems/Machines: Teaching Machines to Learn

As a shopping assistant, service robot Paul recently welcomed customers in an electronics store, asked which products they wanted, and escorted them to the appropriate shelf. On the way, he chatted about the weather and then asked a few feedback questions, for example, whether the customer was satisfied with his service. “He operates in a dynamic, everyday environment in which he recognizes objects or people and reacts to them,” explains Martin Hägele, head of the Robot and Assistive Systems Department at Fraunhofer IPA. The robot uses sensors to gather information about its environment in order to navigate dependably at all times and to locate people to conduct dialogs.

Paul contains the Care-O-Bot4® robot platform, which Fraunhofer IPA originally developed for active support of people in households, hotels, nursing homes and hospitals. Now plans call for increased deployment in companies.

Understanding environments, planning actions, reacting to obstacles, communicating with people - cognitive systems master these challenges by harnessing machine learning methods. Here, machines learn to solve a task on the basis of example data and to transfer what they have learned to new situations. For example, they can plan and optimize processes, make forecasts, recognize patterns or distinctive features, and analyze image and voice signals. These systems form the basis for important future technologies such as autonomous driving or autonomous robots.

Thanks to the software SHORE from Fraunhofer IIS and "affective computing", shopping assistant Paul can even recognize a person's mood and express his own state of mind. This involves machine recognition of emotions in facial expressions, sensor data merging, and analysis of biosignals, such as pulse, voice, gestures or movement. For example, the stress level of car drivers or factory workers can be determined, as can customer wishes or requirements. “Analysts regard affective computing as the commercially fastest growing market in the machine learning field,” explains Dr. Jens-Uwe Garbas, head of the Intelligent Systems Group at Fraunhofer IIS.

Shaping the future with human and artificial intelligence

Artificial intelligence (AI), cognitive systems and machine learning play a transformative role in bringing about the economy and society of the future. For international business and industrial value chains, this means a fundamental structural change, as these technical systems are capable of learning and are increasingly able to apply what they have learned to new situations. They can plan processes, make forecasts and even interact with people. The International Data Corporation predicts that worldwide expenditure on cognitive solutions will reach 40 billion U.S. dollars by 2020.

However, the use of new technologies not only offers exciting new opportunities: it also constantly presents us with challenges. If Germany is to attain technological leadership in the field of AI, it needs not just technological solutions, but also social dialogue. Fears and myths must be countered with scientific facts. Machines will extend our sphere of influence, but they will not take over and become our leaders.

At many of its institutes, the Fraunhofer-Gesellschaft is developing key AI technologies and applications in such areas as robotics, image and language processing, and process optimization. This includes machine learning techniques for industry, the use of cognitive systems in cyber security and the requisite research into artificial neural networks. Our research makes major contributions to the theory and the ethical design of AI while also being closely aligned with the practical needs of our customers.

• Download [PDF 1,2 MB]

Blazing their way through unknown terrain is especially challenging for robots. The Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation IOSB is developing probabilistic methods that assess statements not only as “true” or “false”, but additionally with a certain probability that indicates the degree of uncertainty of the statements. “While the technology is highly complex, it plays a crucial role in mastering all possible uncertainties,” explains Professor Jürgen Beyerer. The robot constructs a type of "world model" for itself by observing the real world; it then reaches adjusted conclusions by making associations, interpreting new information and learning new concepts. This “powerful memory structure” even allows robots to be used in danger zones, which Fraunhofer IOSB demonstrated by taking the fully autonomous vehicle “IOSB.amp Q1” as an example. This mobile platform is able to explore even especially rough terrain on its own. A demanding task that cannot be solved based on logic alone.

Underwater robots that operate in depths where no light signals or radio waves penetrate call for what is probably the highest level of autonomy. They glide through the depths without cables, collect data and return to the research ship on their own. The Shell Ocean Discovery XPRIZE competition challenges teams from around the world to participate in the large-scale mapping of the ocean floor at depths of up to 4000 meters. The only German team competing in the field is the Argonauts from Fraunhofer IOSB.

But even without robots, learning systems are making more and more of an impact in many industrial sectors. For example, machine learning methods can also be used to examine complex situations under real conditions, and can, for example, replace elaborate prototype tests. At the Fraunhofer Institute for Algorithms and Scientific Computing SCAI, a team headed by Professor Jochen Garcke, head of the Numerical Data-Driven Prediction Department, is investigating how the various methods can be adapted to specific technical tasks. For instance, in vehicle development, numerical crash simulations are employed during the development phase for the ongoing optimization of components and sheet metal thicknesses. “Our methods are helping structure the large quantities of complex data,” Garcke explains. “Which data are similar, which are different? This is not easy to see when there are 50 different forming steps.” The investigation has produced innovative methods for development engineers in the automotive industry, who can now conduct simultaneous comparative analyses of data from many simulations.

Particularly in the engineering world, the buzz term "grey box" pops up in the machine learning procedure. While black box models do not take the physical model of the problem that is to be learned into account, it is derived as precisely as possible and used in white box algorithms. Grey box combines these two approaches: Here, a data analysis model is enriched with physical knowledge in order to achieve better analyses.

The Fraunhofer Institute for Industrial Mathematics ITWM also works with system simulations for virtual vehicle development and safeguarding. “Today there are various levels of assistance situations, ranging from fully human guided to autonomous, and the number of information systems in use is steadily rising,” explains Dr. Klaus Dreßler, head of the Mathematical Methods in Dynamics and Durability Department at Fraunhofer ITWM. He is examining how drivers and their assistance systems react to external conditions with the help of RODOS, the first interactive motion simulator based on an industrial robot, which extends far beyond a standard simulator's possibilities. This should allow safeguarding concepts for mixed traffic in the future. “We challenge the intelligence of the autonomous vehicles and observe how they react when, for example, they are cut off by a human-controlled car.”

Construction sites pose a further challenge in road traffic. Drivers behave with uncertainty and automated vehicles have problems understanding complex traffic routing with different information on speed and the course of the lanes. AutoConstruct, a project funded with almost two million euros by the German Federal Ministry of Economic Affairs and Energy, aims at real-time environment recognition in construction sites using series and cost-optimized camera sensor systems for highly and fully automated driving. The project kicked off at the beginning of December 2016. The Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS is substantially involved, and has taken on the job of image recognition and processing. ”With deep learning, a key technology for the automotive sector’s future, we are teaching the software to recognize classic patterns more swiftly and more efficiently,” explains Dr. Stefan Eickeler, responsible for object recognition at Fraunhofer IAIS.

Deep learning draws on multi-layer, artificial neural networks which, in an abstract form, are based on information processing in the human brain. Deep learning is used to analyze highly complex data. Today, however, not even researchers can precisely explain how neural networks reach certain results. The process more or less involves feeding values into a black box and obtaining “surprisingly” applicable results. The team headed by Dr. Wojciech Samek, head of the Machine Learning Research Group at the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI, has been working with the Technische Universität Berlin to develop software that makes it possible to observe a neural network as it thinks. For example, if genetic data from patients is fed into such a network, it can analyze the probability that the patient has a certain genetic disease. “But it would be even more interesting to know which characteristics the program uses to determine its decisions,” says Samek. This could be a specific genetic defect that supplies information for cancer therapy individually tailored to the patient.

At the Fraunhofer Institute for Algorithms and Scientific Computing SCAI, the Bioinformatics Department focuses on speeding up the path from knowledge generation to application for the biotechnological and pharmaceutical industry, primarily in the area of dementia research. This involves combining very different data and distributed knowledge. The department specializes in generating disease models for this purpose. “Machine learning methods help us extract both data and knowledge from the literature on an order of magnitude that extends far beyond the cognitive capabilities of individual scientists,” states Professor Martin Hofmann-Apitius, head of the Bioinformatics Department. “With help from the machines, we are generating models of the world that allow new insights into the mechanisms of dementia that are beyond human capability.”

Industrial production is likewise generating more and more data in the age of Industrie 4.0 and the Internet of Things. “Since 2009, we have been using unsupervised, data-driven learning methods in order to monitor a process's normality and detect errors,” explains Christian Frey, head of the Systems for Measurement, Control and Diagnosis Department at the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB. “Now we can additionally optimize the processes.” The system searches the data for key performance indices and then maps these to qualities, resource efficiency, material or energy consumption, and similar categories. For example, a universal tool to monitor complex chemical processes is already in use.

At the Fraunhofer Institute for Open Communication Systems FOKUS, the work concentrates on the use of time-variable electricity prices in power-intensive processes as well as on “predictive maintenance” for distributed systems and for infrastructure management of smart cities. “For example, we create forecasts for ‘load shifting’,” explains Dr. Florin Popescu, head of the IT4Energy Center Project at Fraunhofer FOKUS. Load shifting is intended to prevent energy peaks in production and to profit from energy rates that vary over time. This forecast is based both on algorithms and on big data integration, and it leads to the necessity of continuous planning adjustments. “We estimate that careful forecasting can cut the operating costs of large, energy-intensive industries by five to ten percent,” Popescu states. “Wind parks and other fixed systems could save even more.”

And if humans and machines work directly next to one another and are to cooperate with one another in industrial production? Mobile assistive robots must also recognize the workers' movements and be able to avoid running into them. In addition, they must independently decide which action is necessary in order to carry out diverse tasks with people and within human environments. Researchers from the Fraunhofer Institute for Factory Operation and Automation IFF are developing ANNIE, a mobile assistive robot, as well as the complex software framework and sensor data processing to allow flexible performance of tasks such as screwing, drilling, welding and bonding. “Mobile assistive robots must display fault-tolerant behavior in order to remain capable of acting in unforeseeable situations,” says Christoph Walter from Fraunhofer IFF, who coordinates and directs the research topics on mobile assistive robots at the institute.

Valued at billions, the cognitive systems market calls for major research and development activities, and will likewise necessitate major investments to be committed. In this area, Fraunhofer is a significant system provider with core competences. The objective is to promote the research and development of cognitive machines as a forward-looking field in Germany, to position it prominently, and to generate innovations. A number of joint projects and initiatives have already been launched. The Fraunhofer Big Data Alliance has initiated a far-reaching investigation into market development and potential innovations in artificial intelligence as an internally funded project. The Young Research Class, Fraunhofer's new career program for young scientists, is also dedicated to the topic of cognitive machines. In 2017, the proof of concept should be ready for the “self-determined learning cognitive assistants” idea. Fraunhofer and the Max Planck Society are cooperating on a project on the topic of machine learning. They hope to reduce large, multi-dimensional machine learning models to the parameters relevant for the results, without having to incur considerable quality losses.

There are still a number of issues to clarify, such as how to handle ethical aspects, and how to achieve further efficiency increases in the methods by harnessing revolutionary technologies such as neuromorphic chips. “All the necessary disciplines and competencies are in place at Fraunhofer at high quality levels,” summarizes Jürgen Beyerer from the Fraunhofer Institute of Optronics, System Technologies, and Image Exploitation IOSB. “Now we need to consolidate the previously unnetworked, parallel activities into visionary, symbiotic project teams."