Publications

Journals and Conferences (23)

C. Street, S. Pütz, M. Mühlig, N. Hawes, and B. Lacerda, “Congestion-Aware Policy Synthesis for Multirobot Systems,” IEEE Transactions on Robotics, pp. 1–19, 2021.
DOI: 10.1109/TRO.2021.3071618 | abstract |
```
@article{paper-street-2021,
  author = {Street, Charlie and Pütz, Sebastian and Mühlig, Manuel and Hawes, Nick and Lacerda, Bruno},
  journal = {IEEE Transactions on Robotics},
  title = {Congestion-Aware Policy Synthesis for Multirobot Systems},
  year = {2021},
  volume = {},
  number = {},
  pages = {1-19},
  doi = {10.1109/TRO.2021.3071618}
}
```
Multirobot systems must be able to maintain performance when robots get delayed during execution. For mobile robots, one source of delays is congestion. Congestion occurs when robots deployed in shared physical spaces interact, as robots present in the same area simultaneously must maneuver to avoid each other. Congestion can adversely affect navigation performance and increase the duration of navigation actions. In this article, we present a multirobot planning framework that utilizes learnt probabilistic models of how congestion affects navigation duration. Central to our framework is a probabilistic reservation table, which summarizes robot plans, capturing the effects of congestion. To plan, we solve a sequence of single-robot time-varying Markov automata, where transition probabilities and rates are obtained from the probabilistic reservation table. We also present an iterative model refinement procedure for accurately predicting execution-time robot performance. We evaluate our framework with extensive experiments on synthetic data and simulated robot behavior.
C. Wang, S. Hasler, M. Mühlig, F. Joublin, A. Ceravola, J. Deigmöller, L. Fischer, and P. An, “Designing Interaction for Multi-agent Cooperative System in an Office Environment ,” in ACM/IEEE International Conference on Human-Robot Interaction, 2021.
DOI: 10.1145/3434074.3447277 | abstract |
```
@inproceedings{paper-wang-2021,
  title = {Designing Interaction for Multi-agent Cooperative System in an Office Environment },
  author = {Wang, Chao and Hasler, Stephan and M{\"u}hlig, Manuel and Joublin, Frank and Ceravola, Antonello and Deigm{\"o}ller, J{\"o}rg and Fischer, Lydia and An, Pengcheng},
  year = {2021},
  publisher = {ACM},
  doi = {10.1145/3434074.3447277},
  booktitle = {ACM/IEEE International Conference on Human-Robot Interaction}
}
```
Future intelligent system will involve various artificial agents, including mobile robots, smart home infrastructure or personal devices, which share data and collaborate with each other to serve users. Designing efficient interactions which can support users to express needs to such intelligent environments, supervise the collaboration of different entities and evaluate the outcomes, will be challengeable. This paper presents the design and implementation of the human-machine interface of Intelligent Cyber-Physical system (ICPS), which is a multi-entity coordination system of robots and other smart agents in a workplace (Honda Research Institute). ICPS gathers sensory data from entities and receives users’ inputs, then optimizes plans to utilize the capability of different entities to serve people.
C. Wang, S. Hasler, M. Mühlig, F. Joublin, A. Ceravola, J. Deigmöller, and L. Fischer, “Designing Interaction for Multi-agent System in an Office Environment,” in CHINESE CHI 2020, 2020.
DOI: 10.1145/3403676.3404349 | abstract |
```
@inproceedings{paper-wang-2020-2,
  title = {Designing Interaction for Multi-agent System in an Office Environment},
  author = {Wang, Chao and Hasler, Stephan and M{\"u}hlig, Manuel and Joublin, Frank and Ceravola, Antonello and Deigm{\"o}ller, J{\"o}rg and Fischer, Lydia},
  year = {2020},
  publisher = {ACM},
  doi = {10.1145/3403676.3404349},
  booktitle = {CHINESE CHI 2020}
}
```
Future intelligent system will involve very various types of artificial agents, such as mobile robots, smart home infrastructure, or personal devices, which share data and collaborate with each other to execute certain tasks. Designing an efficient human-machine interface, which can support users to express needs to the system, supervise the collaboration progress of different entities, and evaluate the result, will be challenging. This paper presents the design and implementation of the human-machine interface of the Intelligent Cyber-Physical system (ICPS), which is a multi-entity coordination system of robots and other smart devices in a working environment. ICPS gathers sensory data from entities and then receives users’ command, then optimizes plans to utilize the capability of different entities to serve people. Using multi-model interaction methods, e.g. graphical interfaces, speech interaction, gestures and facial expressions, ICPS is able to receive inputs from users through different entities, keep users aware of the progress and accomplish the task efficiently.
M. Mühlig, S. Hasler, L. Fischer, and J. Deigmöller, “A Knowledge-based Multi-entity and Cooperative System Architecture,” in ICHMS 2020: 1st IEEE International Conference on Human-Machine Systems, 2020.
DOI: 10.1109/ICHMS49158.2020.9209421 | abstract |
```
@inproceedings{paper-muehlig-2020,
  title = {A Knowledge-based Multi-entity and Cooperative System Architecture},
  author = {M{\"u}hlig, Manuel and Hasler, Stephan and Fischer, Lydia and Deigm{\"o}ller, J{\"o}rg},
  year = {2020},
  month = sep,
  publisher = {IEEE},
  doi = {10.1109/ICHMS49158.2020.9209421},
  booktitle = {ICHMS 2020: 1st IEEE International Conference on Human-Machine Systems}
}
```
Future intelligent systems will become more complex and they will be composed of potential very different artificial agents like mobile and static robots, and smartphones that collect \textellipsisdata and that have capabilities to perform certain tasks. We rely on the assumption that intelligent devices and humans have at least partially disjoint capabilities. Therefore it can be beneficial to combine intelligent devices and humans in order to perform a certain task. In this article we provide a description of our implementation of such a system operating in an office environment. Our system uses available resources (robots, humans) and the actual state of the environment to provide a plan in order to satisfy a given request. We demonstrate the flexibility of our system through varying the available resources and the state of the environment. To our knowledge there are few approaches only that (i) explicitly model and use humans together with intelligent agents in the same plan in this kind of abstraction level, and (ii) \textellipsisthat have a system using heterogeneous agents at the same time.
A. Sadik, C. Goerick, and M. Mühlig, “Modeling and Simulation of a Multi-Robot System Architecture,” in 2019 International Conference on Mechatronics, Robotics and Systems Engineering, 2020.
DOI: 10.1109/MoRSE48060.2019.8998662 | abstract |
```
@inproceedings{paper-sadik-2020,
  title = {Modeling and Simulation of a Multi-Robot System Architecture},
  author = {Sadik, Ahmed and Goerick, Christian and M{\"u}hlig, Manuel},
  year = {2020},
  publisher = {IEEE},
  booktitle = {2019 International Conference on Mechatronics, Robotics and Systems Engineering},
  editor = {IEEE},
  doi = {10.1109/MoRSE48060.2019.8998662},
  city = { Bali, Indonesia }
}
```
A Multi-Robot System (MRS) is the infrastructure of an intelligent cyber-physical system, where the robots understand the need of the human, and hence cooperate together to fulfill this need. Modeling the MRS is a crucial aspect of designing the proper system architecture, because this model can be used to simulate and measure the performance of the proposed architecture. However, an MRS architecture modeling is a very difficult problem, as it contains many dependent behaviors that dynamically change due to the current state of the system. In this paper, we introduce a general-purpose MRS case study, where the humans initiate requests that are achieved by the available robots. These requests require different plans that use the current capabilities of the available robots. After proposing an architecture that defines the solution components, three steps are followed. Firstly, modeling these components via a proper Architecture Description Language (ADL). Business Process Model and Notation (BPMN) is used as an ADL to represent the behaviors of every component, which is an essential need to model the solution. Secondly, simulating these components behaviors and interaction in form of software agents. Java Agent DEvelopment (JADE) middleware is used to develop the proposed model. JADE is based on a reactive agent approach, therefore it can simulate the dynamically interaction among the solution components. Finally, analyze the performance of the solution by defining a number of quantitative measurements, which can be obtained while simulating the system model, therefore the solution can be analyzed and compared to another architecture.
C. Wang, S. Hasler, M. Mühlig, F. Joublin, A. Ceravola, J. Deigmöller, and L. Fischer, “Designing Interaction for Multi-agent Cooperative System in an Office Environment,” in Arxiv, Arxiv, 2020.
abstract |
```
@incollection{paper-wang-2020-1,
  title = {Designing Interaction for Multi-agent Cooperative System in an Office Environment},
  author = {Wang, Chao and Hasler, Stephan and M{\"u}hlig, Manuel and Joublin, Frank and Ceravola, Antonello and Deigm{\"o}ller, J{\"o}rg and Fischer, Lydia},
  year = {2020},
  publisher = {Arxiv},
  booktitle = {Arxiv}
}
```
Future intelligent system will involve very various types of artificial agents, such as mobile robots, smart home infrastructure or personal devices, which share data and collaborate with each other to execute certain tasks. Designing an efficient human-machine interface, which can support users to express needs to the system, supervise the collaboration progress of different entities and evaluate the result, will be challengeable. This paper presents the design and implementation of the human-machine interface of Intelligent Cyber-Physical system (ICPS), which is a multi-entity coordination system of robots and other smart devices in a working environment. ICPS gathers sensory data from entities and then receives users’ command, then optimizes plans to utilize the capability of different entities to serve people. Using multi-model interaction methods, e.g. graphical interfaces, speech interaction, gestures and facial expressions, ICPS is able to receive inputs from users through different entities, keep users aware of the progress and accomplish the task efficiently.
C. Street, B. Lacerda, M. Mühlig, and N. Hawes, “Multi-Robot Planning Under Uncertainty with Congestion-Aware Models,” in Nineteenth International Conference on Autonomous Agents and Multi-Agent Systems , 2020.
DOI: 10.5555/3398761.3398913 | abstract |
```
@inproceedings{paper-street-2020,
  title = {Multi-Robot Planning Under Uncertainty with Congestion-Aware Models},
  author = {Street, Charlie and Lacerda, Bruno and M{\"u}hlig, Manuel and Hawes, Nick},
  year = {2020},
  month = may,
  publisher = {The University of Auckland},
  doi = {10.5555/3398761.3398913},
  booktitle = {Nineteenth International Conference on Autonomous Agents and Multi-Agent Systems }
}
```
When planning for multi-robot navigation tasks under uncertainty, plans should prevent robots from colliding while still reaching their goal. Solutions achieving this fall on a spectrum. At one end are solutions which prevent robots from being in the same part of the environment simultaneously at planning time, ignoring the robots’ capabilities to manoeuvre around each other, whilst at the other end are solutions that solve the problem at execution time, relying solely on these capabilities. Both approaches can lead to inefficient behaviour. In this paper, we present a novel framework in the middle of this spectrum that explicitly reasons over the effect the presence of multiple robots has on navigation performance. We refer to this effect as congestion. We present a structure, called the Probabilistic Reservation Table, which summarises the plans of robots, allowing us to probabilistically model congestion. We show how this structure can be used for planning by proposing an approach that, for each robot, sequentially builds and solves a Markov decision process where the transition probabilities are obtained by querying the Probabilistic Reservation Table. We carry out experiments on synthetic data and in simulation to show the effectiveness of our framework.
N. Einecke, S. Fuchs, B. Bolder, and M. Mühlig, “Living Lab: A 24/7 Human-Machine-Interaction Space in an Office Environment,” in Proceedings of the Poster and Workshop Sessions of AmI-2019, the 2019 European Conference on Ambient Intelligence, 2019, pp. 111–123.
abstract |
```
@inproceedings{paper-einecke-2019-2,
  title = {Living Lab: A 24/7 Human-Machine-Interaction Space in an Office Environment},
  author = {Einecke, Nils and Fuchs, Stefan and Bolder, Bram and M{\"u}hlig, Manuel},
  year = {2019},
  month = nov,
  publisher = {CEUR},
  booktitle = {Proceedings of the Poster and Workshop Sessions of AmI-2019, the 2019 European Conference on Ambient Intelligence},
  city = {Rome, Italy},
  pages = {111-123}
}
```
In this work, we present the SmartLobby, an intelligent environment system integrated into the lobby of a research institute. The SmartLobby is running 24/7, i.e. it can be used any time by anyone without any preparations. The goal of the system is to conduct research in the domain of human machine cooperation. One important first step towards this goal is a detailed human state modeling and estimation. As the system is built into the lobby area, also people having no scientific background are interacting with the system which strongly reduces the bias in the collected data. On software side the SmartLobby mainly integrates state-of-the-art algorithms that enable a thorough analysis of human behavior and state. These algorithms constitute the fundamental basis for the development of higher level system components. Here, we present our system with its various hardware and software components. Thereby, we focus on the realization of a non-intrusive head-eye-tracking as a key component to continuously observe persons using the system. A first multi-week lasting experiment utilizes this capability to investigate a method for content personalization, which demonstrates the effectiveness of the whole system.
N. Einecke, S. Fuchs, B. Bolder, and M. Mühlig, “Living Lab: A 24/7 Human-Machine-Interaction Space in an Office Environment,” in UbiComp/ISWC 2019 Adjunct, 2019.
abstract |
```
@inproceedings{paper-einecke-2019-1,
  title = {Living Lab: A 24/7 Human-Machine-Interaction Space in an Office Environment},
  author = {Einecke, Nils and Fuchs, Stefan and Bolder, Bram and M{\"u}hlig, Manuel},
  year = {2019},
  month = sep,
  publisher = {ACM },
  booktitle = {UbiComp/ISWC 2019 Adjunct}
}
```
In this work, we present a smart room which is integrated into the office environment of a research lab. The smart room is running 24/7 leading to the concept of a living lab, i.e. a lab that can be used any time by anyone without any preparations. Since the room is build into the lobby area of the research lab, also people having no expertise which such an environment can interact with the system.
C. H. Sung, M. Mühlig, M. Gienger, and Y. Uno, “Task-dependent Distribution and Constrained Optimization of Via-points for Smooth Robot Motions,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, Washington, USA, 2015.
DOI: 10.1109/ICRA.2015.7139693 | abstract |
```
@inproceedings{paper-sung-2015-1,
  address = {Seattle, Washington, USA},
  author = {Sung, ChangHyun and M\"{u}hlig, Manuel and Gienger, Michael and Uno, Yoji},
  booktitle = {2015 IEEE International Conference on Robotics and Automation (ICRA)},
  title = {{Task-dependent Distribution and Constrained Optimization of Via-points for Smooth Robot Motions}},
  doi = {10.1109/ICRA.2015.7139693},
  year = {2015}
}
```
This paper presents an effective method for planning and optimizing robot motions in joint space using viapoints. The via-point formulation allows for a sparse movement representation which is inherently smooth according to a minimum jerk criterion. In this research we focus on two aspects. First, we present an initialization method to find a feasible number of via-points and their distribution in time. This initialization takes the difficulty of the planning problem into account and finds a tradeoff between representation complexity (number of via-points) and optimization time. Second, we propose a two-step optimization approach to improve the viapoint parameters to satisfy constraints over the whole motion duration. We show that our methods generate feasible and smooth robot motions with a high success rate for tasks of varying complexity. The methods are evaluated in simulation using four different scenarios with two different robots, a 7-DOF robot arm and a 16-DOF humanoid upper body. Additionally, the experiments with the robot arm are shown in a real-world experiment.
M. Mühlig, A. Hayashi, M. Gienger, S. Iba, and T. Yoshiike, “Receding Horizon Optimization of Robot Motions generated by Hierarchical Movement Primitives,” in IEEE International Conference on Intelligent Robotics Systems (IROS 2014), Chicago, 2014.
DOI: 10.1109/IROS.2014.6942551 | abstract |
```
@inproceedings{paper-muehlig-2014-1,
  address = {Chicago},
  author = {M\"{u}hlig, Manuel and Hayashi, Akinobu and Gienger, Michael and Iba, Soshi and Yoshiike, Takahide},
  booktitle = {IEEE International Conference on Intelligent Robotics Systems (IROS 2014)},
  title = {{Receding Horizon Optimization of Robot Motions generated by Hierarchical Movement Primitives}},
  doi = {10.1109/IROS.2014.6942551},
  year = {2014}
}
```
This paper introduces a motion generation framework that integrates a hierarchical movement primitive (MP) layer with optimal control in form of receding horizon optimization. In order to benefit from fast reactions on the MP-layer, the optimal control layer can be overridden in risky situations to generate quick, though non-optimal solutions. By this, the system fulfills four desirable properties. It continuously adapts the robot’s motion without noticeable delay (1) by optimizing for collision and joint limit avoidance based on a future time horizon instead of the current state only (2). It accounts for the full robot motion that may result from multiple active MPs at the same time (3) and despite a possibly slow optimization still provides the robustness and quick reaction capabilities of MPs (4). The framework has been validated in an experiment in which a humanoid robot performed a task, optimized wrt. collisions and joint limit avoidance, but still could react within 50ms after detection of a potential risk.
A.-L. Vollmer, M. Mühlig, J. J. Steil, K. Pitsch, J. Fritsch, K. J. Rohlfing, and B. Wrede, “Robots Show Us How to Teach Them: Feedback from Robots Shapes Tutoring Behavior during Action Learning,” PLoS ONE, vol. 9, no. 3, p. e91349, Mar. 2014.
DOI: 10.1371/journal.pone.0091349 | abstract |
```
@article{paper-vollmer-2014-1,
  author = {Vollmer, Anna-Lisa and Mühlig, Manuel and Steil, Jochen J. and Pitsch, Karola and Fritsch, Jannik and Rohlfing, Katharina J. and Wrede, Britta},
  journal = {PLoS ONE},
  publisher = {Public Library of Science},
  title = {Robots Show Us How to Teach Them: Feedback from Robots Shapes Tutoring Behavior during Action Learning},
  year = {2014},
  month = mar,
  volume = {9},
  pages = {e91349},
  number = {3},
  doi = {10.1371/journal.pone.0091349}
}
```
Robot learning by imitation requires the detection of a tutor’s action demonstration and its relevant parts. Current approaches implicitly assume a unidirectional transfer of knowledge from tutor to learner. The presented work challenges this predominant assumption based on an extensive user study with an autonomously interacting robot. We show that by providing feedback, a robot learner influences the human tutor’s movement demonstrations in the process of action learning. We argue that the robot’s feedback strongly shapes how tutors signal what is relevant to an action and thus advocate a paradigm shift in robot action learning research toward truly interactive systems learning in and benefiting from interaction.
A.-L. Vollmer, M. Mühlig, K. Rohlfing, B. Wrede, and A. Cangelosi, “Demonstrating actions to a robot: How naîve users correct a robot’s replication of goal and manner-oriented actions,” in The 17th Workshop on the Semantics and Pragmatics of Dialogue (DialDam), 2013.
abstract |
```
@inproceedings{paper-vollmer-2013-1,
  title = {Demonstrating actions to a robot: How naîve users correct a robot's replication of goal and manner-oriented actions},
  author = {Vollmer, Anna-Lisa and M{\"u}hlig, Manuel and Rohlfing, Katharina and Wrede, Britta and Cangelosi, Angelo},
  year = {2013},
  month = dec,
  publisher = {University of Amsterdam},
  booktitle = {The 17th Workshop on the Semantics and Pragmatics of Dialogue (DialDam)},
  city = {Amsterdam}
}
```
Imitation learning in robotics aims at developing mechanisms for robots to learn from a human tutor’s demonstrations. In order to replicate an action correctly, the robot needs to understand a demonstrated action which implies knowing what is important about it. But how do tutor’s convey this information about an action? Recent research has proposed that in action learning the learner’s mental states are aligned to those of the tutor in a bi-directional process. Subscribing to this view, tutoring a robot should really be a bi-directional endeavour in which the robot not merely observes the tutor’s demonstrations, but actively takes part by giving feedback. In adult-infant tutoring interactions, the infant learner’s feedback reveals information about his/her state of attention and understanding and has been found to influence the adult tutor’s action demonstrations. In a human-robot interaction study where human tutor and robot learner take turns in demonstrating and replicating actions, we investigate how naı̈ve users interacting with a robot for the first time modify their action demonstrations to emphasize what is important about the action, when the robot replicates the action correctly or incorrectly.
K. Pitsch, A.-L. Vollmer, and M. Mühlig, “Robot feedback shapes the tutor’s presentation: How a robot’s online gaze strategies lead to micro-adaptation of the human’s conduct,” Interaction Studies, vol. 14, no. 2, pp. 268–295, 2013.
DOI: 10.1075/is.14.2.06pit | abstract |
```
@article{paper-pitsch-2013-1,
  author = {Pitsch, Karola and Vollmer, Anna-Lisa and M\"{u}hlig, Manuel},
  affiliation = {University of Bielefeld, Germany / University of Plymouth / Honda Research Institute Europe, Germany},
  title = {{Robot feedback shapes the tutor’s presentation: How a robot’s online gaze strategies lead to micro-adaptation of the human’s conduct}},
  journal = {Interaction Studies},
  publisher = {John Benjamins Publishing Company},
  issn = {1572-0373},
  pages = {268--295},
  volume = {14},
  issue = {2},
  doi = {10.1075/is.14.2.06pit},
  year = {2013}
}
```
The paper investigates the effects of a humanoid robot’s online feedback during a tutoring situation in which a human demonstrates how to make a frog jump across a table. Motivated by micro-analytic studies of adult-child-interaction, we investigated whether tutors react to a robot’s gaze strategies while they are presenting an action. And if so, how they would adapt to them. Analysis reveals that tutors adjust typical “motionese” parameters (pauses, speed, and height of motion). We argue that a robot – when using adequate online feedback strategies – has at its disposal an important resource with which it could proactively shape the tutor’s presentation and help generate the input from which it would benefit most. These results advance our understanding of robotic “Social Learning” in that they suggest a paradigm shift towards considering human and robot as one interational learning system.

T. Luksch, M. Gienger, M. Mühlig, and T. Yoshiike, “Adaptive Movement Sequences and Predictive Decisions based on Hierarchical Dynamical Systems,” in 25th IEEE/RSL International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, 2012.

@inproceedings{paper-luksch-2012-2,
  author = {Luksch, Tobias and Gienger, Michael and M\"{u}hlig, Manuel and Yoshiike, Takahide},
  title = {{Adaptive Movement Sequences and Predictive Decisions based on Hierarchical Dynamical Systems}},
  booktitle = {{25th IEEE/RSL International Conference on Intelligent Robots and Systems (IROS)}},
  year = {2012},
  address = {Vilamoura, Portugal},
  month = oct
}

T. Luksch, M. Gienger, M. Mühlig, and T. Yoshiike, “A Dynamical Systems Approach to Adaptive Sequencing of Movement Primitives,” in Proceedings of the 7th German Conference on Robotics (ROBOTIK), Munich, Germany, 2012.
abstract |
```
@inproceedings{paper-luksch-2012-1,
  author = {Luksch, Tobias and Gienger, Michael and M\"{u}hlig, Manuel and Yoshiike, Takahide},
  title = {{A Dynamical Systems Approach to Adaptive Sequencing of Movement Primitives}},
  booktitle = {{Proceedings of the 7th German Conference on Robotics (ROBOTIK)}},
  year = {2012},
  address = {Munich, Germany},
  month = may
}
```
This paper introduces a control concept for motion generation of redundant robots based on combinations of movement primitives (MP). It addresses the question of how to create continuous and smooth sequences of actions or transitions between different motion skills while avoiding the necessity of recurrent planning. MPs are defined on task coordinates and modeled as dynamical systems with attractor behavior featuring additional signals to ease their coordination. Sequences and transitions between skills are realized in a unified way as bifurcating dynamical systems based on continuous-time recurrent neural networks. The neural output is used as activation signal for MPs. It is shown how the parameters of these dynamical systems can be chosen to generate a desired behavior. First results are shown in a physical simulation environment on a high-DoF robot with human-like upper body. The system can create smooth transients of MPs in sequences as well as during changes of strategies, notably showing more than only local adaptation capabilities.
M. Mühlig, M. Gienger, and J. J. Steil, “Interactive imitation learning of object movement skills,” Autonomous Robots, vol. 32, no. 2, pp. 97–114, 2012.
DOI: 10.1007/s10514-011-9261-0 | abstract |
```
@article{paper-muehlig-2012-1,
  author = {M\"{u}hlig, Manuel and Gienger, Michael and Steil, Jochen J.},
  affiliation = {Honda Research Institute Europe, Carl-Legien-Strasse 30, 63073 Offenbach/Main, Germany},
  title = {{Interactive imitation learning of object movement skills}},
  journal = {Autonomous Robots},
  publisher = {Springer Netherlands},
  issn = {0929-5593},
  keyword = {Computer Science},
  pages = {97-114},
  volume = {32},
  issue = {2},
  doi = {10.1007/s10514-011-9261-0},
  year = {2012}
}
```
In this paper we present a new robot control and learning system that allows a humanoid robot to extend its movement repertoire by learning from a human tutor. The focus is learning and imitating motor skills to move and position objects. We concentrate on two major aspects. First, the presented teaching and imitation scenario is fully interactive. A human tutor can teach the robot which is in turn able to integrate newly learned skills into different movement sequences online. Second, we combine a number of novel concepts to enhance the flexibility and generalization capabilities of the system. Generalization to new tasks is obtained by decoupling the learned movements from the robot’s embodiment using a task space representation. It is chosen automatically from a commonly used task space pool. The movement descriptions are further decoupled from specific object instances by formulating them with respect to so-called linked objects. They act as references and can interactively be bound to real objects. When executing a learned task, a flexible kinematic description allows to change the robot’s body schema online and thereby apply the learned movement relative to different body parts or new objects. An efficient optimization scheme adapts movements to such situations performing online obstacle and self-collision avoidance. Finally, all described processes are combined within a comprehensive architecture. To demonstrate the generalization capabilities we show experiments where the robot performs a movement bimanually in different environments, although the task was demonstrated by the tutor only one-handed.
N. Einecke, M. Mühlig, J. Schmüdderich, and M. Gienger, “‘Bring it to me’ – Generation of Behavior-Relevant Scene Elements for Interactive Robot Scenarios,” in 2011 IEEE International Conference on Robotics and Automation (ICRA), 2011.
DOI: 10.1109/ICRA.2011.5979952 | abstract |
```
@inproceedings{paper-einecke-2011-1,
  author = {Einecke, Nils and M\"{u}hlig, Manuel and Schm\"{u}dderich, Jens and Gienger, Michael},
  booktitle = {{2011 IEEE International Conference on Robotics and Automation (ICRA)}},
  title = {{"Bring it to me" -- Generation of Behavior-Relevant Scene Elements for Interactive Robot Scenarios}},
  year = {2011},
  doi = {10.1109/ICRA.2011.5979952}
}
```
Humanoid robots are intended to act and interact in dynamically changing environments in the presence of humans. Current robotic systems are usually able to move in dynamically changing environments because of an inbuilt depth and obstacle sensing. However, for acting in their environment the internal representation of such systems is usually constructed by hand and known in advance. In contrast, this paper presents a system that dynamically constructs its internal scene representation using a model-based vision approach. This enables our system to approach and grasp objects in an previously unknown scene. We combine standard stereo with model-based image fitting techniques for a real-time estimation of the position and orien- tation of objects. The model-based image processing allows for an easy transfer to the internal, dynamic scene representation. For movement generation we use a task-level whole-body control approach that is coupled with a movement optimization scheme. Furthermore, we present a novel method that constrains the robot to keep certain objects in the FOV while moving.We demonstrate the successful interplay between model-based vision, dynamic scene representation, and movement generation by means of some interactive reaching and grasping tasks.
M. Mühlig, M. Gienger, and J. J. Steil, “Human-Robot Interaction for Learning and Adaptation of Object Movements,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 4901–4907.
DOI: 10.1109/IROS.2010.5649229 | abstract |
```
@inproceedings{paper-muehlig-2010-1,
  author = {M\"{u}hlig, Manuel and Gienger, Michael and Steil, Jochen J.},
  booktitle = {{2010 IEEE/RSJ International Conference on Intelligent Robots and Systems}},
  pages = {4901--4907},
  title = {{Human-Robot Interaction for Learning and Adaptation of Object Movements}},
  year = {2010},
  doi = {10.1109/IROS.2010.5649229}
}
```
In this paper we present a new robot control and learning framework. By integrating previously presented as well as new methods, the robot is able to learn an invariant and generic movement representation from a human tutor. We argue that in order to apply such generic representations to new situations and thus create a flexible system, the use of interaction is beneficial. The interaction is based on a kinematically controlled model of a human tutor, which is used as a model-based filter and also for recognizing postures that influence the interaction. In addition, a new movement segmentation scheme is presented that is based on correlating movements by the tutor’s hand with the salient objects in the scene. The focus of this paper is on the interactive learning aspects of the system and particular emphasis is given to an experiment in which the humanoid robot ASIMO learns from a human tutor. The system includes extensive generalization capabilities that result from an online adaption of the robot’s body schema and the exploitation of inter-trial variance from multiple demonstrations. This enables the robot to reproduce the movement in new situations. For example, a stacking task that the tutor performed one-handed can be executed bi- manually by the robot.
M. Gienger, M. Mühlig, and J. J. Steil, “Imitating Object Movement Skills with Robots - A Task-Level Approach Exploiting Generalization and Invariance,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 1262–1269.
DOI: 10.1109/IROS.2010.5649990 | abstract |
```
@inproceedings{paper-gienger-2010-1,
  author = {Gienger, Michael and M\"{u}hlig, Manuel and Steil, Jochen J.},
  booktitle = {{2010 IEEE/RSJ International Conference on Intelligent Robots and Systems}},
  pages = {1262--1269},
  title = {{Imitating Object Movement Skills with Robots - A Task-Level Approach Exploiting Generalization and Invariance}},
  year = {2010},
  doi = {10.1109/IROS.2010.5649990}
}
```
This paper presents an architecture for learning and reproducing movements with a robot in interaction with a human teacher. We focus on the movement representation and propose three enhancements to increase generalization capabilities: Firstly, we introduce a flexible task-level movement representation that is based on neuropsychological findings. Movement is represented in task-oriented frames of reference, and generalizes to a variety of different situations. Secondly, we propose a mechanism to decouple the task descriptors from the perceived objects in the robot’s environment. This allows to formulate a set of generic controllers, and to interactively create associations with perceived objects. Thirdly, we introduce a method to dynamically modify the system’s body schema to account for structural changes such as having grasped a tool. The changes are consistently treated in the kinematics computations. This permits to generalize movements to be carried out in different ways, for instance with different hands or bi-manually. A set of experiments in an interactive imitation learning situation underline the capabilities of the proposed concepts. I.
M. Rudolph, M. Mühlig, M. Gienger, and H.-J. Böhme, “Learning the Consequences of Actions: Representing Effects as Feature Changes,” in 2010 International Symposium on Learning and Adaptive Behavior in Robotic Systems, 2010.
DOI: 10.1109/EST.2010.9 | abstract |
```
@inproceedings{paper-rudolph-2010-1,
  author = {Rudolph, Mathias and M\"{u}hlig, Manuel and Gienger, Michael and B\"{o}hme, Hans-Joachim},
  booktitle = {{2010 International Symposium on Learning and Adaptive Behavior in Robotic Systems}},
  title = {{Learning the Consequences of Actions: Representing Effects as Feature Changes}},
  year = {2010},
  doi = {10.1109/EST.2010.9}
}
```
In advanced Programming by Demonstration (PbD) it is important to give a robot the ability to understand the effects of an action. This ability can enable a robot to not only mimic an action but to imitate, by determining whether an action succeeded or not, or to emulate, by finding another action that causes the same effects as observed. In this paper we propose a system that uses a Bayesian Network structure to store actions as a representation of their effects. The effects in turn are implicitly stored as representation of feature changes in the perceived environment. In a more general form the system can be used to differentiate between actions. In a more specific form it can be used to learn complex mapping functions. We will show three different experiments. The first one shows how to learn actions as a representation of effects. The second one shows how a our system can be used to learn a complex mapping function on robot movement and in the third experiment, we illustrates how to combine these independently learned systems to achieve more complex tasks. I.
M. Mühlig, M. Gienger, J. J. Steil, and C. Goerick, “Automatic Selection of Task Spaces for Imitation Learning,” in 2009 IEEE International Conference on Intelligent Robots and Systems, 2009, pp. 4996–5002.
DOI: 10.1109/IROS.2009.5353894 | abstract |
```
@inproceedings{paper-muehlig-2009-2,
  author = {M\"{u}hlig, Manuel and Gienger, Michael and Steil, Jochen J. and Goerick, Christian},
  booktitle = {{2009 IEEE International Conference on Intelligent Robots and Systems}},
  pages = {4996--5002},
  title = {{Automatic Selection of Task Spaces for Imitation Learning}},
  year = {2009},
  doi = {10.1109/IROS.2009.5353894}
}
```
Previous work shows that the movement rep- resentation in task spaces offers many advantages for learn- ing object-related and goal-directed movement tasks through imitation. It allows to reduce the dimensionality of the data that is learned and simplifies the correspondence problem that results from different kinematic structures of teacher and robot. Further, the task space representation provides a first generalization, for example wrt. differing absolute positions, if bi-manual movements are represented in relation to each other. Although task spaces are widely used, even if they are not mentioned explicitly, they are mostly defined a priori. This work is a step towards an automatic selection of task spaces. Observed movements are mapped into a pool of possibly even conflicting task spaces and we present methods that analyze this task space pool in order to acquire task space descriptors that match the observation best. As statistical measures cannot explain importance for all kinds of movements, the presented selection scheme incorporates additional criteria such as an attention-based measure. Further, we introduce methods that make a significant step from purely statistically-driven task space selection towards model-based movement analysis using a simulation of a complex human model. Effort and discomfort of the human teacher is being analyzed and used as a hint for important task elements. All methods are validated with real- world data, gathered using color tracking with a stereo vision system and a VICON motion capturing system.
M. Mühlig, M. Gienger, S. Hellbach, J. J. Steil, and C. Goerick, “Task-level Imitation Learning using Variance-based Movement Optimization,” in 2009 IEEE International Conference on Robotics and Automation, 2009, pp. 1177–1184.
DOI: 10.1109/ROBOT.2009.5152439 | abstract |
```
@inproceedings{paper-muehlig-2009-1,
  author = {M\"{u}hlig, Manuel and Gienger, Michael and Hellbach, Sven and Steil, Jochen J. and Goerick, Christian},
  booktitle = {{2009 IEEE International Conference on Robotics and Automation}},
  issn = {1050-4729},
  pages = {1177--1184},
  publisher = {IEEE},
  title = {{Task-level Imitation Learning using Variance-based Movement Optimization}},
  year = {2009},
  doi = {10.1109/ROBOT.2009.5152439}
}
```
Recent advances in the field of humanoid robotics increase the complexity of the tasks that such robots can perform. This makes it increasingly difficult and inconvenient to program these tasks manually. Furthermore, humanoid robots, in contrast to industrial robots, should in the distant future behave within a social environment. Therefore, it must be possible to extend the robot’s abilities in an easy and natural way. To address these requirements, this work investigates the topic of imitation learning of motor skills. The focus lies on providing a humanoid robot with the ability to learn new bi- manual tasks through the observation of object trajectories. For this, an imitation learning framework is presented, which allows the robot to learn the important elements of an observed movement task by application of probabilistic encoding with Gaussian Mixture Models. The learned information is used to initialize an attractor-based movement generation algorithm that optimizes the reproduced movement towards the fulfillment of additional criteria, such as collision avoidance. Experiments performed with the humanoid robot ASIMO show that the proposed system is suitable for transferring information from a human demonstrator to the robot. These results provide a good starting point for more complex and interactive learning tasks.

Books (2)

M. Mühlig, “A Whole Systems Approach to Robot Imitation Learning of Object Movement Skills,” PhD thesis, Shaker, 2011 [Online]. Available at: http://www.shaker.de/shop/978-3-8440-0489-2
abstract |
```
@phdthesis{thesis-muehlig-2011-1,
  author = {M\"{u}hlig, Manuel},
  title = {{A Whole Systems Approach to Robot Imitation Learning of Object Movement Skills}},
  school = {Bielefeld University},
  year = {2011},
  month = sep,
  isbn = {978-3-8440-0489-2},
  publisher = {Shaker},
  url = {http://www.shaker.de/shop/978-3-8440-0489-2}
}
```
Imitation learning has become a popular paradigm to extend the abilities of robots by demonstrating new skills. Many methods have been proposed that allow a robot to detect such demonstrations and to learn from them. However, there is a significant drawback with state-of-the-art imitation learning approaches. Most of them consider imitation learning of object movement skills as an isolated, datadriven method for learning object trajectories. They neglect much of the information that is provided by the human interaction partner and therefore miss many opportunities for increasing the generalization capabilities. The work at hand addresses this drawback by proposing a whole systems view on imitation learning. The main contribution of this thesis is an architecture for interactive imitation learning of object movement skills. Its purpose is to enable learning skills from only a few demonstrations, but still to be able to extensively generalize to new situations. This is achieved by various methods. Firstly, a probabilistic learning scheme in combination with movement optimization is suggested. It allows to exploit the variance information from multiple demonstrations. While imitating, the robot can diverge from variant parts of the movement to respect additional criteria regarding the robot’s limits. Furthermore, a novel method for automatically selecting skill-dependent task spaces is presented. These task spaces represent a skill in relative coordinates of specific object feature points, such as their top or bottom sides. That way, the learned skill is decoupled from specific objects and from the robot’s embodiment. In particular, it enables the robot to perform a skill in different ways, such as one-handed or bimanually. This is achieved by introducing the concept of a dynamic body schema. All of the presented methods respect that learning is performed in interaction with a human tutor. The tutor is modelled by the system, which allows to detect certain postures for instructing the robot. Additionally, the model is used to estimate the non-measurable internal state of the tutor, like the effort or discomfort of certain poses. This allows to deduce skill-relevance of specific phases of a demonstrated movement. The presented system also comprises an attention mechanism, which is directly coupled to the robot control scheme using the novel concept of linked objects. Consequently, the tutor can highlight relevant objects, from which the robot either learns or to which it applies a learned skill. All of the proposed methods are not presented in isolation. Instead, the thesis emphasizes the whole systems view by integrating them into a consistent architecture. The generalization capabilities of this architecture go beyond the state of the art, which is validated by several experiments. For instance, one experiment shows that a child-sized humanoid robot with 26 degrees of freedom is able to learn the skill of stacking objects. In addition to imitating the skill as demonstrated, the robot is able to generalize it to different objects, situations that contain obstacles, and to a bimanual performance. Even more, the skill learned by the humanoid robot can also be reproduced by other robots.
M. Mühlig, “Task Learning of Bimanual Object Handling,” Master's thesis, Ilmenau University of Technology, 2008.
abstract |
```
@mastersthesis{thesis-muehlig-2008-1,
  author = {M\"{u}hlig, Manuel},
  title = {{Task Learning of Bimanual Object Handling}},
  school = {Ilmenau University of Technology},
  year = {2008},
  month = apr
}
```
Due to recent advances in the field of humanoid robotics, the complexity of the tasks that those robots can perform rises significantly. This makes it increasingly difficult and inconvenient to program these tasks manually. Furthermore, humanoid robots, in contrast to industrial robots, should in the distant future act within a social environment. Therefore, it must be possible to extend the robot’s abilities in an easy and natural way. To address these requirements, this work investigates the topic of imitation learning of motor skills. The focus of this work lies on providing the humanoid robot ASIMO with the ability to learn new bi-manual tasks through the observation of object trajectories. For this, an imitation learning framework is presented which allows the robot to learn the essence of an observed movement task by application of probabilistic encoding with Gaussian Mixture Models. The learned information is used to initialize an attractor-based movement generation algorithm which optimizes the reproduced movement towards the fulfillment of additional criteria, such as collision avoidance. During this optimization, the variance information from multiple demonstrations is used as an importance measure. This allows the robot to diverge from unimportant task elements if other criteria are more crucial. Experiments performed with a simulation of the humanoid robot ASIMO show that the proposed system is suitable for transferring information from a human demonstrator to the robot. These results provide a good starting point for more complex and interactive learning tasks. Furthermore, it is shown that the cost function, which is used during the movement optimization, provides a reasonable quantitative measure of the robot’s imitation performance.

Patents (3)

M. Mühlig, M. Gienger, and A. Hayashi, “Method for improving operation of a robot.” Patent, 05-Oct-2016.

@misc{patent-muehlig-2016,
  author = {Mühlig, Manuel and Gienger, Michael and Hayashi, Akinobu},
  title = {Method for improving operation of a robot},
  year = {2016},
  month = oct,
  day = {05},
  note = {EP 3075496 A1},
  version = {A1},
  howpublished = {Patent},
  nationality = {EP},
  filing_num = {15162447.5},
  yearfiled = {2015},
  monthfiled = {April},
  dayfiled = {02},
  pat_refs = {},
  us_class = {}
}

M. Gienger and M. Mühlig, “Robot with hand-object movement correlations for online temporal segmentation of movement tasks.” Patent, 02-Nov-2011.

@misc{patent-gienger-2011-2,
  author = {Gienger, Michael and Mühlig, Manuel},
  title = {Robot with hand-object movement correlations for online temporal segmentation of movement tasks},
  year = {2011},
  month = nov,
  day = {02},
  note = {EP 2366502 B1},
  version = {B1},
  howpublished = {Patent},
  nationality = {EP},
  filing_num = {10154799.0},
  yearfiled = {2010},
  monthfiled = {February},
  dayfiled = {26},
  pat_refs = {},
  ipc_class = {B25J   9/16},
  us_class = {}
}

M. Gienger, M. Mühlig, and J. J. Steil, “Robot with automatic selection of task-specific representations for imitation learning.” Patent, 19-Jan-2011.

@misc{patent-gienger-2011-1,
  author = {Gienger, Michael and Mühlig, Manuel and Steil, Jochen J.},
  title = {Robot with automatic selection of task-specific representations for imitation learning},
  year = {2011},
  month = jan,
  day = {19},
  note = {EP 2224303 B1},
  version = {B1},
  howpublished = {Patent},
  nationality = {EP},
  filing_num = {09153866.0},
  yearfiled = {2009},
  monthfiled = {February},
  dayfiled = {27},
  pat_refs = {EP-A- 1 074 352; EP-A- 1 537 959; US-A1- 2007 239 641},
  ipc_class = {G05B  19/42}
}

Misc (4)

M. Mühlig, M. Gienger, and J. J. Steil, “Goal-directed imitation learning from humans.” Poster at Robot Learning Summer School 2009, Lisboa, Portugal, Jul-2009.

@misc{poster-muehlig-2009-1,
  author = {M\"{u}hlig, Manuel and Gienger, Michael and Steil, Jochen J.},
  title = {{Goal-directed imitation learning from humans}},
  year = {2009},
  month = jul,
  howpublished = {Poster at Robot Learning Summer School 2009, Lisboa, Portugal}
}

M. Mühlig, “Echtzeitfähige Umsetzung der visuellbasierten Bestimmung objektspezifischer Flussfelder.” Studienarbeit, 2007.
abstract |
```
@misc{misc-muehlig-2007-2,
  author = {M\"{u}hlig, Manuel},
  title = {{Echtzeitf\"{a}hige Umsetzung der visuellbasierten Bestimmung objektspezifischer Flussfelder}},
  year = {2007},
  howpublished = {Studienarbeit}
}
```
Diese Studienarbeit beschäftigt sich mit der echtzeitfähigen Implementierung des in der Diplomarbeit [Lau, 2007] vorgestellten Ansatzes zur Bewegungssegmentierung. Um dieses Ziel zu erreichen, werden Möglichkeiten untersucht, Teile des Gesamtsystems zu parallelisieren und mit Hilfe von NVIDIA CUDA effizient auf der Grafikkarte zu berechnen. Weitere Schwerpunkte sind die Integration in die ROBOTS Softwarearchitektur des Fachgebiets Neuroinformatik und Kognitive Robotik der TU-Ilmenau sowie die Verwendungsmöglichkeit der Implementierung in verschiedenen Hardwareumgebungen. Das Kapitel 2 beschreibt zunächst den Aufbau des zu implementierenden Systems sowie den Ablauf der Segmentierung. Da bei der Implementierung auf parallele Architekturen zurückgegriffen wird, erläutert Abschnitt 3 die Grundlagen die dafür nötig sind. Der Inhalt der Implementierung sowie Ergebnisse und Probleme werden in Kapitel 4 beschrieben. Kapitel 5 liefert schließlich eine Zusammenfassung der Arbeit sowie mögliche Verbesserungen des Systems.
M. Mühlig, “Literaturrecherche zu Verfahren für die Detektion von Kanten in Farbbildern.” Hauptseminararbeit, 2007.
abstract |
```
@misc{misc:Muehlig:2007:1,
  author = {M\"{u}hlig, Manuel},
  title = {{Literaturrecherche zu Verfahren f\"{u}r die Detektion von Kanten in Farbbildern}},
  year = {2007},
  howpublished = {Hauptseminararbeit}
}
```
Die Kantendetektion ist einer der wichtigsten Bestandteile der Bildverarbeitung. Viele Methoden der Bildverarbeitung bauen auf der Kantendetektion auf, so dass ihre Genauigkeit und Geschwindigkeit einen direkten Einfluss haben. Während die Kantendetektion in Grauwertbildern ein intensiv erforschter Bereich ist, trifft dies auf die Kantendetektion in Farbbilder noch nicht zu. Diese Hauptseminararbeit gibt einen Überblick über gängige Verfahren der Kantendetektion in Farbbildern. Dazu werden die verfügbaren Möglichkeiten systematisiert und in Hinblick auf Vorteile bezüglich der Kantendetektion in Grauwertbildern untersucht. Kapitel 2 befasst sich zunächst mit den Grundlagen. Es wird erläutert welche Unterschiede zur klassischen Kantendetektion in Grauwertbildern existieren und welche Vorteile man sich verspricht. Anschließend daran werden in Kapitel 3 ausgewählte Verfahren vorgestellt und bewertet.
M. Mühlig, “Untersuchung der Möglichkeiten fraktaler Features zur Bildanalyse.” Hauptseminararbeit, 2006.
abstract |
```
@misc{misc-muehlig-2006-1,
  author = {M\"{u}hlig, Manuel},
  title = {{Untersuchung der M\"{o}glichkeiten fraktaler Features zur Bildanalyse}},
  year = {2006},
  howpublished = {Hauptseminararbeit}
}
```
Die Bildanalyse bildet einen wichtigen Aspekt der Bildverarbeitung im Allgemeinen und umfasst einige grundlegende Methoden um Aussagen über die Struktur von Bildern zu treffen. Da diese Methoden in ihrer Art sehr unterschiedlich sind, kann die Granularität und die subjektive Bedeutung der Aussagen stark variieren. Zum einen sind ganzheitliche Aussagen wie z.B. über die Farbe, Intensität oder Verteilung von Bildpunkten, zum anderen aber auch strukturelle Betrachtungen der Teilelemente von Bilddaten (z.B. Texturierung, Dichte, Rauheit) möglich. Klassische Verfahren der Bildanalyse decken bereits einen großen Teil dieser Fragestellungen ab. Es gibt jedoch auch scheinbar vielversprechende alternative Ansätze, die sich mit der Nutzung von Fraktalen in der Bildanalyse beschäftigen. Diese Hauptseminararbeit liefert einen Überblick über Verfahren, die sich mit der Nutzung fraktaler Features in der Bildanalyse beschäftigen. Im ersten Teil werden zunächst die Grundlagen von Fraktalen erläutert, welche für das Verständnis ihrer Nutzung unabdingbar sind. Dazu gehören auch die fraktale Dimension sowie iterierte Funktionen-Systeme (IFS). Anschließend werden einige Verfahren der Bildanalyse mit fraktalen Features näher vorgestellt und bewertet.