Figure 1: Native Structure, Learning, and Development in an Integrated Robot Learning Architecture.
In the Dynamic Modeling component of Figure 1, events are changes in the pattern of membership in a set of prototypical models of dynamic behavior. Control decisions happen when one or more of the working grasp controllers produces such an event. Models describing histories of observations are accumulated as the robot interacts with the task over time and the pattern of membership in these models and the controllers themselves constitute states and actions in a Semi-Markov Decision Problem (SMDP).
This approach allows the expansion of the representation into regions where important information is hidden from instantaneous observers or is too expensive to observe. The Feature Learning component of Figure 1 is used to construct robust predictive state information.
The Discrete Event Dynamic System (DEDS) model illustrated in Figure 1 restricts the range of interactions permitted with the environment to those that meet design specifications. Together with the reward function, the DEDS model focuses exploration on the horizon of available control knowledge in order to build internal representations for important (sub)tasks. Although biological development can mean lots of things, in our framework, development for robot systems means DEDS control of the actions and states in an increasingly less restrictive sequence to shape behavior incrementally.
The proposed adaptive controller is designed to acquire strategies for control and control knowledge simultaneously through extended and cumulative interactions with an open set of tasks. The Laboratory for Perceptual Robotics is examining this architecture in the context of reaching, grasping, and manipulation tasks, walking robots, distributed, and multi-robot systems including arrays of networked mobile sensor systems.