Every agent starts life with significant structure, our objective is to understand what kind of structure is required, and how it develops into conceptual knowledge. The components of this prior structure are called physical primitives. The choice and representation of physical primitives is important for many reasons, not the least of which is that they are the field on which the nativist/constructivist debate plays out (see Section 1.1).
The design of physical primitives is well-motivated on psychological and engineering grounds. The agent's most primitive actions will be implemented in the form of closed-loop controllers that are asymptotically stable and that produce equilibria that are correct solutions to generic subtasks. This decision is consistent with current research in infant motor development and adult motor control [29][25][120][119] and robotics [47][110][112].
Modeling primitive actions as controllers has three related consequences. First, controllers suppress local perturbations by virtue of their closed-loop structure. Some variations in the context of a control task are simply rejected by the action of the controller. Second, controllers provide a basis for abstraction. Instead of dealing with a continuous state space, a behavioral scheme need only worry about control activation and convergence events. The result is a discrete model of controller/world interactions. Third, time is abstracted; in particular, durations of actions are not prescribed. This is appropriate, because the duration of a physical schema depends on the task and the context.
Using higher level primitives dramatically reduces the complexity of behavioral representations but preserves flexibility. In our model, behavior is constructed as a sequence of controllers bound to system resources. When a control objective is met for a particular resource commitment, a predicate is asserted in an abstract model of the system behavior. The ``state'' of the system is a vector of such predicates each element of which asserts a possible property of convergence for some controller and resource combination. This predicate vector can be viewed as the universe of possible subgoals available to an agent with these native controllers.
This representation provides a complete description of all physically plausible behavioral states and the control transitions between them. Having assessed its state in the predicated space, an agent can use such a representation to enumerate the range of sensory and motor alternatives available from this state. Even one ply of this local ``behavior graph'' might be huge in practice, so additional guidance is required and a variety of formal techniques can be used to provide that guidance. If we view the structure as a Markov Decision Process (MDP), then we may apply a variety of reinforcement learning techniques in order to derive behavioral policies (Section 1.2.4). If we view the structure as a Discrete Event Dynamic System (DEDS) then there exist tools for analyzing and interacting with the processes for acquiring behavior. DEDS provide a means of proving that certain predicate states can't occur and as a consequence, a means of investigating the role of constraints as ``bootstraps'' for a learning system. For example, shaping and maturational mechanisms can be approximated by constraining exploration and/or biasing the behavior acquisition process.