PDP Approach in General

Parallel Distributed Processing (PDP) approach is a relatively new way to study psychological phenomena compared to more traditional formalization of human cognition. Whereas most conventional psychological theories postulate serial-ordered mechanisms to account for various aspects of human cognition, the PDP approach assumes that people understand through the interplay of multiple sources of knowledge, and as such, parts of the mechanism interact with each other simultaneously. Specifically, PDP models propose sets of large number of inter-connected information processing units as the mechanistic accounts of human cognitive phenomena. The units stand for conceptual objects (such as features, letters, words, etc.) or abstrct elements and so each contains certain aspect of the information. They influence other aspects and at the same time are influenced by them. Information processing takes place through the interaction among these units.

Figure 1 illustrates the basic components of a PDP network system. A typical PDP model begins with a set of processing units. At each point in time, each unit has an activation state, and generates an output according to a particular threshold function. Units are connected to one another to form a pattern of connectivity. Each connection between two units carries a weight that specifies how the output of the first unit feeds into the second unit as input. A connection can be either excitatory if the weight is a positive number, or inhibitory if the weight is a negative number. The absolute value of the weight, however, decides the strength of the connection. Because a unit receives input from a number of other units, a propagation rule is applied to determine the overall input to the unit. This net input, together with the current activation state of the unit, are then combined to produce a new state of activation according to a certain activation rule. Finally, connection weights undergo modification with experience. Thus the system can evolve by changing the pattern of connectivity.

Figure 1: The basic components of a PDP network

The PDP approach contrasts the conventional modeling of human cognition on two important issues. First, in PDP models what is stored is the pattern of connectivity (connection strengths) between units. Hence knowledge is represented by the pattern of activity distributed over many processing units. This distributed representation contrasts the one-unit-one-concept representational system in conventional psychological theories. Secondly, in the PDP approach, each processing unit in the system acts on and is simultaneously acted on by the other units. Computation takes the form of cooperative and competitive interactions among large numbers of processing units. Information processing happens in a parallel fashion. There is no distinctive processing stage as proposed in many conventional models.

The PDP approach has certain appeals among cognitive psychologists. The mechanisms it proposes to account for various aspects of human cognition, such as perception, reading, learning, memory, etc., are computationally sufficient and, to a certain degree, psychologically accurate. The current simulation is designed to demonstrate how the PDP approach can be applied to the study of human language.