The reason NMF discovers strokes as the functional parts of digits is because the nonnegativity constraints allow it to learn from positive coactivation of image pixels in the data. The NMF algorithm adapts the basis W in order to learn the appropriate coactivations. In constrast, PCA has no constraints on the sign of the activation, and learns simultaneously from both positive and negative activations in the data. It should be noted that biological neural networks may use similar types of constraints to achieve analogous representations. The firing rates of neurons cannot be negative, and the strengths of synapses do not generally change sign. These one-sided constraints could possibly be important in developing the sparsely, distributed coding of sensory input that give rise to robust biological information processing.
The NMF algorithm is also generally applicable to problems outside the domain of image analysis. We have also applied it to the semantic analysis of text documents [Salton,Landauer], as well as to the analysis of routing patterns in data networks. In each of these cases, the algorithm learns to decompose the input data into their constituent parts. This enables any additional processing of the data to be robust against perturbations that can change only a small number of these features [Biederman]. Our current research in this area involves incorporating the NMF parts decomposition of the input data into a hierarchical representation that would be appropriate for high level control of systems such as our robot. Along with future improvements in sensory, motor, communications, and processing hardware, advances in these new learning algorithms will hopefully allow artificial embedded systems to someday exhibit the computational complexity and robustness found in biological systems.