Markovian Bias in Linguistic Computation via State Space Trajectories

Peter Tino

Abstract

The talk will concentrate on some interesting properties of non-autonomous-dynamical-system-based linguistic computation on fractal substrate. In particular, if the dynamical maps are contractions, clusters of trajectory points correspond to Markovian prediction contexts. By applying traditional knowledge extraction methods one can construct predictive models corresponding to a class of Markov models, called variable memory length Markov models (VLMM). Since widely used types of recurrent networks networks are often initialized as contractive  (Lipschitz continuous) systems, VLMM should be employed as the null hypothesis against which
the amount of induced knowledge should be tested. I will also sketch two different outlooks at  such situations:

1. claims can be proved in the framework of statistical learning theory showing the advantage of starting learning from simple underlying dynamics governed by attractive fixed points of  the contractive non-autonomous system (a sort of Ockham's razor)
2. a rigorous fractal analysis of state space trajectories can be performed,  showing a straightforward relation between complexity of the input symbol stream (topological entropy of the language) and geometric complexity of the state trajectories (fractal dimension).