Mildly context sensitive grammar formalism

Background

By 1985, several researchers in descriptive and mathematical linguistics had provided evidence against the hypothesis that the syntactic structure of natural language can be adequately described by context-free grammars. At the same time, the step to the next level of the Chomsky hierarchy, to context-sensitive grammars, appeared both unnecessary and undesirable. In an attempt to pinpoint the exact formal power required for the adequate description of natural language syntax, Aravind Joshi characterized "grammars (and associated languages) that are only slightly more powerful than context-free grammars (context-free languages)". He called these grammars mildly context-sensitive grammars and the associated languages mildly context-sensitive languages.

Joshi’s characterization of mildly context-sensitive grammars was biased toward his work on tree-adjoining grammar (TAG). However, together with his students Vijay Shanker and David Weir, Joshi soon discovered that TAGs are equivalent, in terms of the generated string languages, to the independently introduced head grammar (HG). This was followed by two similar equivalence results, for linear indexed grammar (LIG) and combinatory categorial grammar (CCG), which showed that the notion of mildly context-sensitivity is a very general one and not tied to a specific formalism.

The TAG-equivalent formalisms were generalized by the introduction of linear context-free rewriting systems (LCFRS). These grammars define an infinite hierarchy of string languages in between the context-free and the context-sensitive languages, with the languages generated by the TAG-equivalent formalisms at the lower end of the hierarchy. Independently of and almost simultaneously to LCFRS, Hiroyuki Seki et al. proposed the essentially identical formalism of multiple context-free grammar (MCFG). LCFRS/MCFG is sometimes regarded as the most general formalism for specifying mildly context-sensitive grammars. However, several authors have noted that some of the characteristic properties of the TAG-equivalent formalisms are not preserved by LCFRS/MCFG, and that there are languages that have the characteristic properties of mildly context-sensitivity but are not generated by LCFRS/MCFG.

Recent years have seen increased interest in the restricted class of well-nested linear context-free rewriting systems/multiple context-free grammars, which define a class of grammars that properly includes the TAG-equivalent formalisms but is properly included in the unrestricted LCFRS/MCFG hierarchy.

Characterization

Despite a considerable amount of work on the subject, there is no generally accepted formal definition of mild context-sensitivity.

According to the original characterization by Joshi, a class of mildly context-sensitive grammars should have the following properties:

1. limited cross-serial dependencies
2. constant growth
3. polynomial parsing

In addition to these, it is understood that every class of mildly context-sensitive grammars should be able to generate all context-free languages.

Joshi’s characterization is not a formal definition. He notes:

Other authors have proposed alternative characterizations of mild context-sensitivity, some of which take the form of formal definitions. For example, Laura Kallmeyer takes the perspective that mild context-sensitivity should be defined as a property of classes of languages rather than, as in Joshi’s characterization, classes of grammars. Such a language-based definition leads to a different notion of the concept than Joshi’s.

Cross-serial dependencies

The term cross-serial dependencies refers to certain characteristic word ordering patterns, in particular to the verb–argument patterns observed in subordinate clauses in Dutch and Swiss German. These are the very patterns that can be used to argue against the context-freeness of natural language; thus requiring mildly context-sensitive grammars to model cross-serial dependencies means that these grammars must be more powerful than context-free grammars.

Kallmeyer identifies the ability to model cross-serial dependencies with the ability to generate the copy language

C O P Y = { w w ∣ w ∈ { a , b } ∗ }

and its generalizations to two or more copies of w, up to some bound. These languages are not context-free, which can be shown using the pumping lemma.

Constant growth

A formal language is of constant growth if every string in the language is longer than the next shorter strings by at most a (language-specific) constant. Languages that violate this property are often considered to be beyond human capacity, although some authors have argued that certain phenomena in natural language do show a growth that cannot be bounded by a constant .

Most mildly context-sensitive grammar formalisms (in particular, LCFRS/MCFG) actually satisfy a stronger property than constant growth called semilinearity. A language is semilinear if its image under the Parikh-mapping (the mapping that "forgets" the relative position of the symbols in a string, effectively treating it as a bag of words) is a regular language. All semilinear languages are of constant growth, but not every language with constant growth is semilinear.

Polynomial parsing

A grammar formalism is said to have polynomial parsing if its membership problem can be solved in deterministic polynomial time. This is the problem to decide, given a grammar G written in the formalism and a string w, whether w is generated by G – that is, whether w is "grammatical" according to G. The time complexity of this problem is measured in terms of the combined size of G and w.

Under the view on mild context-sensitivity as a property of classes of languages, polynomial parsing refers to the language membership problem. This is the problem to decide, for a fixed language L, whether a given string w belongs to L. The time complexity of this problem is measured in terms of the length of w; it ignores the question how L is specified.

Note that both understandings of polynomial parsing are idealizations in the sense that for practical applications one is interested not only in the yes/no question whether a sentence is grammatical, but also in the syntactic structure that the grammar assigns to the sentence.

Formalisms

Over the years, a large number of grammar formalisms have been introduced that satisfy some or all of the characteristic properties put forth by Joshi. Several of them have alternative, automaton-based characterizations that are not discussed in this article; for example, the languages generated by tree-adjoining grammar can be characterized by embedded pushdown automata.

Formalisms equivalent to TAG

Formalisms equivalent to general LCFRS/MCFG

Formalisms equivalent to well-nested LCFRS/MCFG

Relations among the formalisms

Linear context-free rewriting systems/multiple context-free grammars form a two-dimensional hierarchy of generative power with respect to two grammar-specific parameters called fan-out and rank. More precisely, the languages generated by LCFRS/MCFG with fan-out f ≥ 1 and rank r ≥ 3 are properly included in the class of languages generated by LCFRS/MCFG with rank r + 1 and fan-out f, as well as the class of languages generated by LCFRS/MCFG with rank r and fan-out f + 1. In the presence of well-nestedness, this hierarchy collapses to a one-dimensional hierarchy with respect to fan-out; this is because every well-nested LCFRS/MCFG can be transformed into an equivalent well-nested LCFRS/MCFG with the same fan-out and rank 2. Within the LCFRS/MCFG hierarchy, the context-free languages can be characterized by the grammars with fan-out 1; for this fan-out there is no difference between general and well-nested grammars. The TAG-equivalent formalisms can be characterized as well-nested LCFRS/MCFG of fan-out 2.