link grammar

Link grammar connects the words in a sentence with links, similar in form to a catena. Unlike the catena or a traditional dependency grammar, the marking of the head-dependent relationship is optional for most languages, becoming mandatory only in free-word-order languages (such as Turkish,{{cite thesis |author=Özlem İstek |url=http://www.cs.bilkent.edu.tr/~ilyas/PDF/THESES/ozlem_istek_thesis.pdf |title=A Link Grammar for Turkish |type=Master's thesis |publisher=Bilkent University |place=Ankara, Turkey |year=2006 |access-date=2023-08-23}}{{better source needed|date=August 2023}} Finnish, Hungarian). That is, in English, the subject-verb relationship is "obvious", in that the subject is almost always to the left of the verb, and thus no specific indication of dependency needs to be made. In the case of subject-verb inversion, a distinct link type is employed. For free word-order languages, this can no longer hold, and a link between the subject and verb must contain an explicit directional arrow to indicate which of the two words is which.

Link grammar also differs from traditional dependency grammars by allowing cyclic relations between words. Thus, for example, there can be links indicating both the head verb of a sentence, the head subject of the sentence, as well as a link between the subject and the verb. These three links thus form a cycle (a triangle, in this case). Cycles are useful in constraining what might otherwise be ambiguous parses; cycles help "tighten up" the set of allowable parses of a sentence.

For example, in the parse

+---->WV--->+

+--Wd--+-Ss-+--Pa--+

| | | |

LEFT-WALL he runs fast

the LEFT-WALL indicates the start of the sentence, or the root node. The directional WV link (with arrows) points at the head verb of the sentence; it is the Wall-Verb link.[http://www.abisource.com/projects/link-grammar/dict/section-WV.html WV Link type] The Wd link (drawn here without arrows) indicates the head noun (the subject) of the sentence. The link type Wd indicates both that it connects to the wall (W) and that the sentence is a declarative sentence (the lower-case "d" subtype).[http://www.abisource.com/projects/link-grammar/dict/section-W.html W link type] The Ss link indicates the subject-verb relationship; the lower-case "s" indicating that the subject is singular.[http://www.abisource.com/projects/link-grammar/dict/section-S.html S link type] Note that the WV, Wd and Ss links for a cycle. The Pa link connects the verb to a complement; the lower-case "a" indicating that it is a predicative adjective in this case.[http://www.abisource.com/projects/link-grammar/dict/section-P.html P link type]

Parsing is performed in analogy to assembling a jigsaw puzzle (representing the parsed sentence) from puzzle pieces (representing individual words).{{cite arXiv |author1=Daniel D. K. Sleator |author2=Davy Temperley |title=Parsing English with a Link Grammar |year=1991 |eprint=cmp-lg/9508004}}[http://www.abisource.com/projects/link-grammar/dict/introduction.html An Introduction to the Link Grammar Parser] A language is represented by means of a dictionary or lexis, which consists of words and the set of allowed "jigsaw puzzle shapes" that each word can have. The shape is indicated by a "connector", which is a link-type, and a direction indicator + or - indicating right or left. Thus for example, a transitive verb may have the connectors S- & O+ indicating that the verb may form a Subject ("S") connection to its left ("-") and an object connection ("O") to its right ("+"). Similarly, a common noun may have the connectors D- & S+ indicating that it may connect to a determiner on the left ("D-") and act as a subject, when connecting to a verb on the right ("S+"). The act of parsing is then to identify that the S+ connector can attach to the S- connector, forming an "S" link between the two words. Parsing completes when all connectors have been connected.

A given word may have dozens or even hundreds of allowed puzzle-shapes (termed "disjuncts"): for example, many verbs may be optionally transitive, thus making the O+ connector optional; such verbs might also take adverbial modifiers (E connectors) which are inherently optional. More complex verbs may have additional connectors for indirect objects, or for particles or prepositions. Thus, a part of parsing also involves picking one single unique disjunct for a word; the final parse must satisfy (connect) all connectors for that disjunct.{{cite conference |author1=Dennis Grinberg |author2=John Lafferty |author3=Daniel Sleator |title=A Robust Parsing Algorithm for Link Grammar |conference=Proceedings of the Fourth International Workshop on Parsing Technologies, Prague |year=1995 |url=https://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/tr95-125.pdf |access-date=2023-08-28}}

Connectors may also include head-dependent indicators h and d. In this case, a connector containing a head indicator is only allowed to connect to a connector containing the dependent indicator (or to a connector without any h-d indicators on it). When these indicators are used, the link is decorated with arrows to indicate the link direction.

A recent extension simplifies the specification of connectors for languages that have little or no restrictions on word-order, such as Lithuanian. There are also extensions to make it easier to support languages with concatenative morphologies.

The parsing algorithm also requires that the final graph is a planar graph, i.e. that no links cross. This constraint is based on empirical psycho-linguistic evidence that, indeed, for most languages, in nearly all situations, dependency links really do not cross.{{cite conference |author=J. Havelka |year=2007 |title=Beyond projectivity: multilingual evaluation of constraints and measures on non-projective structures |conference=Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics |pages=608–615 |place=Prague, Czech Republic |publisher=Association for Computational Linguistics}}{{cite journal |author=R. Ferrer i Cancho |title=Why do syntactic links not cross? |journal=EPL |volume=76 |number=6 |year=2006 |pages=1228–1234|doi=10.1209/epl/i2006-10406-0 |bibcode=2006EL.....76.1228C |hdl=2117/180367 |hdl-access=free }} There are rare exceptions, e.g. in Finnish, and even in English; they can be parsed by link-grammar only by introducing more complex and selective connector types to capture these situations.

Connectors can have an optional floating-point cost markup, so that some are "cheaper" to use than others, thus giving preference to certain parses over others. That is, the total cost of parse is the sum of the individual costs of the connectors that were used; the cheapest parse indicates the most likely parse. This is used for parse-ranking multiple ambiguous parses. The fact that the costs are local to the connectors, and are not a global property of the algorithm makes them essentially Markovian in nature.{{cite conference |author1=John Lafferty |author2=Daniel Sleator |author3=Davey Temperley |title=Grammatical Trigrams: a Probabilistic Model of Link Grammar |conference=Proceedings of the AAAI Conference on Probabilistic Approaches to Natural Language |year=1992 |url=https://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/gram3gram.pdf}}_{{cite arXiv |author=Ramon Ferrer-i-Cancho |year=2013 |title=Hubiness, length, crossings and their relationships in dependency trees |class=cs.CL |eprint=1304.4086}}{{cite journal |author=D. Temperley |year=2008 |title=Dependency length minimization in natural and artificial languages |journal=Journal of Quantitative Linguistics |volume=15 |number=3 |pages=256–282|doi=10.1080/09296170802159512 }}{{cite book |author=E. Gibson |year=2000 |chapter=The dependency locality theory: A distance-based theory of linguistic complexity |editor1=Marantz, A. |editor2=Miyashita, Y. |editor3=O'Neil, W. |title=Image, Language, Brain: Papers from the first Mind Articulation Project Symposium |publisher=MIT Press |place=Cambridge, MA}}{{cite journal |author=Haitao Liu |url=http://www.lingviko.net/JCS.pdf |title=Dependency distance as a metric of language comprehension difficulty |year=2008 |journal=Journal of Cognitive Science |volume=9 |number=2 |pages=159–191|doi=10.17791/jcs.2008.9.2.159 }}{{cite journal |author1=Richard Futrell |author2=Kyle Mahowald |author3=Edward Gibson |title=Large-scale evidence of dependency length minimization in 37 languages |year=2015 |journal=PNAS |volume=112 |number=33 |pages=10336–10341 |doi=10.1073/pnas.1502134112|doi-access=free |pmid=26240370 |pmc=4547262 |bibcode=2015PNAS..11210336F }}

The assignment of a log-likelihood to linkages allows link grammar to implement the semantic selection of predicate-argument relationships. That is, certain constructions, although syntactically valid, are extremely unlikely. In this way, link grammar embodies some of the ideas present in operator grammar.

Because the costs are additive, they behave like the logarithm of the probability (since log-likelihoods are additive), or equivalently, somewhat like the entropy (since entropies are additive). This makes link grammar compatible with machine learning techniques such as hidden Markov models and the Viterbi algorithm, because the link costs correspond to the link weights in Markov networks or Bayesian networks.

The link grammar link types can be understood to be the types in the sense of type theory.{{cite conference |author1=Daniel Sleator |author2=Davey Temperley |title=Parsing English with a Link Grammar |conference=Third International Workshop on Parsing Technologies |year=1993 |url=https://www.cs.cmu.edu/afs/cs.cmu.edu/project/link/pub/www/papers/ps/LG-IWPT93.pdf}} (See section 6 on categorial grammar). In effect, the link grammar can be used to model the internal language of certain (non-symmetric) compact closed categories, such as pregroup grammars. In this sense, link grammar appears to be isomorphic or homomorphic to some categorial grammars. Thus, for example, in a categorial grammar the noun phrase "the bad boy" may be written as

:$$

{\text{the} \atop \text{NP/N,}}

{\text{bad} \atop \text{N/N,}}

{\text{boy} \atop \text{N}}

whereas the corresponding disjuncts in link grammar would be

the: D+;

bad: A+;

boy: D- & A-;

The contraction rules (inference rules) of the Lambek calculus can be mapped to the connecting of connectors in link grammar. The + and - directional indicators correspond the forward and backward-slashes of the categorical grammar. Finally, the single-letter names A and D can be understood as labels or "easy-to-read" mnemonic names for the rather more verbose types NP/N, etc.

The primary distinction here is then that the categorical grammars have two type constructors, the forward and backward slashes, that can be used to create new types (such as NP/N) from base types (such as NP and N). Link-grammar omits the use of type constructors, opting instead to define a much larger set of base types having compact, easy-to-remember mnemonics.

A basic rule file for an SVO language might look like:

D+;

{D−} & S+;

{D−} & O−;

S− & {O+};

Thus the English sentence, "The boy painted a picture" would appear as:

+-----O-----+

+-D-+--S--+ +--D--+

| | | | |

The boy painted a picture

Similar parses apply for Chinese.{{cite journal |author=Carol Liu |url=https://www.link.cs.cmu.edu/link/papers/index.html |title=Towards A Link Grammar for Chinese |journal=Computer Processing of Chinese and Oriental Languages |publisher=Chinese Language Computer Society |year=2001}}

Conversely, a rule file for a null subject SOV language might consist of the following links:

S+;

O+;

{O−} & {S−};

And a simple Persian sentence, man nAn xordam (من نان خوردم) 'I ate bread' would look like:{{cite web |author1=John Dehdari |author2=Deryle Lonsdale |url=http://www.ling.ohio-state.edu/~jonsafari/papers/dehdari_lonsdale_2005.pdf |archive-url=https://web.archive.org/web/20081203084810/http://www.ling.ohio-state.edu/~jonsafari/papers/dehdari_lonsdale_2005.pdf |archive-date=2008-12-03 |title=A Link Grammar for Persian |year=2005 |website=Ohio-state.edu}}{{cite journal |author1=Armin Sajadi |author2=A. Abdollahzadeh |url=http://web.cs.dal.ca/~sajadi/wp-content/uploads/2012/10/sajadi2006.pdf |title=Farsi Syntactic Analysis using Link Grammar |archive-url=https://web.archive.org/web/20140401052857/http://web.cs.dal.ca/~sajadi/wp-content/uploads/2012/10/sajadi2006.pdf |archive-date=2014-04-01 |language=fa |journal=Letter of Research Center of Intelligent Signal Processing |volume=1 |number=9 |pages=25–37 |year=2006}}{{cite journal |author1=A. Sajadi |author2=M. Homayounpour |title=Representation of Farsi Morphological Knowledge using Link Grammar |language=fa |journal=Letter of Research Center of Intelligent Signal Processing |volume=1 |number=9 |pages=41–55 |year=2006}}

+-----S-----+

| +--O--+

| | |

man nAn xordam

VSO order can be likewise accommodated, such as for Arabic.{{cite conference |author1=Warren Casbeer |author2=Jon Dehdari |author3=Deryle Lonsdale |url=http://www.ling.ohio-state.edu/~jonsafari/papers/casbeer-etal2007_preprint.pdf |archive-url=https://web.archive.org/web/20140512101153/http://www.ling.ohio-state.edu/~jonsafari/papers/casbeer-etal2007_preprint.pdf |archive-date=2014-05-12 |title=A Link Grammar parser for Arabic |conference=Perspectives on Arabic Linguistics: Papers from the annual symposium on Arabic linguistics |volume=XX |place=Kalamazoo, Michigan |date=March 2006}}

In many languages with a concatenative morphology, the stem plays no grammatical role; the grammar is determined by the suffixes. Thus, in Russian, the sentence 'вверху плыли редкие облачка' might have the parse:Документация по [http://www.abisource.com/projects/link-grammar/russian/doc/ связям] и по [http://www.abisource.com/projects/link-grammar/russian/doc/morph/ классам слов] доступна.[http://slashzone.ru/parser/ Грамматика связей (Link Grammar)]

+------------Wd-----------+---------------SIp---------------+

| +-------EI------+ +--------Api-------+

| | +--LLCZD-+ +-LLAQZ+ +--LLCAO-+

| | | | | | | |

LEFT-WALL вверху.e плы.= =ли.vnndpp ре.= =дкие.api облачк.= =а.ndnpi

The subscripts, such as '.vnndpp', are used to indicate the grammatical category. The primary links: Wd, EI, SIp and Api connect together the suffixes, as, in principle, other stems could appear here, without altering the structure of the sentence. The Api link indicates the adjective; SIp denotes subject-verb inversion; EI is a modifier. The Wd link is used to indicate the head noun; the head verb is not indicated in this sentence. The LLXXX links serve only to attach stems to suffixes.

The link-grammar can also indicate phonological agreement between neighboring words. For example:

+---------Ost--------+

+------>WV------>+ +------Ds**x-----+

+----Wd---+-Ss*b-+ +--PHv-+----A----+

| | | | | |

LEFT-WALL that.j-p is.v an abstract.a concept.n

Here, the connector 'PH' is used to constrain the determiners that can appear before the word 'abstract'. It effectively blocks (makes it costly) to use the determiner 'a' in this sentence, while the link to 'an' becomes cheap. The other links are roughly as in previous examples: S denoting subject, O denoting object, D denoting determiner. The 'WV' link indicates the head verb, and the 'W' link indicates the head noun. The lower-case letters following the upper-case link types serve to refine the type; so for example, Ds can only connect to a singular noun; Ss only to a singular subject, Os to a singular object. The lower-case v in PHv denotes 'vowel'; the lower-case d in Wd denotes a declarative sentence.

The Vietnamese language sentence "Bữa tiệc hôm qua là một thành công lớn" - "The party yesterday was a great success" may be parsed as follows:Nguyễn Thị Thu Hương, Nguyễn Thúc Hải, Nguyễn Thanh Thủy "[http://vjs.ac.vn/index.php/jcc/article/view/1451 Parsing complex - compound sentences with an extension of Vietnamese link parser combined with discourse segmenter]" Journal of Computer Science and Cybernetics, Vol 28, No 4 (2012)

Image:Vietnames link grammar example.png

{{Infobox software

| name = Link Grammar parser

| logo =

| logo caption =

| screenshot =

| caption =

| collapsible =

| author =

| developer = OpenCog

| released = {{Start date and age|1991|10}}

| discontinued =

| latest release version = 5.12.5

| latest release date = {{Start date and age|2024|05|31}} {{URL|https://www.gnucash.org/link-grammar/downloads/}} {{URL|https://github.com/opencog/link-grammar/tags}}

| latest preview version =

| latest preview date =

| frequently updated =

| programming language = C++; originally C

| operating system = Cross-platform

| platform = GNU

| size =

| language =

| status =

| genre = NLP

| license = LGPLv2

| alexa =

| website = {{URL|https://opencog.github.io/link-grammar-website/}}

}}

The link grammar syntax parser is a library for natural language processing written in C. It is available under the LGPL license. The parser[http://www.abisource.com/projects/link-grammar/ AbiWord — Link Grammar Parser] is an ongoing project. Recent versions include improved sentence coverage, Russian, Persian and Arabic language support, prototypes for German, Hebrew, Lithuanian, Vietnamese and Turkish, and programming API's for Python, Java, Common LISP, AutoIt and OCaml, with 3rd-party bindings for Perl,[http://search.cpan.org/~dbrian/Lingua-LinkParser/ Lingua-LinkParser (Perl interfaces)] Ruby{{Cite web |url=http://www.deveiate.org/projects/Ruby-LinkParser |title=Ruby Link Parser interfaces |access-date=2019-02-01 |archive-url=https://web.archive.org/web/20160304234317/http://www.deveiate.org/projects/Ruby-LinkParser |archive-date=2016-03-04 |url-status=dead }} and JavaScript node.js.[https://github.com/dijs/link-grammar javaScript node.js library]

A current major undertaking is a project to learn the grammar and morphology of new languages, using unsupervised learning algorithms.[http://wiki.opencog.org/w/Language_learning OpenCog Language Learning][https://arxiv.org/abs/1401.3372 Learning Language from a Large (Unannotated) Corpus]

The link-parser program along with rules and word lists for English may be found in standard Linux distributions, e.g., as a Debian package, although many of these are years out of date.[http://packages.debian.org/link-grammar Debian - Package Search Results - link-grammar]

File:Abiword grammar.jpg checks grammar using link grammar]]

AbiWord, a free word processor, uses link grammar for on-the-fly grammar checking. Words that cannot be linked anywhere are underlined in green.

The semantic relationship extractor RelEx,{{Cite web |url=http://opencog.org/wiki/RelEx_Dependency_Relationship_Extractor |title=RelEx Dependency Relationship Extractor |access-date=2013-11-21 |archive-url=https://web.archive.org/web/20090728162235/http://opencog.org/wiki/RelEx_Dependency_Relationship_Extractor |archive-date=2009-07-28 |url-status=dead }} layered on top of the link grammar library, generates a dependency grammar output by making explicit the semantic relationships between words in a sentence. Its output can be classified as being at a level between that of SSyntR and DSyntR of Meaning-Text Theory. It also provides framing/grounding, anaphora resolution, head-word identification, lexical chunking, part-of-speech identification, and tagging, including entity, date, money, gender, etc. tagging. It includes a compatibility mode to generate dependency output compatible with the Stanford parser,[http://nlp.stanford.edu/software/lex-parser.shtml The Stanford Parser: A statistical parser] and Penn Treebank[http://www.cis.upenn.edu/~treebank/ The Penn Treebank Project] {{webarchive|url=https://web.archive.org/web/20131109202842/http://www.cis.upenn.edu/~treebank/ |date=2013-11-09 }}-compatible POS tagging.

Link grammar has also been employed for information extraction of

biomedical texts{{Cite conference |author1=Jing Ding |author2=Daniel Berleant |author3=Jun Xu |author4=Andy W. Fulmer |title=Extracting biochemical interactions from MEDLINE using a link grammar parser |book-title=Proceedings of the Fifteenth IEEE Conference on Tools with Artificial Intelligence (ICTAI), 2003 |date=November 2003 |pages=467–471 |isbn=0-7695-2038-3 |url=https://ualr.edu/jdberleant/papers/LGPmanuscript8-8-03a.pdf |archive-url=https://wayback.archive-it.org/all/20110331201726/http://ifsc.ualr.edu/jdberleant/papers/LGPmanuscript8-8-03a.pdf |url-status=dead |archive-date=2011-03-31 |access-date=2023-08-27 }}Sampo Pyysalo, Tapio Salakoski, Sophie Aubin and Adeline Nazarenko, "[http://www.biomedcentral.com/1471-2105/7/S3/S2 Lexical Adaptation of Link Grammar to the Biomedical Sublanguage: a Comparative Evaluation of Three Approaches]", BMC Bioinformatics 7(Suppl 3):S2 (2006). and

events described in news articles,{{Cite conference

|author1=Harsha V. Madhyastha |author2=N. Balakrishnan |author3=K. R. Ramakrishnan | title = Event Information Extraction Using Link Grammar

| book-title = 13th International WorkShop on Research Issues in Data Engineering: Multi-lingual Information Management (RIDE'03)

| pages = 16

| doi = 10.1109/RIDE.2003.1249841

| year = 2003

}} as well as experimental machine translation systems from English to German, Turkish, Indonesian.{{Cite conference

|author1=Teguh Bharata Adji |author2=Baharum Baharudin |author3=Norshuhani Zamin | title = Applying Link Grammar Formalism in the Development of English-Indonesian Machine Translation System

| book-title = Intelligent Computer Mathematics, 9th International Conference, AISC 2008, 15th Symposium, Calculemus 2008, 7th International Conference, Birmingham, UK, Proceedings

| pages = 17–23

| doi = 10.1007/978-3-540-85110-3_3

| year = 2008

}} and Persian.A.Sajadi and M.R Borujerdi, "Machine Translation Using Link Grammar", Submitted

to the Journal of Computational Linguistics, MIT Press (Feb 2009)Sajadi, A., Borujerdi, M. "Machine Translation Based on Unification Link Grammar" Journal of Artificial Intelligence Review. DOI=10.1007/s10462-011-9261-7, Pages 109-132, 2013.

The link grammar link dictionary is used to generate and verify the syntactic correctness of three different natural language generation systems: NLGen,Ruiting Lian, et al, "Sentence generation for artificial brains: a glocal similarity matching approach", Neurocomputing (Elsevier) (2009, submitted for publication). NLGen2Blake Lemoine, [http://www.louisiana.edu/~bal2277/NLGen2.doc NLGen2: A Linguistically Plausible, General Purpose Natural Language Generation System] (2009) and microplanner/surreal.[http://wiki.opencog.org/w/Microplanner Microplanner] and [http://wiki.opencog.org/w/Surface_Realization_(SuReal) Surface Realization (SuReal)] It is also used as a part of the NLP pipeline in the OpenCog AI project.

{{full citations needed|date=August 2023}}

{{Reflist}}

{{Scholia|topic}}

• [http://www.link.cs.cmu.edu/link/ The original Link Grammar homepage] (which has been replaced by the [https://web.archive.org/web/20240723233431/http://www.nl.abisource.com/projects/link-grammar/ current project]. Archived from [http://www.nl.abisource.com/projects/link-grammar/ the original])
• [http://www.link.cs.cmu.edu/link/submit-sentence-4.html Online English demonstration] (for an older, out-of-date version; many bugs have been fixed since this version.)
• [http://mars.cs.utu.fi/biolg/ BioLG], a modification of the Link Grammar Parser adapted for the biomedical domain (many, but not all, BioLG enhancements have been folded back into the main link-grammar distribution).
• [https://www.youtube.com/watch?v=Wk8zAr0R9zQ Parsing sentences with Link Grammar and Python] by [http://jeffelmore.org/2012/03/23/parsing-sentences-with-link-grammar-and-python/ Jeff Elmore] at [https://us.pycon.org/2012/ PyCon 2012]

• [https://archive.today/20121212094047/http://www.ling.ohio-state.edu/~jonsafari/arabiclg/bin/arabiclg.html Arabic Link Grammar extension] ([https://web.archive.org/web/20110527160257/http://www.ling.ohio-state.edu/~jonsafari/arabiclg/arabiclg.20060829.tar.bz2 Source package])
• [https://web.archive.org/web/20110527160336/http://www.ling.ohio-state.edu/~jonsafari/persianlg/persianlg-0.8.5.tar.gz Persian Link Grammar extension]
• [https://web.archive.org/web/20050409030238/http://students.cs.byu.edu/~jonsafar/persianlg.html Online Persian demonstration]
• [http://sz.ru/parser/ Russian Link Grammar demonstration]

Category:Dependency grammar

Category:Natural language parsing