UDPipe is a natural language processing toolkit for tokenising, tagging, lemmatising and parsing text. Its R interface provides access to a linguistic annotation pipeline that can process documents in numerous languages using pretrained statistical models.
The package is designed for researchers and developers who need structured linguistic information extracted from unstructured text. It can identify individual tokens, assign grammatical categories, determine base word forms and describe syntactic relationships between words.
Annotation results are returned in tabular form, allowing them to be examined using conventional data analysis techniques. The package also provides facilities for extracting terminology and identifying statistically significant combinations of words. Models can be trained from appropriately annotated linguistic resources.
This is free and open source software.
Key Features
- Performs tokenisation, sentence segmentation, part-of-speech tagging and lemmatisation.
- Identifies grammatical relationships through dependency parsing.
- Supports multilingual text annotation using pretrained Universal Dependencies models.
- Produces structured annotation containing document, sentence and token identifiers.
- Provides access to word forms, lemmas, grammatical tags and dependency relationships.
- Supports training custom models using annotated CoNLL-U datasets.
- Extracts keywords using Rapid Automatic Keyword Extraction (RAKE).
- Identifies collocations and calculates association measures between frequently co-occurring terms.
- Extracts multiword expressions using part-of-speech patterns.
- Provides functions for constructing and manipulating document-term matrices.
- Supports term-frequency and TF-IDF weighting.
- Offers functions for investigating semantic similarity and term relationships.
- Supports text mining and topic modelling using annotated linguistic features.
- Returns results suitable for integration with other R statistical and visualisation packages.
Website: github.com/bnosac/udpipe
Support:
Developer: Jan Wijffels
License: Mozilla Public License v2.0
UDPipe is written in C++ and R. Learn C++ with our recommended free books and free tutorials. Learn R with our recommended free books and free tutorials.
Related Software
| R Natural Language Processing Tools | |
|---|---|
| tidytext | Text mining using dplyr, ggplot2, and other tidy tools |
| quanteda | R package for Quantitative Analysis of Textual Data |
| text2vec | Framework with API for text analysis and natural language processing |
| wordcloud | Create attractive word clouds |
| tm | Text Mining Infrastructure in R |
| srtringi | Fast and portable character string processing in R |
| Stringr | String manipulation in R |
| UDPipe | Tokenization, Tagging, Lemmatization and Dependency Parsing |
| tokenizers | Convert natural language text into tokens |
| spacyr | R wrapper around the Python spaCy package |
| Word Vectors | Build and explore embedding models |
| syuzhet | Extraction of sentiment and sentiment-based plot arcs from text |
| textTinyR | Text processing for small or big data |
| sentimentr | Dictionary based sentiment analysis |
| textclean | Collection of tools to clean and normalize text |
| TALL | Explore, model, and visualize textual data |
| corpustools | Various tools for analyzing text corpora |
| topicmodels | Interface to LDA and CTM models |
| text | Analyzing natural language with transformers-based large language models |
| RTextTools | Automatic text classification via supervised learning |
Read our verdict in the software roundup.
 LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |
Please read our Comment Policy before commenting.