tidytext brings text mining into tidy R workflows with tokenisation, sentiment lexicons, term frequencies, and data visualisation.
Read more
Tag: natural language processing
quanteda – quantitative analysis of textual data
quanteda provides powerful quantitative text analysis with corpus management, tokenisation, document-feature matrices, and dictionaries.
Read more
tm – text mining infrastructure for R
tm offers an extensible R text-mining framework with document corpora, preprocessing tools, term-document matrices, and metadata.
Read more
stringi – fast and portable Unicode string processing
stringi delivers fast Unicode string processing with regular expressions, collation, transliteration, encoding and text normalisation.
Read more
stringr – consistent string manipulation in R
stringr simplifies string manipulation in R with consistent functions for matching, extracting, replacing, splitting and joining text.
Read more
UDPipe – multilingual linguistic annotation in R
UDPipe provides multilingual tokenisation, part-of-speech tagging, lemmatisation, dependency parsing, and keyword extraction in R.
Read more
Word Vectors – build and explore word embeddings in R
Word Vectors trains and explores word embeddings in R with word2vec models, nearest-neighbour searches and cosine similarity.
Read more
spacyr – R interface to the spaCy NLP library
spacyr connects R to spaCy for tokenisation, part-of-speech tagging, lemmatisation, named entity recognition and dependency parsing.
Read more
MITIE – named-entity and relation extraction toolkit
MITIE provides named-entity recognition, entity extraction and relation detection with trainable models and multilingual interfaces.
Read more
text2vec – text analysis and word embeddings in R
text2vec supports efficient text vectorisation, document-term matrices, word embeddings, similarity analysis, and topic modelling in R.
Read more
Moses – statistical machine translation system
Moses trains and decodes statistical machine translation models from parallel text, with phrase-based, hierarchical, and factored models.
Read more
TiMBL – Tilburg Memory-Based Learner
TiMBL implements memory-based classifiers for NLP, including weighted nearest-neighbour learning, decision trees, and model evaluation.
Read more
MeTA – text retrieval and data analysis toolkit
MeTA combines text tokenization, indexing, classification, ranking, topic models and linguistic processing in one research toolkit.
Read more
CRF++ – conditional random field sequence labelling toolkit
CRF++ provides trainable conditional random fields for sequence tagging, supporting feature templates, n-best output and probabilities.
Read more
BLLIP Parser – statistical constituency parser
BLLIP Parser analyses English sentence structure using a generative constituency parser and discriminative reranking models.
Read more
Colibri Core – efficient corpus pattern modelling
Colibri Core extracts n-grams, skipgrams and flexgrams from text corpora, builds compact pattern models, and computes pattern statistics.
Read more
Xapian – probabilistic information retrieval library
Xapian is an information retrieval library with ranked full-text search, stemming, spelling suggestions, synonyms and query expansion.
Read more
flair – framework for natural language processing
flair provides neural NLP models, embeddings and training tools for named entity recognition, sequence tagging and text classification.
Read more
PyTorch-NLP – utilities for natural language processing research
PyTorch-NLP supplies text encoders, word vectors, dataset readers, neural network modules, samplers and evaluation metrics.
Read more
Snips NLU – natural language understanding library
Snips NLU parses natural language commands into structured intents and slots for private conversational assistant applications.
Read more