Natural Language Processing

textrecipes – text preprocessing for machine learning workflows

textrecipes extends the recipes framework with preprocessing operations designed specifically for textual data. It provides a consistent way to transform raw text into numerical representations suitable for statistical modelling and machine learning.

Text often requires a different sequence of preparation steps from conventional numerical or categorical variables. Documents may need to be separated into tokens, normalized, filtered, and converted into feature representations before they can be passed to a model.

The package integrates these operations into reusable preprocessing recipes. This helps maintain consistency between training data and new observations, reducing the need to construct separate text-processing pipelines.

This is free and open source software.

Key Features

  • Tokenizes character variables into words or other units suitable for subsequent processing.
  • Removes common stopwords that may contribute little useful information to a predictive model.
  • Filters tokens according to frequency and configurable vocabulary limits.
  • Creates term-frequency features that represent the occurrence of individual tokens.
  • Calculates term frequency-inverse document frequency (TF-IDF) representations.
  • Feature hashing maps tokens into fixed-dimensional numerical representations.
  • Supports stemming to reduce related word forms to a common representation.
  • Includes preprocessing operations for converting text into structured token sequences.
  • Provides topic-modelling transformations using latent Dirichlet allocation.
  • Integrates with the recipes framework for preparing, training, and applying preprocessing steps.
  • Allows text transformations to be combined with conventional numerical and categorical preprocessing operations.
  • Supports the creation of sparse feature representations for modelling high-dimensional textual data.

Website: github.com/tidymodels/textrecipes
Support:
Developer: Emil Hvitfeldt and contributors
License: MIT License

textrecipes is written in R. Learn R with our recommended free books and free tutorials.


Related Software

R Natural Language Processing Tools
tidytextText mining using dplyr, ggplot2, and other tidy tools
quantedaR package for Quantitative Analysis of Textual Data
text2vecFramework with API for text analysis and natural language processing
wordcloudCreate attractive word clouds
tmText Mining Infrastructure in R
srtringiFast and portable character string processing in R
StringrString manipulation in R
UDPipeTokenization, Tagging, Lemmatization and Dependency Parsing
tokenizersConvert natural language text into tokens
spacyrR wrapper around the Python spaCy package
Word VectorsBuild and explore embedding models
syuzhetExtraction of sentiment and sentiment-based plot arcs from text
textTinyRText processing for small or big data
sentimentrDictionary based sentiment analysis
textcleanCollection of tools to clean and normalize text
TALLExplore, model, and visualize textual data
corpustoolsVarious tools for analyzing text corpora
topicmodelsInterface to LDA and CTM models
textAnalyzing natural language with transformers-based large language models
RTextToolsAutomatic text classification via supervised learning

Read our verdict in the software roundup.


Best Free and Open Source Software LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.

We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more.

Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion.

Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README.
Subscribe

Please read our Comment Policy before commenting.

Notify of
guest
0 Comments
Oldest
Newest Most Voted