Natural Language Processing

wordvector – train and explore word and document vectors

wordvector is a package for creating and analysing vector representations of words and documents. It works with the quanteda text-analysis framework, allowing researchers to train models using tokenized corpora.

Vector representations place words or documents into a numerical space where semantic relationships can be explored mathematically. Words that occur in similar contexts may develop similar representations, making it possible to investigate related terminology and patterns in large text collections.

The package supports several approaches to representation learning, making it useful for linguistic research, document analysis, and exploratory studies of textual data.

This is free and open source software.

Key Features

  • Trains word2vec models using tokenized text supplied through quanteda.
  • Supports continuous bag-of-words training for learning word representations from surrounding context.
  • Creates document embeddings using doc2vec techniques.
  • Provides latent semantic analysis for exploring relationships between terms and documents.
  • Calculates cosine similarity between word vectors.
  • Finds words with similar vector representations to specified query terms.
  • Supports arithmetic operations on embedding vectors to investigate semantic relationships.
  • Includes analogy operations for evaluating relationships between multiple words.
  • Configurable training dimensions and vocabulary frequency thresholds.
  • Parallel computation can accelerate training on larger text collections.
  • Works with quanteda corpus and token-processing functions.
  • Provides numerical representations that can be used for subsequent statistical analysis and visualization.

Website: github.com/koheiw/wordvector
Support:
Developer: Kohei Watanabe
License: Apache License 2.0

wordvector is written in R. Learn R with our recommended free books and free tutorials.


Related Software

R Natural Language Processing Tools
tidytextText mining using dplyr, ggplot2, and other tidy tools
quantedaR package for Quantitative Analysis of Textual Data
text2vecFramework with API for text analysis and natural language processing
wordcloudCreate attractive word clouds
tmText Mining Infrastructure in R
srtringiFast and portable character string processing in R
StringrString manipulation in R
UDPipeTokenization, Tagging, Lemmatization and Dependency Parsing
tokenizersConvert natural language text into tokens
spacyrR wrapper around the Python spaCy package
Word VectorsBuild and explore embedding models
syuzhetExtraction of sentiment and sentiment-based plot arcs from text
textTinyRText processing for small or big data
sentimentrDictionary based sentiment analysis
textcleanCollection of tools to clean and normalize text
TALLExplore, model, and visualize textual data
corpustoolsVarious tools for analyzing text corpora
topicmodelsInterface to LDA and CTM models
textAnalyzing natural language with transformers-based large language models
RTextToolsAutomatic text classification via supervised learning

Read our verdict in the software roundup.


Best Free and Open Source Software LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.

We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more.

Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion.

Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README.
Subscribe

Please read our Comment Policy before commenting.

Notify of
guest
0 Comments
Oldest
Newest Most Voted