Vaex is an open source program and Python library to visualize and explore large tabular datasets.
It can calculate statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid up to a billion (109) objects/rows per second.
The original motivation to develop Vaex is the Gaia astronomical catalogue containing over a billion stars (at least for data release 1, DR1). Vaex can visualize the complete Gaia catalogue in one second.
Features:
- Graphical interface for most common uses cases.
- Visualize and explore big tabular data interactively
- Renders histograms, density plots and volume rendering plots for visualization in the order of a billion (109) objects in the order of 1 second.
- Explore the dataset by using visual queries and Boolean expressions to visualize subsets of the data.
- Statistics such as mean, sum, count, standard deviation etc, can all be calculated on an N-dimensional grid.
- For exploration it supports selection in 1 and 2d, but it can also analyze the columns (dimensions) to find subspaces which are richer in information than others.
- Overplot vectors, for instance mean motions, tensors (for instance mean velocity dispersion tensor).
- Custom expressions, e.g. log(sqrt(x**2+y**2)), calculated on the fly.
- Uses memory mapping, zero memory copy policy and lazy computations for best performance. Memory mapped files avoids unnecessary reading and copying of data. Binning or aggregating the data on a grid, using simple optimized algorithms. Columnar storage of data avoids reading unnecessary data and enables maximum performance of hard drives.
- Publish quality output (using matplotlib).
- Linked views: selecting in 1 view will also select it in different views.
- Data formats supported:
- hdf5 (Hierarchical Data Format): gadget, Vaex’s own format;
- hdf5 from Amuse;
- fits bintable;
- VOtable over SAMP;
- gadget native format.
- Client/server architecture: Delegate computations to a remote server.
The Vaex library generates the same plots and more, and offers integration with Jupyter/IPython notebook.
- pip and conda installable.
- Make custom plot and statistics.
- Calculate statistics on a N-dimensional grid and visualize it.
- Create interactive Jupyter/IPython notebooks.
- Publication quality plots with matplotlib.
- Interactive plots with bqplot or Bokeh.
- Combine the notebook with the graphical interface in one kernel.
Website: vaex.io
Support: Documentation, GitHub
Developer: Maarten A. Breddels
License: MIT License
Vaex is written in Python. Learn Python with our recommended free books and free tutorials.
Return to Python Visualization Packages Home Page | Return to Python Data Analysis Home Page
| Popular series | |
|---|---|
 | The largest compilation of the best free and open source software in the universe. Each article is supplied with a legendary ratings chart helping you to make informed decisions. |
 | Hundreds of in-depth reviews offering our unbiased and expert opinion on software. We offer helpful and impartial information. |
 | Replace proprietary software with open source alternatives: Google, Microsoft, Apple, Adobe, IBM, Autodesk, Oracle, Atlassian, Corel, Cisco, Intuit, and SAS. |
 | Machine Learning explores practical applications of machine learning and deep learning from a Linux perspective. This is a new series. |
 | New to Linux? Read our Linux for Starters series. We start right at the basics and teach you everything you need to know to get started with Linux. |
 | Essential Linux system tools focuses on small, indispensable utilities, useful for system administrators as well as regular users. |
 | Linux utilities to maximise your productivity. Small, indispensable tools, useful for anyone running a Linux machine. |
 | Surveys popular streaming services from a Linux perspective: Amazon Music Unlimited, Myuzi, Spotify, Deezer, Tidal. |
 | Saving Money with Linux looks at how you can reduce your energy bills running Linux. |
 | Home computers became commonplace in the 1980s. Emulate home computers including the Commodore 64, Amiga, Atari ST, ZX81, Amstrad CPC, and ZX Spectrum. |
 | Now and Then examines how promising open source software fared over the years. It can be a bumpy ride. |
 | Linux at Home looks at a range of home activities where Linux can play its part, making the most of our time at home, keeping active and engaged. |
 | Linux Candy reveals the lighter side of Linux. Have some fun and escape from the daily drudgery. |
 | Getting Started with Docker helps you master Docker, a set of platform as a service products that delivers software in packages called containers. |
 | Best Free Android Apps. We showcase free Android apps that are definitely worth downloading. There's a strict eligibility criteria for inclusion in this series. |
 | These best free books accelerate your learning of every programming language. Learn a new language today! |
 | These free tutorials offer the perfect tonic to our free programming books series. |
 | Linux Around The World showcases usergroups that are relevant to Linux enthusiasts. |
 | Stars and Stripes is an occasional series looking at the impact of Linux in the USA. |