AWS SDK for pandas, formerly AWS Data Wrangler, extends pandas workflows to Amazon Web Services data and analytics infrastructure. It provides a collection of functions for exchanging tabular data between DataFrames, object storage, databases and managed analytical services.
The library is intended to simplify common extract, transform and load operations. Instead of manually coordinating multiple service-specific interfaces, users can perform many data transfer, query and metadata operations through a consistent analytical API.
AWS SDK for pandas is particularly useful for data engineering workflows involving Amazon S3, Athena, Glue and Redshift. It helps connect local data preparation and analysis with cloud-based storage and querying services.
This is free and open source software.
Key Features
- Read and write pandas DataFrames using Amazon S3.
- Support CSV, Parquet, JSON and other documented data formats.
- Execute SQL queries with Amazon Athena and retrieve results as DataFrames.
- Integrate with AWS Glue Data Catalog for table metadata and dataset discovery.
- Read and write analytical data using Amazon Redshift.
- Connect to supported relational databases including PostgreSQL, MySQL and Microsoft SQL Server.
- Work with partitioned datasets and supported schema evolution workflows.
- Read and write supported DynamoDB data.
- Integrate with Amazon Timestream for time series information.
- Access supported OpenSearch and Neptune data sources.
- Perform supported dataset operations across S3 and AWS analytical services.
- Distribute selected operations through the Ray execution backend.
- Use AWS sessions and configuration settings to control service access.
Website: aws-sdk-pandas.readthedocs.io
Support:
Developer: AWS Professional Services
License: Apache License 2.0
AWS Data Wrangler is written in Python. Learn Python with our recommended free books and free tutorials.
Related Software
| Python Data Analysis | |
|---|---|
| pandas | High-level building block for doing practical, real world data analysis |
| NumPy | Core package for scientific computing with Python |
| SciPy | Ecosystem for mathematics, science, and engineering |
| Polars | DataFrame interface on top of an OLAP Query Engine |
| statsmodels | Statistical modeling and econometrics in Python |
| Dask | Advanced parallelism for analytics |
| Orange | Component-based framework for machine learning and data mining |
| Modin | Drop-in replacement for pandas |
| Vaex | Fast visualization of big data |
| AWS DW | Extends the power of pandas library |
| yt | Multi-code Toolkit for Analyzing and Visualizing Volumetric Data |
| HoloViews | Make Data Analysis and Visualization Seamless |
| datatable | Manipulate 2-dimensional tabular data structures |
| xarray | Work with labelled multi-dimensional arrays and datasets |
| pyjanitor | Extend pandas with readable data-cleaning functions |
| Optimus | Agile Data Preparation Workflows |
Read our verdict in the software roundup.
 LinuxLinks helps you discover the best free and open source software for Linux. Explore our extensive collection of recommended applications, including alternatives to popular proprietary software.We also publish software reviews, Linux tutorials, hardware reviews, free programming books and articles covering the wider open source community. Browse our Linux resources to discover more. Know a useful free and open source Linux application we haven't covered? Suggest a program for inclusion. Has your software been featured on LinuxLinks? Display one of our Featured on LinuxLinks badges on your website or GitHub README. |
Please read our Comment Policy before commenting.