HTTP

Wget2 – successor of Wget

Wget2 is a utility for non-interactive download of files from the Web. It supports HTTP and HTTPS protocols, as well as retrieval through HTTP(S) proxies.

Wget2 is non-interactive, meaning that it can work in the background, while the user is not logged on. This allows you to start a retrieval and disconnect from the system, letting Wget2 finish the work. By contrast, most of the Web browsers require constant user’s presence, which can be a great hindrance when transferring a lot of data.

Wget2 can follow links in HTML, XHTML, CSS, RSS, Atom and sitemap files to create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as recursive downloading. While doing that, Wget2 respects the Robot Exclusion Standard (/robots.txt). Wget2 can be instructed to convert
the links in downloaded files to point at the local files, for offline viewing.

Wget2 has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports partial downloads, it may continue the download from where it left off.

This is free and open source software.

Features include:

  • Support for HTTP/1.1 and HTTP/2.0 protocol
  • brotli decompression support.
  • zstandard decompression support, RFC8478.
  • lzip decompression support.
  • HPKP – HTTP Public Key Pinning (RFC7469) with persistent database.
  • TCP Fast Open for plain text and for HTTPS.
  • TLS Session Resumption including persistent session data cache.
  • TLS False Start.
  • HTTP2 support via nghttp2 and GnuTLS ALPN including streaming/pipelining.
  • OCSP stapling + OCSP server querying as a fallback.
  • Use libpsl for cookie domain checking (using Public Suffix List).
  • Support link conversion.
  • Support for RFC 6266 compliant Content-Disposition.
  • RFC 6797 HSTS (HTTP Strict Transport Security).
  • Support for bzip2 Content-Encoding / Accept-Encoding compression type.
  • Support for XZ Content-Encoding / Accept-Encoding compression type.
  • Character encoding of input files may be specified despite from local and remote encoding.
  • Support scanning RSS 2.0 feeds from local files.
  • Support scanning RSS 2.0 feeds.
  • Support scanning Atom 1.0 feeds from local files .
  • Support scanning Atom 1.0 feeds.
  • Support scanning URLs from local Sitemap XML file.
  • Support scanning sitemap files given in robots.txt (Sitemap XML, gzipped Sitemap XML, plain text) including sitemap index files.
  • Support arbitrary number of proxies for parallel downloads.
  • Multithreaded download of single files.
  • Internationalized Domain Names in Applications.
  • ICEcast / SHOUTcast support via library.
  • Respect /robots.txt “Robot Exclusion Standard”.
  • IDN support for international domains.
  • autotools support.
  • Proxy support.
  • Cookies (session/non-session), detection of supercookies via Mozilla Public Suffix List.
  • Recursive download of websites with or without spanning hosts.
  • Download single web pages / resources.
  • zlib/gzip compressed HTTP/HTTPS downloads (gzip, deflate).
  • Number of parallel download threads is adjustable.
  • Include directive for config files (wildcards allowed).
  • Support for keep-alive connections.
  • Included CSS, HTML, XML parser needed for recursive downloads.
  • gettext support.
  • HTTPS via libgnutls (and basic WolfSSL support).
  • Support for Metalink RFC 6249 (Metalink/HTTP: Mirrors and Hashes).
  • Support for Metalink RFC 5854 (Metalink Download Description Format / .meta4 files).
  • Support for Metalink 3.
  • Metalink checksumming via libgnutls.
  • DNS lookup cache.
  • IPv4 and IPv6 support.
  • Cross-platform support – runs under Linux, OpenBSD, FreeBSD, Solaris, macOS, and Windows.

Website: gitlab.com/gnuwget/wget2
Support: Documentation
Developer: Tim Rühsen, Darshit Shah, Giuseppe Scrivano, and many contributors
License: GNU General Public License v3.0

Wget2 in action

Wget2 is written in C. Learn C with our recommended free books and free tutorials.

Alternatives to curl | Alternatives to Wget | Return to Command line HTTP Clients


Popular series
Free and Open Source SoftwareThe largest compilation of the best free and open source software in the universe. Each article is supplied with a legendary ratings chart helping you to make informed decisions.
ReviewsHundreds of in-depth reviews offering our unbiased and expert opinion on software. We offer helpful and impartial information.
The Big List of Active Linux Distros is a large compilation of actively developed Linux distributions.
Alternatives to Proprietary SoftwareReplace proprietary software with open source alternatives: Google, Microsoft, Apple, Adobe, IBM, Autodesk, Oracle, Atlassian, Corel, Cisco, Intuit, and SAS.
GamesAwesome Free Linux Games Tools showcases a series of tools that making gaming on Linux a more pleasurable experience. This is a new series.
Artificial intelligence iconMachine Learning explores practical applications of machine learning and deep learning from a Linux perspective. We've written reviews of more than 40 self-hosted apps. All are free and open source.
Guide to LinuxNew to Linux? Read our Linux for Starters series. We start right at the basics and teach you everything you need to know to get started with Linux.
Alternatives to popular CLI tools showcases essential tools that are modern replacements for core Linux utilities.
System ToolsEssential Linux system tools focuses on small, indispensable utilities, useful for system administrators as well as regular users.
ProductivityLinux utilities to maximise your productivity. Small, indispensable tools, useful for anyone running a Linux machine.
AudioSurveys popular streaming services from a Linux perspective: Amazon Music Unlimited, Myuzi, Spotify, Deezer, Tidal.
Saving Money with LinuxSaving Money with Linux looks at how you can reduce your energy bills running Linux.
Home ComputersHome computers became commonplace in the 1980s. Emulate home computers including the Commodore 64, Amiga, Atari ST, ZX81, Amstrad CPC, and ZX Spectrum.
Now and ThenNow and Then examines how promising open source software fared over the years. It can be a bumpy ride.
Linux at HomeLinux at Home looks at a range of home activities where Linux can play its part, making the most of our time at home, keeping active and engaged.
Linux CandyLinux Candy reveals the lighter side of Linux. Have some fun and escape from the daily drudgery.
DockerGetting Started with Docker helps you master Docker, a set of platform as a service products that delivers software in packages called containers.
Android AppsBest Free Android Apps. We showcase free Android apps that are definitely worth downloading. There's a strict eligibility criteria for inclusion in this series.
Programming BooksThese best free books accelerate your learning of every programming language. Learn a new language today!
Programming TutorialsThese free tutorials offer the perfect tonic to our free programming books series.
Linux Around The WorldLinux Around The World showcases usergroups that are relevant to Linux enthusiasts. Great ways to meet up with fellow enthusiasts.
Stars and StripesStars and Stripes is an occasional series looking at the impact of Linux in the USA.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Please read our Comment FAQ before posting a comment.

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments