Did you know the secret behind – how people were able to download stuff over the Internet when they didn’t have download managers? Well, Wget command not only overcomes the problem but also offers more flexibility than a download manager can ever offer. However, when you get to know about Wget commands and Wget command examples, you’ll get a better grip on the answer to the previous question.
There are few more problems which can be solved using Wget commands which you may have thought of earlier sometimes. How do I mirror an entire site? How do I download a particular web page? How do I download a file that requires a specific referral? How do I download Mp3 files from a website and store them all on my computer? How do I keep retrying the download while I’m out for another task?
These are some of the fundamental questions you must be having on your mind. I’ll bet you’re excited enough to skip the section to start using the Wget commands, but I won’t advise doing that unless you get to know more about it in details.
Also Read: Svchost.exe: What is Svchost.exe and How to Fix Svchost.exe High CPU Usage?
Contents
What is a Wget Command?
Wget is a free utility to achieve non-interactive downloads. That’s the simplest way of describing what Wget is. Simply put, when you want to automate a download task which doesn’t require repeated user’s input/user interaction, Wget serves as a hero.
You may use Wget command perfectly on Windows, Mac, and Linux. Well, Linux being the exception because it supports the utility without the need to set it up. In a nutshell, Linux comes with the native support for Wget command utility. Don’t worry. We’ll also let you know how to get it working on Windows/Mac powered system.
You may consider calling it “GNU Wget” because the Free Software Foundation has published it.
Why do we Require using Wget Command?
Of course, we need to utilize a command to achieve the necessary task. But what tasks exactly? Let’s break them down for you:
- Clone/Backup a Website
- Download A WebPage for Offline reading
- Mirroring FTP sites which contain a lot of useful resources
- Overcoming the problem of Slow/Unusable network connections
- Faster Access to files behind firewalls
- Download Restricted Content
- Downloading specific type of files from a particular website
In practice, there’s a lot of things that can be done using Wget commands. However, the points mentioned above are just the overview of what can be achieved using Wget commands.
Check: 127+ Amazing Google Now Voice Commands List to Do Magical Things with Google
20 Wget Command Example to Do Cool Things with Wget Commands
Here, you’ll witness 20 of the best examples of Wget commands which you can try it on your system as well.
Example 1: Wget Command to download a particular file from a specified URL.
It refers to the situation where we have the exact URL of the file but instead of downloading it through a browser (where it may fail anytime), we would utilize Wget command to download the file.
The format to download a file from a URL would be:
wget https://www.samplewebsite.com
For instance, I tried downloading Ampps for Linux, here’s what I did:
ankush@ankush-X550LD:~$ wget http://s4.softaculous.com/a/ampps/files/Ampps-3.5-x86_64.run --2016-07-20 07:50:41-- http://s4.softaculous.com/a/ampps/files/Ampps-3.5-x86_64.run< Resolving s4.softaculous.com (s4.softaculous.com)... 138.201.24.83 Connecting to s4.softaculous.com (s4.softaculous.com)|138.201.24.83|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 375936801 (359M) [application/octet-stream] Saving to: ‘Ampps-3.5-x86_64.run’ Ampps-3.5-x86_64.ru 6%[> ] 22.86M 67.9KB/s eta 1h 45m
Example 2: Wget Command to download a file and rename the file at once.
It would download the file with the same name, but when it completes, the name would be changed automatically to what you’ve chosen to keep.
Syntax:
wget -O customname.html https://exampledomain.com
Here’s an example along with the desired output:
wget -O mywish.html https://techreviewpro.com --2016-07-22 23:58:58-- https://techreviewpro.com/ Resolving techreviewpro.com (techreviewpro.com)... 162.144.210.54 Connecting to techreviewpro.com (techreviewpro.com)|162.144.210.54|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: ‘mywish.html’ mywish.html [ <=> ] 73.28K 35.7KB/s in 2.0s 2016-07-22 23:59:03 (35.7 KB/s) - ‘mywish.html’ saved [75039]
Example 3: Wget Command to continue/resume the download which had failed due to network connectivity or other similar reasons.
Now, you no longer require a downloader to help resume a download. If there’s a direct file link to the resource, it’ll be very easy to resume the incomplete download with Wget.
Format:
wget -c http://example.com/anyfile.zip
Example 4: Wget Command to limit the download rate to save bandwidth.
Do you want a download process not to utilize the full bandwidth? Therefore, saving extra bandwidth for web browsing and doing other stuff on the internet at the same time. To do that, you need to limit the download rate.
Format:
wget --limit-rate=20k http://www.exampledomain.com/filnename.zip
Example 5: Wget Command to download all the files at once which are in a proper sequence.
You may be needing to download a list of images stored on the server with sequential file names, such as 1,2,3, So, you’ll have to follow the format below.
Format:
wget http://sample.com/imagesiploaded/{1..20}.jpg
Example 6: Wget Command to download bulk files of a single file type.
You can make use of this command only when you know the directory and which the type of files you’ve to download now.
Format:
wget http://www.example.com/directory/*.extension
Example 7: Wget Command to check whether the download link exists or not
Format:
wget --spider http://www.example.com/filename.zip
Here’s how it should look like:
- When the remote file does not exist:
wget --spider https://www.xyz.com/c.jpg Spider mode enabled. Check if remote file exists. --2016-07-23 08:56:03-- https://www.xyz.com/c.jpg Resolving www.xyz.com (www.xyz.com)... 184.169.138.0 Connecting to www.xyz.com (www.xyz.com)|184.169.138.0|:443... failed: Connection refused.
- When remote file exists:
wget --spider https://techreviewpro-techreviewpro.netdna-ssl.com/wp-content/uploads/2016/05/Image-001-2.jpg Spider mode enabled. Check if remote file exists. --2016-07-23 08:58:09-- https://techreviewpro-techreviewpro.netdna-ssl.com/wp-content/uploads/2016/05/Image-001-2.jpg Resolving techreviewpro-techreviewpro.netdna-ssl.com (techreviewpro-techreviewpro.netdna-ssl.com)... 198.232.125.32 Connecting to techreviewpro-techreviewpro.netdna-ssl.com (techreviewpro-techreviewpro.netdna-ssl.com)|198.232.125.32|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 24442 (24K) [image/jpeg] Remote file exists.
Example 8: Wget Command to download a file in the Background.
Format:
wget -b http://website.com/file.zip
Here’s how it looks like:
wget -b http://s4.softaculous.com/a/ampps/files/Ampps-3.5-x86_64.run Continuing in background, pid 28483. Output will be written to ‘wget-log.3’.
You can check the progress by entering the following command: (Use the output log file as mentioned in the output)
tail -f wget-log.3
Wait! Did you try it just for an experiment, but you want to cancel it anyway? Here’s the command to kill the process:
pkill wget
Example 9: Wget Command to send a request as the fake browser to download a file
Format:
wget --user-agent="custom name or a desired browser name along with the version" <URL-of-the-file>
Here’s how it looks like if you are thinking to download a file:
wget --user-agent="Chromium51.0/Version 51.0.2704.79 Ubuntu 16.04 (64-bit)" https://techreviewpro.com
Example 10: Wget Command to download a particular file which retries the connection multiple times when disconnected.
This Wget command comes handy whenever you are initiating a download, but you’re unsure of the network connectivity. And, at the same time, you are going to pick something up from outside. So, you can automate the retries by using the command. It’s basically like you’re retrying the download remotely.
Format:
wget --tries=115 <URL-of-the-file>
Example 11: Wget Command to set a download limit if you’re unsure of the file size.
This one’s a good option if you don’t get an idea of the file size. Also, if you’re on metered connection, this would be useful. So that, you could resume the download whenever the monthly download limits resets.
Note: Here Q1m denotes that the download will stop after 1 MB of the file has been downloaded.
Format:
wget -Q1m -i <URL-of-the-file>
Example 12: Wget Command to Save the log file separately instead displaying it in the terminal.
Format:
wget -o customname.log URL
You’ll observe a log file in the desired location which contains the output message as you normally saw inside the terminal previously.
Example 13: Wget Command to download only particular type of files
Format:
wget -r -A.jpg URL-of-the-file
Example 14: Wget Command to download a file through FTP protocol without the need of FTP clients like FileZilla.
If you are a regular FTP user, you must be having an FTP client like FileZilla. Software utilities make the task easier. But, if you don’t want the FTP client and keep your system clean from unnecessary software services, Wget commands would save you!
Format:
wget --ftp-user=username --ftp-password=passphrase URL-of-the-file
Example 15: Wget Command to Download Multiple Files
Note: Create a text file with the same name you are going to use with Wget commands and save the URLs to the text file (one in each line). It’s recommended to put the file in the home directory. If you’ve created in a different directory, then you would need to navigate to the specified directory using the cd command.
Format:
wget -i anynameyouset.txt
Example 16: Wget Command to Clone/Mirror a site using Wget commands
WEBSITE-URLFormat:
$ wget --mirror -p Download-URL
Example 17: Wget Command to Check & Fetch the new version of a file
Let’s say you have downloaded a file. And, to check with the server whether there’s a newer version available or not, you can use Wget commands utilizing timestamp.
Note: If the timestamp header is missing from the website, it will re-fetch the file.
Format:
wget -N https://wwwmywebsite.com/filename.zip
Example 18: Wget Command to Download a specific type of file from a particular directory of the website (here – images)
Format:
wget ‐‐level=1 ‐‐recursive ‐‐no-parent ‐‐accept jpg,JPG https://mywebsite.com/images
Example 19: Wget Command to Download a file that requires a specific referral domain
Some promotional files can be downloaded only through the specified referral domain. You can take the help of Wget commands in order mock the referral domain and download the file.
Format:
wget ‐‐refer=http://promotionalhostsite.com ‐‐user-agent="Chrome51.0.1" http://targetsitewiththefile.com
Example 20: Wget Command to Get the directory listing of a site in an HTML file
If you want to know the directory listings from a site and store it offline. You can do it easily with the help of Wget commands as mentioned below.
Format:
wget FTP URL links cutomname.html
Here’s what it looks like in action:
wget ftp://ftp.gnu.org/pub/gnu/ --2016-07-23 19:19:56-- ftp://ftp.gnu.org/pub/gnu/ => ‘.listing’ Resolving ftp.gnu.org (ftp.gnu.org)... 208.118.235.20, 2001:4830:134:3::b Connecting to ftp.gnu.org (ftp.gnu.org)|208.118.235.20|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD (1) /pub/gnu ... done. ==> PASV ... done. ==> LIST ... done. .listing [ <=> ] 24.43K 19.4KB/s in 1.3s 2016-07-23 19:20:10 (19.4 KB/s) - ‘.listing’ saved [25019] Removed ‘.listing’. Wrote HTML-ized index to ‘index.html.11’ [37133].
The Pro Review From TechReviewPro
Finally, you’ve got to know about the best Wget command examples that you should be using right now. So, Wget commands are useful to facilitate various kinds of download. It provide us with the ability to control and modify the way we want to have control over a download. Most of all, after you’ve mastered the skills using Wget commands, you won’t be needing a download manager ever.
Confused about a command? Have an alternative to the mentioned Wget command? Let us know your thoughts through the comments section below.
Don’t Miss:
- 15 Cool Notepad Tricks and Hacks to Do Cool Things with Notepad
- 10 Best Ubuntu Text Editors to Alter Ubuntu Default Text Editor
- How to Create WiFi Hotspot Using Command Prompt and Share Files Easily?
- 7 Best CSS Editors to Create and Edit Your CSS Files
- 17 Cool Computer Command Prompt Tricks and Hacks for Windows Users
