Recently I have been dabbling with my computer a lot. Maybe that is one of the perks of studying computer science. One of the troubles I was facing was to download files using the terminal on my computer. This is because somehow my windows GUI was not working, so I decided to switch to the command prompt.

I thought many of you might get into the same situation at some point in your lives, and then this article can serve as a support for you all.

With my initial search I discovered it’s pretty easy to download a file from an insecure website from your command prompt. This requires using a simple command “wget”. Just open the terminal on your computer and type the following command:

$wget "URL/4?action=AttachFile&do=view&target="filename with extension"

Seems quite simple, and if the website requires a user name and a password then just input:

$wget --user "username" --password "password" "URL/4?action=AttachFile&do=view&target= "filename with extension"

But wait, what if you want to download multiple files? I am sure for 100 more files you don’t want to type the file names each time. To accomplish this all you have to do is to store all file names in another file . Let’s say the name of file with list of all file names is file_list.txt. To create file_list.txt, all you have to do is to download the file names using “curl” command inside “for” loop:

for file in $(curl "URL" grep href | sed 's/.href="//' | sed 's/".//' | grep '^[a-zA-Z].*'); do
curl
"URL" done

The “URL” for curl can be obtained by going to the network tab on chrome. Chrome –>More tools–>Developer tools –> Network. Click on “download” button. The “URL” will show up under network tab. Copy the “URL” and paste it in “Curl” command above.

You will get the URL of all the file downloads. Simply cut the file names using cut command ex:

$cut -b 53-90 file_list.txt

You will have all the names in file_list.txt.

Now once you have it you can simply run the following command:

$cat file_list.txt | xargs -i -t wget --user "username" --password "password" "URL/{}"

And now you are all set. There are other ways around this, but I believe this method is one of the easiest. You will find all your 100 files downloaded within 15-20 minutes.

Let me know if you ever thought of this idea.While computer science is used to run rockets, it is not rocket science itself.

Author: Sadiya Ahmad

One Reply to “Trick to download just about anything”

  1. Dennis Thomas says:

    My only comment at this time is that you may have to adjust the cut command to get the right filename. The command you included is built with the assumption that the file names are located from the 53rd to the 90th charcter. This might be different depending on the source of the file list.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *