Objective
Learn how to use cURL to make HTTP requests to interact with online content.
Distributions
All GNU/Linux distributions
Requirements
Just cURL. It’s available in every distribution’s repository.
Difficulty
Easy
Conventions
# – requires root access either via su or sudo.
$ – run as your regular user
Introduction
cURL is a command line multi-tool for interacting with the web. cURL is capable of acting like a web browser, downloading files, accessing APIs, and even signing you into online accounts. cURL can be scripted, and it handles everything from a simple command line interface.
Grabbing a Page
Getting the HTML of a web pages is one of the most basic things that cURL can do. Try getting our home page.
$ curl https://linuxconfig.org
cURL dumps all of the HTML of the web page in the terminal. It looks messy, but that’s what it’s supposed to do.
Try another one. Debian’s website is a little easier to read.
$ curl https://www.debian.org/
HTTP Headers
A lot of the time, you don’t need a whole web page. All of that HTML can be hard to wade through, and all you need is information about the connection. cURL has the -I for just that.
$ curl -I https://linuxconfig.org
Now, you only get the HTTP headers that give you information about your connection to the site.
It works the same with Debian.
$ curl -I https://www.debian.org/
Controlling Output
All of that output isn’t very useful if you can’t save it. It’d be a pain to run cURL every time you need it, and what if it changes? Thankfully, you absolutely can save cURL’s output with the -o flag.
$ curl -o linuxconfig.html https://linuxconfig.org
You just saved the source code of the LinuxConfig home page to a file called linuxconfig.html.
This is much more useful for multimedia files, though. Try grabbing Google’s logo.
$ curl -o google.png https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png
You can also use the -O flag to tell cURL to use the existing name of the file.
$ curl -O https://getfedora.org/static/images/fedora_infinity_140x140.png
HTTP Requests and Logging In
With cURL, you can interact with web pages via HTTP. You can actually use cURL to log into a website. Take a look at this example from our Hydra Pentesting Guide.
curl -v --data 'log=username&pwd=realpassword&wp-submit=Log+In&testcookie=1' --cookie 'wordpress_test_cookie=WP+Cookie+check' http://localhost/wp-login.php
The --data or -d flag passes data along to a form using a big string variables and the names of the associated form fields. You can also send cookies with cURL.
Try searching DuckDuckGo for Linux.
$ curl -v -d 'q=linux' https://duckduckgo.com
The results may be a mess, but the search itself is simple. In case you were wondering, the -v flag adds the headers to the regular output.
Closing Thoughts
cURL is incredibly powerful. You’ve only scratched the surface, but you should have a good idea how cURL can interact with web pages and download content from the Internet. cURL can give you some real insights into how the web works, so it’s definitely worth experimenting with some more.
