Время на прочтение
9 мин
Количество просмотров 75K
Curl (client URL) — это инструмент командной строки на основе библиотеки libcurl для передачи данных с сервера и на сервер при помощи различных протоколов, в том числе HTTP, HTTPS, FTP, FTPS, IMAP, IMAPS, POP3, POP3S, SMTP и SMTPS. Он очень популярен в сфере автоматизации и скриптов благодаря широкому диапазону функций и поддерживаемых протоколов. В этой статье мы расскажем, как использовать curl в Windows на различных примерах.
▍ Установка в Windows
Во всех современных версиях Windows, начиная с Windows 10 (версия 1803) и Server 2019, исполняемый файл curl поставляется в комплекте, поэтому ручная установка не требуется. Чтобы определить местоположение curl и его версию в системе, можно использовать следующие команды:
where curl
curl --version
Определение местоположения и версии curl в Windows
Команда curl —version также выводит список протоколов и функций, поддерживаемых текущей версией curl. Как видно из показанного выше скриншота, к использованию встроенной утилиты curl всё готово. Если вместо этого отображается сообщение об ошибке, curl может быть недоступен потому, что вы используете более раннюю версию Windows (например, Windows 8.1 или Server 2016). В таком случае вам потребуется установить curl в Windows вручную.
▍ Синтаксис curl
Команда curl использует следующий синтаксис:
curl [options...] [url]Инструмент поддерживает различные опции, которые мы рассмотрим ниже. Как и в любом инструменте командной строки, вы можете использовать для получения справки команду curl —help.
Получение справки при помощи команды curl
Для получения подробной справки можно использовать команду curl —help all. Справка разделена на категории, поэтому при помощи curl —help category можно просмотреть все темы.
Ознакомившись с синтаксисом curl, давайте рассмотрим различные способы применения этого инструмента на примерах.
▍ HTTP-запрос GET
При использовании curl с URL и без указания опций запрос по умолчанию использует метод GET протокола HTTP. Попробуйте выполнить такую команду:
curl https://4sysops.comПриведённая выше команда по сути эквивалентна curl —request GET 4sysops.com, отправляющей запрос GET к 4sysops.com по протоколу HTTPS. Чтобы указать версию протокола HTTP (например, http/2), используйте опцию —http2:
curl --http2 https://4sysops.comВ случае URL, начинающихся с HTTPS, curl сначала пытается установить соединение http/2 и автоматически откатывается к http/1.1, если это не удаётся. Также он поддерживает другие методы, например, HEAD, POST, PUT и DELETE. Для использования этих методов вместе с командой curl нужно указать опцию —request (или -X), за которой следует указание метода. Стоит заметить, что список доступных методов зависит от используемого протокола.
▍ Получение информации об удалённом файле
Если вы администратор, то иногда вам могут быть интересны только заголовки HTTP. Их можно получить при помощи опции —head (или -I). Иногда URL может перенаправлять пользователя в другую точку. В таком случае опция —location (или -L) позволяет curl выполнять перенаправления. Также можно использовать —insecure (или -k), чтобы разрешить незащищённые подключения и избежать ошибок с сертификатом TLS в случае, если целевой URL использует самоподписанный сертификат. Пользуйтесь этой опцией только при абсолютной необходимости. Все эти три опции можно скомбинировать в одну краткую запись, как показано в следующей команде:
curl -kIL 4sysops.com
Опции просмотра заголовков запросов, включения незащищённого соединения и использования перенаправлений
Как можно заметить, такая краткая запись особенно полезна для комбинирования нескольких опций. Приведённая выше команда по сути эквивалентна команде curl —insecure —head —location 4sysops.com.
Опция —head (или -I) также даёт основную информацию об удалённом файле без его скачивания. Как показано на скриншоте ниже, при использовании curl с URL удалённого файла он отображает различные заголовки, дающие информацию об удалённом файле.
curl -IL https://curl.se/windows/dl-7.85.0_5/curl-7.85.0_5-win64-mingw.zip
Использование curl для просмотра основной информации удалённых файлов
Заголовок Content-Length обозначает размер файла (в байтах), Content-Type сообщает о типе медиафайла (например, image/png, text/html), Server обозначает тип серверного приложения (Apache, Gunicorn и так далее), Last-Modified показывает дату последнего изменения файла на сервере, а заголовок Accept-Ranges обозначает поддержку частичных запросов для скачивания от клиента, что по сути определяет возможность продолжения прерванной загрузки.
▍ Скачивание файла
Для скачивания файла и сохранения с тем же именем, что и на сервере, можно использовать curl с опцией —remote-name (или -O). Показанная ниже команда скачивает последнюю версию curl для Windows с официального сайта:
curl -OL https://curl.se/windows/latest.cgi?p=win64-mingw.zip
Скачивание файла с именем по умолчанию и индикатором прогресса
При необходимости для нахождения ресурса добавляется опция -L, разрешающая перенаправления. Если нужно сохранить файл с новым именем, используйте опцию —output (или -o). Кроме того, при использовании команды curl в скрипте может понадобиться отключить индикатор прогресса, что можно сделать при помощи опции —silent (или -s). Эти две опции можно скомбинировать:
curl -sLo curl.zip https://curl.se/windows/latest.cgi?p=win64-mingw.zip
Скачивание файла без индикатора и сохранение под произвольным именем
▍ Продолжение прерванного скачивания
Наличие Accept-Ranges: bytes в заголовке ответа в буквальном смысле обозначает, что сервер поддерживает скачивания с возможностью продолжения. Чтобы продолжить прерванное скачивание, можно использовать опцию —continue-at (или -C), получающую смещение (в байтах). Обычно указывать смещение непросто, поэтому curl предоставляет простой способ продолжения прерванной загрузки:
curl -OLC - https://releases.ubuntu.com/22.04/ubuntu-22.04.1-desktop-amd64.iso
Продолжение прерванного скачивания
Как видно из скриншота, я скачивал iso-файл Ubuntu, но скачивание было прервано. Затем я снова запустил команду curl с опцией -C, и передача продолжилась с того диапазона байтов, на котором была прервана. Знак минус (—) рядом с -C позволяет curl автоматически определить, как и где продолжить прерванное скачивание.
▍ Аутентификация с Curl
Также Curl поддерживает аутентификацию, что позволяет скачать защищённый файл, предоставив учётные данные при помощи опции —user (or -u), принимающей имя пользователя и пароль в формате username:password. Если не вводить пароль, curl попросит ввести его в режиме no-echo.
curl -u surender -OL https://techtutsonline.com/secretFiles/sample.zip
Скачивание файла с аутентификацией по имени пользователя и паролю
Если вы используете Basic authentication, то необходимо передать имя пользователя и пароль, а значит, воспользоваться защищённым протоколом наподобие HTTPS (вместо HTTP) или FTPS (вместо FTP). Если по каким-то причинам приходится использовать протокол без шифрования, то убедитесь, что вы используете способ аутентификации, не передающий учётные данные в виде простого текста (например, аутентификацию Digest, NTLM или Negotiate).
Также curl поддерживает использование файлов конфигурации .curlrc, _curlrc и .netrc, позволяющих задавать различные опции curl в файле, а затем добавлять файл в команду при помощи опции curl —config (или curl -K), что особенно полезно при написании скриптов.
▍ Выгрузка файла
Опция —upload-file (или -T) позволяет выгружать локальный файл на удалённый сервер. Показанная ниже команда выгружает файл из локальной системы на удалённый веб-сервер по протоколу FTPS:
curl -kT C:\Users\Surender\Downloads\sample1.zip -u testlab\surender ftps://192.168.0.80/awesomewebsite.com/files/
Выгрузка файла на удалённый сервер
Опция -k добавляется для устранения проблем с сертификатами на случай, если веб-сервер использует самоподписанный сертификат. Наклонная черта в конце URL сообщает curl, что конечная точка является папкой. Можно указать несколько имён файлов, например «{sample1.zip,sample2.zip}». Ниже показано, как с помощью одной команды curl можно выгрузить на сервер несколько файлов:
curl -kT sample[1-5].zip -u testlab\surender ftps://192.168.0.80/awesomewebsite.com/files/
Выгрузка нескольких файлов на сервер
▍ Последовательность команд
Как говорилось ранее, curl поддерживает различные методы в зависимости от используемого протокола. Дополнительные команды можно отправлять при помощи —quote (или -Q) для выполнения операции до или после обычной операции curl. Например, можно скачать файл с удалённого сервера по протоколу FTPS и удалить файл с сервера после успешного скачивания. Для этого нужно выполнить следующую команду:
curl -u testlab\surender -kO "ftps://192.168.0.80/awesomewebsite.com/files/sample1.zip" -Q "-DELE sample1.zip"
Удаление файла после успешного скачивания
В показанном выше примере я скачал файл sample1.zip с FTPS-сервера при помощи опции -O. После опции -Q я добавил минус (-) перед командой DELE, что заставляет curl отправить команду DELE sample1.zip сразу после успешного скачивания файла. Аналогично, если вы хотите отправить команду на сервер до выполнения операции curl, используйте плюс (+) вместо минуса.
▍ Изменение user-agent
Информация user-agent сообщает серверу тип клиента, отправляющего запрос. При отправке запроса curl на сервер по умолчанию используется user-agent curl/<version>. Если сервер настроен так, чтобы блокировать запросы curl, можно задать собственный user-agent при помощи опции —user-agent (или -A). Показанная ниже команда отправляет стандартный user-agent Google Chrome:
curl -kIA "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0" https://awesomewebsite.com/files/secretFile.zip
Использование собственного user-agent с командой curl, чтобы избежать блокировки сервером
На показанном выше скриншоте видно, что обычный запрос curl был отклонён веб-сервером (с ответом 403 Forbidden), но при передаче другого user-agent запрос выполняется успешно, возвращая ответ 200 OK.
▍ Отправка куки
По умолчанию запрос curl не отправляет и не сохраняет куки. Для записи куки можно использовать опцию —cookie-jar (или -c), а отправить куки можно опцией —cookie (or -b):
curl -c /path/cookie_file https://awesomewebsite.com/
curl -b /path/cookie_file https://awesomewebsite.com/Первая команда записывает файл куки, а вторая отправляет куки с запросом curl. Также можно отправить куки в формате ‘name = value’:
curl -b 'session=abcxyz' -b 'loggedin=true' http://echo.hoppscotch.io
Отправка нескольких куки командой curl
Я воспользовался веб-сайтом echo.hoppscotch.io для демонстрации заголовков HTTP-запросов, которые обычно невидимы клиентам, отправляющим запрос. Если вы не хотите пользоваться этим веб-сайтом, то можете применить опцию –verbose (или -v) для отображения запроса в сыром виде (который отображает и заголовки запросов).
▍ Использование прокси-сервера
Если вы пользуетесь прокси-сервером для подключения к интернету, в curl можно указать прокси опцией —proxy (или -x). Если прокси-сервер требует аутентификации, то добавьте —proxy-user (или -U):
curl -x 192.168.0.250:8088 -U username:password https://awesomewebsite.com/Прокси-сервер указывается в формате server:port, а пользователь прокси — в формате username:password. Можно не вводить пароль пользователя прокси, тогда curl попросит ввести его в режиме no-echo.
Использование прокси-сервера и аутентификации
▍ Дополнительные заголовки запросов
Иногда вместе с запросом к серверу необходимо отправить дополнительную информацию. В curl это можно сделать при помощи —header (или -H), как показано в следующей команде:
curl -vkIH "x-client-os: Windows 11 Enterprise (x64)" https://awesomewebsite.com
Указание дополнительных заголовков для запроса curl
Можно отправлять любую информацию, недоступную через стандартные заголовки HTTP-запросов. В этом примере я отправил название своей операционной системы. Также я добавил опцию -v для включения verbose-вывода, отображающего дополнительный заголовок, отправляемый вместе с каждым моим запросом curl.
▍ Отправка электронного письма
Так как curl поддерживает протокол SMTP, его можно использовать для отправки электронного письма. Показанная ниже команда позволяет отправить электронное письмо при помощи curl:
curl --insecure --ssl-reqd smtps://mail.yourdomain.com –-mail-from sender@yourdomain.com –-mail-rcpt receiver@company.com --user sender@yourdomain.com --upload-file email_msg.txt
Отправка электронного письма командой curl
Давайте вкратце перечислим использованные здесь опции:
- Опция —insecure (или -k) используется, чтобы избежать ошибки сертификата SSL. Мы уже применяли её ранее.
- Опция —ssl-reql используется для апгрейда соединения передачи простого текста до зашифрованного соединения, если оно поддерживается SMTP-сервером. Если вы уверены, что ваш SMTP-сервер поддерживает SSL, то можно использовать непосредственно имя сервера smtps (например, smtps://smtp.yourdomain.com), как показано на скриншоте.
- Опция —mail-from используется для указания адреса электронной почты отправителя.
- Опция mail-rcpt указывает адрес электронной почты получателя.
- Опция —user (или -u) отправляет имя пользователя для аутентификации, оно должно совпадать с адресом mail-from, потому что в противном случае письмо может быть отклонено или помечено как спам.
- Опция —upload-file (или -T) используется для указания файла, в котором находится отправляемое письмо.
На скриншоте ниже показано письмо, полученное мной во входящие:
Просмотр письма, отправленного с помощью curl
Это всего лишь несколько примеров использования curl — на самом деле их гораздо больше. Я настоятельно рекомендую проверить справку по curl и поэкспериментировать с ней.
А вы используете curl? И если да, то для чего?
Telegram-канал с полезностями и уютный чат
Curl (client URL) is a command-line tool powered by the libcurl library to transfer data to and from the server using various protocols, such as HTTP, HTTPS, FTP, FTPS, IMAP, IMAPS, POP3, POP3S, SMTP, and SMTPS. It is highly popular for automation and scripts due to its wide range of features and protocol support. In this article, you will learn how to use curl in Windows with various examples. Let’s get started.
Contents
- Install curl on Windows
- Curl syntax
- HTTP GET request
- Get remote file information.
- Download a file
- Resume interrupted download
- Authentication with Curl
- Upload a file
- Quote a command
- Change the user-agent
- Send a cookie
- Use a proxy server
- Additional request headers
- Send an email
- Author
- Recent Posts
Surender Kumar has more than twelve years of experience in server and network administration. His fields of interest are Windows servers, Active directory, PowerShell, web servers, networking, Linux, virtualization, and Kubernetes. He loves writing for his blog.
Install curl on Windows
All the modern Windows versions, starting with Windows 10 (version 1803) and Server 2019, have the curl executable pre-installed, so there is no need for a manual installation. To determine the curl location and version in your system, you can use the following commands:
where curl curl --version
Determine the location and version of curl in Windows
The curl —version command also lists the protocols and features supported by the current curl version. If you see an output, as shown in the screenshot above, you’re all set to use the built-in curl utility. If you get an error message instead, curl might not be available, probably because you’re on an earlier version of Windows (e.g., Windows 8.1 or Server 2016). In that case, you might need to manually setup curl in Windows.
Curl syntax
The curl command uses the following syntax:
curl [options...] [url]
It supports various options, which we will discuss later in this post. As with any other command-line tool, you can use the curl —help command to get help.
Getting help with the curl command
To get detailed help, you can use curl —help all. The help section is divided into categories, so the curl —help category gets you an overview of all the categories.
Now that you’ve become familiar with curl syntax, let’s discuss various use cases with the help of examples.
HTTP GET request
When you use curl against a URL without specifying any option, the request defaults to the GET method of the HTTP protocol. Try this:
curl https://4sysops.com
The above command is essentially equivalent to curl —request GET https://4sysops.com, which sends a GET request to 4sysops.com using the HTTPS protocol. To specify the HTTP protocol version (e.g., http/2), use the —http2 option, as shown below:
curl --http2 https://4sysops.com
For URLs starting with HTTPS, curl first tries to negotiate to establish a http/2 connection and automatically falls back to http/1.1 if the negotiation fails. It also supports other methods, such as HEAD, POST, PUT, and DELETE. To use these methods, along with the curl command, use the —request (or -X) option, followed by the method. Notice that the methods that are available depend on the protocol being used.
Get remote file information.
As an admin, you might want to be interested in HTTP headers only. This can be done using the —head (or -I) option. Sometimes, a URL might redirect you to another location. In that case, —location (or -L) allows the curl to follow the redirects. You can also use —insecure (or -k) to allow insecure connections to avoid any TLS certificate errors if the target URL is using a self-signed certificate. Use this only when absolutely necessary. All three of these options can be combined in short-notation, as shown in the following command:
curl -kIL 4sysops.com
View request headers allow insecure connection and follow redirect options with curl
You can see that short-notation is particularly useful for combining multiple options. The above command is essentially equivalent to the curl —insecure —head —location 4sysops.com command.
The —head (or -I) option also gives you basic information about a remote file without actually downloading it. As shown in the screenshot below, when you use curl with a remote file URL, it displays various headers to give you information about the remote file.
curl -IL https://curl.se/windows/dl-7.85.0_5/curl-7.85.0_5-win64-mingw.zip
Use curl to view the basic information about remote files
The Content-Length header indicates the size of the file (in bytes), Content-Type reveals the media type of the file (for instance image/png, text/htm), Server indicates the type of server application (Apache, Gunicron, etc.), Last-Modified shows the date when file was last changed on the server, and the Accept-Ranges header indicates the support of partial requests from the client for downloads, which essentially means you can resume an interrupted download.
Download a file
You can use curl with the —remote-name option (or -O, in short) to download a file and save it with the same name as on the server. The following command downloads the latest version of curl for Windows from the official website:
curl -OL https://curl.se/windows/latest.cgi?p=win64-mingw.zip
Downloading a file with a default name and progress indicator using curl
The -L option is added to follow redirects, if needed, for locating the resource. If you want to save the file with a new name, use the —output (or -o) option instead. Furthermore, while using the curl command in a script, you might want to suppress the progress indicator using —silent (or -s). Both options can be combined, as shown in the following command:
curl -sLo curl.zip https://curl.se/windows/latest.cgi?p=win64-mingw.zip
Silently download a file and save with a custom name using curl
Resume interrupted download
The presence of Accept-Ranges: bytes in the response header literally means that the server supports resumable downloads. To resume an interrupted download, you can use —continue-at (or -C), which accepts an offset (in bytes). Generally, specifying an offset is tricky, so curl offers an easy way of resuming an interrupted download:
curl -OLC - https://releases.ubuntu.com/22.04/ubuntu-22.04.1-desktop-amd64.iso
Resuming an interrupted download with curl
As you can see in the screenshot, I was downloading an Ubuntu iso file, which was interrupted. When I ran the curl command again with the -C option, the transfer was resumed from the byte range where it was interrupted. The minus sign (—) next to -C allows the curl to automatically figure out how and where to resume the interrupted download.
Authentication with Curl
Curl also supports authentication, allowing you to download a protected file by supplying credentials with the —user (or -u) option, which accepts a username and password in the username:password format. If you skip typing the password, curl will prompt you to type it in no-echo mode.
curl -u surender -OL https://techtutsonline.com/secretFiles/sample.zip
Downloading a file using username and password authentication with curl
If you use a basic authentication method, you have to transfer a username and password, which means that you should use a secure protocol such as HTTPS (instead of HTTP) or FTPS (instead of FTP). If, for some reason, you have to use an unencrypted protocol, make sure you use an authentication method that doesn’t transmit credentials in clear text (for instance, Digest, NTLM, or Negotiate authentication).
Curl also supports the use of .curlrc, _curlrc, and .netrc config files, allowing you to define various curl options in a file and then to include the file in your command with curl —config (or curl -K), which is particularly useful for scripting.
Upload a file
The —upload-file (or -T) option allows you to upload a local file to a remote server. The following command shows how to upload a file from a local system to a remote web server using the FTPS protocol:
curl -kT C:\Users\Surender\Downloads\sample1.zip -u testlab\surender ftps://192.168.0.80/awesomewebsite.com/files/
Uploading a file to a remote server using curl
The -k option is included to avoid certificate errors if the web server uses a self-signed certificate. The trailing slash at the end of the URL tells curl that the destination is a directory. You could specify multiple file names, such as “{sample1.zip,sample2.zip}.” The following command shows how to upload multiple files with a single curl command:
curl -kT sample[1-5].zip -u testlab\surender ftps://192.168.0.80/awesomewebsite.com/files/
Upload multiple files to a remote server using curl
Quote a command
As already discussed, curl supports various methods based on the underlying protocol being used. You can send additional commands using —quote (or -Q) to perform a particular operation either before or after the regular curl operation; for instance, if you want to download a file from a remote server using the FTPS protocol and want the file to be removed from the server once it has been downloaded successfully. To do this, you can run the command shown below:
curl -u testlab\surender -kO "ftps://192.168.0.80/awesomewebsite.com/files/sample1.zip" -Q "-DELE sample1.zip"
Delete a file after successful download using curl command
Here, I downloaded the sample1.zip file from an FTPS server with the help of the -O option. After the -Q option, I added a minus sign (-) just before the DELE command, which tells the curl to send the DELE sample1.zip command immediately after the file is downloaded successfully. Likewise, if you want to send a command to the server before performing the actual curl operation, use a plus (+) sign instead of a minus sign.
Change the user-agent
The user-agent tells a server what type of client is sending the request. When you send a curl request to the server, the curl/<version> user-agent is used by default. If the server is configured to block the curl requests, you can specify a custom user-agent using —user-agent (or -A). The following command sends a common Google Chrome user-agent:
curl -kIA "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0" https://awesomewebsite.com/files/secretFile.zip
Use a custom user agent with a curl command to avoid server blocks
The above screenshot shows that a normal curl request was forbidden by the web server (with a 403 Forbidden response), but when I passed a custom user-agent, the request was successful, returning a 200 OK response.
Send a cookie
By default, the curl request does not send or store cookies. To write a cookie, use the —cookie-jar (or -c) option, and with —cookie (or -b), you can send a cookie:
curl -c /path/cookie_file https://awesomewebsite.com/ curl -b /path/cookie_file https://awesomewebsite.com/
The first command writes a cookie file, and the second command sends the cookie with a curl request. You can also send a cookie in ‘name = value’‘ format, as shown below:
curl -b 'session=abcxyz' -b 'loggedin=true' http://echo.hoppscotch.io
Send multiple cookies using a curl command
I used the echo.hoppscotch.io website to view HTTP request headers that aren’t normally visible to clients sending a request. If you don’t want to use this website, you could use the –verbose (or -v) option to see your request in raw form (which will show request headers, too).
Use a proxy server
Do you use a proxy server to connect to the internet? No problem! Curl lets you specify a proxy server using the —proxy (or -x) option. If your proxy server requires authentication, add —proxy-user (or -U):
curl -x 192.168.0.250:8088 -U username:password https://awesomewebsite.com/
The proxy server is specified in the server:port format, and the proxy user is specified in the username:password format. Again, you could skip typing the password for the proxy user, and curl will prompt you to enter it in no-echo mode.
Use a proxy server and authentication with a curl command
Additional request headers
Sometimes, you might want to send additional information along with your request to the server. With curl, you can do so easily by using —header (or -H), as shown in the following command:
curl -vkIH "x-client-os: Windows 11 Enterprise (x64)" https://awesomewebsite.com
Specify additional headers with a curl request
You could send any information that isn’t available with standard HTTP request headers. In this example, I sent my operating system name. I also added the -v option this time to enable verbose output, which displayed the additional header being sent along with my curl request.
Send an email
Since curl supports the SMTP protocol, you could use it to send an email message. The following command shows how to send an email using curl:
curl --insecure --ssl-reqd smtps://mail.yourdomain.com –-mail-from sender@yourdomain.com –-mail-rcpt receiver@company.com --user sender@yourdomain.com --upload-file email_msg.txt
Send an email message using a curl command
Let’s quickly discuss the options used:
- The —insecure (or -k) command is used to avoid an SSL certificate error. We have used this before.
- The —ssl-reql option is used to upgrade a plain-text connection to encrypted connection if supported by the SMTP server. Alternatively, if you’re sure your SMTP server supports SSL, you could directly use the smtps server name (e.g., smtps://smtp.yourdomain.com), as you can see in the screenshot.
- The —mail-from option is used to define the sender’s (from) email address.
- The mail-rcpt option specifies the recipient’s email address.
- The —user (or -u) option sends the username for authentication, which should match the mail-from address, because otherwise your message might be rejected or flagged as spam.
- The —upload-file (or -T) option is used to specify a file that contains the email message to send.
The following screenshot shows the email message I received in my inbox:
Viewing the email message sent with curl
Viewing the email message sent with curl
Subscribe to 4sysops newsletter!
These are just a few examples, but there is a lot more you can do with curl. I highly recommend checking out curl help and experimenting with it. You’ll notice what a powerful command curl is.
cURL is a command line tool and a library which can be used to receive and send data between a client and a server or any two machines connected over the internet. It supports a wide range of protocols like HTTP, FTP, IMAP, LDAP, POP3, SMTP and many more.
Due to its versatile nature, cURL is used in many applications and for many use cases. For example, the command line tool can be used to download files, testing APIs and debugging network problems. In this article, we shall look at how you can use the cURL command line tool to perform various tasks.
Install cURL
Linux
Most Linux distributions have cURL installed by default. To check whether it is installed on your system or not, type curl in your terminal window and press enter. If it isn’t installed, it will show a “command not found” error. Use the commands below to install it on your system.
For Ubuntu/Debian based systems use:
sudo apt update sudo apt install curl
For CentOS/RHEL systems, use:
sudo yum install curl
On the other hand, for Fedora systems, you can use the command:
sudo dnf install curl
MacOS
MacOS comes with cURL preinstalled, and it receives updates whenever Apple releases updates for the OS. However, in case you want to install the most recent version of cURL, you can install the curl Homebrew package. Once you install Homebrew, you can install it with:
brew install curl
Windows
For Windows 10 version 1803 and above, cURL now ships by default in the Command Prompt, so you can use it directly from there. For older versions of Windows, the cURL project has Windows binaries. Once you download the ZIP file and extract it, you will find a folder named curl-<version number>-mingw. Move this folder into a directory of your choice. In this article, we will assume our folder is named curl-7.62.0-win64-mingw, and we have moved it under C:\.
Next, you should add cURL’s bin directory to the Windows PATH environment variable, so that Windows can find it when you type curl in the command prompt. For this to work, you need to follow these steps:
- Open the “Advanced System Properties” dialog by running
systempropertiesadvancedfrom the Windows Run dialog (Windows key + R). - Click on the “Environment Variables” button.
- Double-click on “Path” from the “System variables” section, and add the path
C:\curl-7.62.0-win64-mingw\bin. For Windows 10, you can do this with the “New” button on the right. On older versions of Windows, you can type in;C:\curl-7.62.0-win64-mingw\bin(notice the semicolon at the beginning) at the end of the “Value” text box.
Once you complete the above steps, you can type curl to check if this is working. If everything went well, you should see the following output:
C:\Users\Administrator>curl curl: try 'curl --help' or 'curl --manual' for more information
cURL basic usage
The basic syntax of using cURL is simply:
curl <url>
This fetches the content available at the given URL, and prints it onto the terminal. For example, if you run curl example.com, you should be able to see the HTML page printed, as shown below:
This is the most basic operation cURL can perform. In the next few sections, we will look into the various command line options accepted by cURL.
Downloading Files with cURL
As we saw, cURL directly downloads the URL content and prints it to the terminal. However, if you want to save the output as a file, you can specify a filename with the -o option, like so:
curl -o vlc.dmg http://ftp.belnet.be/mirror/videolan/vlc/3.0.4/macosx/vlc-3.0.4.dmg
In addition to saving the contents, cURL switches to displaying a nice progress bar with download statistics, such as the speed and the time taken:
Instead of providing a file name manually, you can let cURL figure out the filename with the -O option. So, if you want to save the above URL to the file vlc-3.0.4.dmg, you can simply use:
curl -O http://ftp.belnet.be/mirror/videolan/vlc/3.0.4/macosx/vlc-3.0.4.dmg
Bear in mind that when you use the -o or the -O options and a file of the same name exists, cURL will overwrite it.
If you have a partially downloaded file, you can resume the file download with the -C - option, as shown below:
curl -O -C - http://ftp.belnet.be/mirror/videolan/vlc/3.0.4/macosx/vlc-3.0.4.dmg
Like most other command line tools, you can combine different options together. For example, in the above command, you could combine -O -C - and write it as -OC - .
Anatomy of a HTTP request/response
Before we dig deeper into the features supported by cURL, we will discuss a little bit about HTTP requests and responses. If you are familiar with these concepts, you directly skip to the other sections.
To request a resource such as a webpage, or to submit some data to a server, a HTTP client (such as a browser or cURL) makes a HTTP request to the server The server responds back with a HTTP response, which contains the “contents” of that page.
HTTP requests contain the request method, URL, some headers, and some optional data as part of the “request body”. The request method controls how a certain request should be processed. The most common types of request methods are “GET” and “POST”. Typically, we use “GET” requests to retrieve a resource from the server, and “POST” to submit data to the server for processing. “POST” requests typically contain some data in the request body, which the server can use.
HTTP responses are similar and contain the status code, some headers, and a body. The body contains the actual data that clients can display or save to a file. The status code is a 3-digit code which tells the client if the request succeeded or failed, and how it should proceed further. Common status codes are 2xx (success), 3xx (redirect to another page), and 4xx/5xx (for errors).
HTTP is an “application layer protocol”, and it runs over another protocol called TCP. It takes care of retransmitting any lost data, and ensures that the client and server transmit data at an optimal rate. When you use HTTPS, another protocol called SSL/TLS runs between TCP and HTTP to secure the data.
Most often, we use domain names such as google.com to access websites. Mapping the domain name to an IP address occurs through another protocol called DNS.
You should now have enough background to understand the rest of this article.
Following redirects with cURL
By default, when cURL receives a redirect after making a request, it doesn’t automatically make a request to the new URL. As an example of this, consider the URL http://www.facebook.com. When you make a request using to this URL, the server sends a HTTP 3XX redirect to https://www.facebook.com/. However, the response body is otherwise empty. So, if you try this out, you will get an empty output:
If you want cURL to follow these redirects, you should use the -L option. If you repeat make a request for http://www.facebook.com/ with the -L flag, like so:
curl -L http://www.facebook.com/
Now, you will be able to see the HTML content of the page, similar to the screenshot below. In the next section, we will see how we can verify that there is a HTTP 3XX redirect.
Please bear in mind that cURL can only follow redirects if the server replied with a “HTTP redirect”, which means that the server used a 3XX status code, and it used the “Location” header to indicate the new URL. cURL cannot process Javascript or HTML-based redirection methods, or the “Refresh header“.
If there is a chain of redirects, the -L option will only follow the redirects up to 500 times. You can control the number of maximum redirects that it will follow with the --max-redirs flag.
curl -L --max-redirs 700 example.com
If you set this flag to -1, it will follow the redirects endlessly.
curl -L --max-redirs -1 example.com
When debugging issues with a website, you may want to view the HTTP response headers sent by the server. To enable this feature, you can use the -i option.
Let us continue with our previous example, and confirm that there is indeed a HTTP 3XX redirect when you make a HTTP request to http://www.facebook.com/, by running:
curl -L -i http://www.facebook.com/
Notice that we have also used -L so that cURL can follow redirects. It is also possible to combine these two options and write them as -iL or -Li instead of -L -i.
Once you run the command, you will be able to see the HTTP 3XX redirect, as well as the page HTTP 200 OK response after following the redirect:
If you use the -o/-O option in combination with -i, the response headers and body will be saved into a single file.
Viewing request headers and connection details
In the previous section, we have seen how you can view HTTP response headers using cURL. However, sometimes you may want to view more details about a request, such as the request headers sent and the connection process. cURL offers the -v flag (called “verbose mode”) for this purpose, and it can be used as follows:
curl -v https://www.booleanworld.com/
The output contains request data (marked with >), response headers (marked with <) and other details about the request, such as the IP used and the SSL handshake process (marked with *). The response body is also available below this information. (However, this is not visible in the screenshot below).
Most often, we aren’t interested in the response body. You can simply hide it by “saving” the output to the null device, which is /dev/null on Linux and MacOS and NUL on Windows:
curl -vo /dev/null https://www.booleanworld.com/ # Linux/MacOS curl -vo NUL https://www.booleanworld.com/ # Windows
Silencing errors
Previously, we have seen that cURL displays a progress bar when you save the output to a file. Unfortunately, the progress bar might not be useful in all circumstances. As an example, if you hide the output with -vo /dev/null, a progress bar appears which is not at all useful.
You can hide all these extra outputs by using the -s header. If we continue with our previous example but hide the progress bar, then the commands would be:
curl -svo /dev/null https://www.booleanworld.com/ # Linux/MacOS curl -svo NUL https://www.booleanworld.com/ # Windows
The -s option is a bit aggressive, though, since it even hides error messages. For your use case, if you want to hide the progress bar, but still view any errors, you can combine the -S option.
So, if you are trying to save cURL output to a file but simply want to hide the progress bar, you can use:
curl -sSvo file.html https://www.booleanworld.com/
When testing APIs, you may need to set custom headers on the HTTP request. cURL has the -H option which you can use for this purpose. If you want to send the custom header X-My-Custom-Header with the value of 123 to https://httpbin.org/get, you should run:
curl -H 'X-My-Custom-Header: 123' https://httpbin.org/get
(httpbin.org is a very useful website that allows you to view details of the HTTP request that you sent to it.)
The data returned by the URL shows that this header was indeed set:
You can also override any default headers sent by cURL such as the “User-Agent” or “Host” headers. The HTTP client (in our case, cURL) sends the “User-Agent” header to tell the server about the type and version of the client used. Also, the client uses the “Host” header to tell the server about the site it should serve. This header is needed because a web server can host multiple websites at a single IP address.
Also, if you want to set multiple headers, you can simply repeat the -H option as required.
curl -H 'User-Agent: Mozilla/5.0' -H 'Host: www.google.com' ...
However, cURL does have certain shortcuts for frequently used flags. You can set the “User-Agent” header with the -A option:
curl -A Mozilla/5.0 http://httpbin.org/get
The “Referer” header is used to tell the server the location from which they were referred to by the previous site. It is typically sent by browsers when requesting Javascript or images linked to a page, or when following redirects. If you want to set a “Referer” header, you can use the -e flag:
curl -e http://www.google.com/ http://httpbin.org/get
Otherwise, if you are following a set of redirects, you can simply use -e ';auto' and cURL will take care of setting the redirects by itself.
Making POST requests with cURL
By default, cURL sends GET requests, but you can also use it to send POST requests with the -d or --data option. All the fields must be given as key=value pairs separated by the ampersand (&) character. As an example, you can make a POST request to httpbin.org with some parameters:
curl --data "firstname=boolean&lastname=world" https://httpbin.org/post
From the output, you can easily tell that we posted two parameters (this appears under the “form” key):
Any special characters such as @, %, = or spaces in the value should be URL-encoded manually. So, if you wanted to submit a parameter “email” with the value “[email protected]”, you would use:
curl --data "email=test%40example.com" https://httpbin.org/post
Alternatively, you can just use --data-urlencode to handle this for you. If you wanted to submit two parameters, email and name, this is how you should use the option:
curl --data-urlencode "[email protected]" --data-urlencode "name=Boolean World" https://httpbin.org/post
If the --data parameter is too big to type on the terminal, you can save it to a file and then submit it using @, like so:
curl --data @params.txt example.com
So far, we have seen how you can make POST requests using cURL. If you want to upload files using a POST request, you can use the -F (“form”) option. Here, we will submit the file test.c, under the parameter name file:
curl -F [email protected] https://httpbin.org/post
This shows the content of the file, showing that it was submitted successfully:
Submitting JSON data with cURL
In the previous section, we have seen how can submit POST requests using cURL. You can also submit JSON data using the --data option. However, most servers expect to receive a POST request with key-value pairs, similar to the ones we have discussed previously. So, you need to add an additional header called ‘Content-Type: application/json’ so that the server understands it’s dealing with JSON data and handles it appropriately. Also, you don’t need to URL-encode data when submitting JSON.
So if you had the following JSON data and want to make a POST request to https://httpbin.org/post:
{
"email": "[email protected]",
"name": ["Boolean", "World"]
}
Then, you can submit the data with:
curl --data '{"email":"[email protected]", "name": ["Boolean", "World"]}' -H 'Content-Type: application/json' https://httpbin.org/post
In this case, you can see the data appear under the json value in the httpbin.org output:
You can also save the JSON file, and submit it in the same way as we did previously:
curl --data @data.json https://httpbin.org/post
Changing the request method
Previously, we have seen how you can send POST requests with cURL. Sometimes, you may need to send a POST request with no data at all. In that case, you can simply change the request method to POST with the -X option, like so:
curl -X POST https://httpbin.org/post
You can also change the request method to anything else, such as PUT, DELETE or PATCH. One notable exception is the HEAD method, which cannot be set with the -X option. The HEAD method is used to check if a document is present on the server, but without downloading the document. To use the HEAD method, use the -I option:
curl -I https://www.booleanworld.com/
When you make a HEAD request, cURL displays all the request headers by default. Servers do not send any content when they receive a HEAD request, so there is nothing after the headers:
Replicating browser requests with cURL
If you want to replicate a request made through your browser through cURL, you can use the Chrome, Firefox and Safari developer tools to get a cURL command to do so.
The steps involved are the same for all platforms and browsers:
- Open developer tools in Firefox/Chrome (typically F12 on Windows/Linux and Cmd+Shift+I on a Mac)
- Go to the network tab
- Select the request from the list, right click it and select “Copy as cURL”
The copied command contains all the headers, request methods, cookies etc. needed to replicate the exact same request. You can paste the command in your terminal to run it.
Making cURL fail on HTTP errors
Interestingly, cURL doesn’t differentiate between a successful HTTP request (2xx) and a failed HTTP request (4xx/5xx). So, it always returns an exit status of 0 as long as there was no problem connecting to the site. This makes it difficult to write shell scripts because there is no way to check if the file could be downloaded successfully.
You can check this by making a request manually:
curl https://www.booleanworld.com/404 -sSo file.txt
You can see that curl doesn’t print any errors, and the exit status is also zero:
If you want to consider these HTTP errors as well, you can use the -f option, like so:
curl https://www.booleanworld.com/404 -fsSo file.txt
Now, you can see that cURL prints an error and also sets the status code to 22 to inform that an error occured:
Making authenticated requests with cURL
Some webpages and APIs require authentication with an username and password. There are two ways to do this. You can mention the username and password with the -u option:
curl -u boolean:world https://example.com/
Alternatively, you can simply add it to the URL itself, with the <username>:<password>@<host> syntax, as shown:
curl https://boolean:[email protected]/
In both of these methods, curl makes a “Basic” authentication with the server.
Testing protocol support with cURL
Due to the wide range of protocols supported by cURL, you can even use it to test protocol support. If you want to check if a site supports a certain version of SSL, you can use the --sslv<version> or --tlsv<version> flags. For example, if you want to check if a site supports TLS v1.2, you can use:
curl -v --tlsv1.2 https://www.booleanworld.com/
The request takes place normally, which means that the site supports TLSv1.2. Now, let us check if the site supports SSL v3:
curl -v --sslv3 https://www.booleanworld.com/
This command throws a handshake_failed error, because the server doesn’t support this version of SSL.
Please note that, depending on your system and the library version/configuration, some of these version options may not work. The above output was taken from Ubuntu 16.04’s cURL. However, if you try this with cURL in MacOS 10.14, it gives an error:
You can also test for HTTP protocol versions in the same way, by using the flags --http1.0, --http1.1 or --http2.
Setting the Host header and cURL’s --resolve option
Previously, we have discussed about how a web server chooses to serve different websites to visitors depending upon the “Host” header. This can be very useful to check if your website has virtual hosting configured correctly, by changing the “Host” header. As an example, say you have a local server at 192.168.0.1 with two websites configured, namely example1.com and example2.com. Now, you can test if everything is configured correctly by setting the Host header and checking if the correct contents are served:
curl -H 'Host: example1.com' http://192.168.0.1/ curl -H 'Host: example1.com' http://192.168.0.1/
Unfortunately, this doesn’t work so well for websites using HTTPS. A single website may be configured to serve multiple websites, with each website using its own SSL/TLS certificate. Since SSL/TLS takes place at a lower level than HTTP, this means clients such as cURL have to tell the server which website we’re trying to access at the SSL/TLS level, so that the server can pick the right certificate. By default, cURL always tells this to the server.
However, if you want to send a request to a specific IP like the above example, the server may pick a wrong certificate and that will cause the SSL/TLS verification to fail. The Host header only works at the HTTP level and not the SSL/TLS level.
To avoid the problem described above, you can use the --resolve flag. The resolve flag will send the request to the port and IP of your choice but will send the website name at both SSL/TLS and HTTP levels correctly.
Let us consider the previous example. If you were using HTTPS and wanted to send it to the local server 192.168.0.1, you can use:
curl https://example1.com/ --resolve example1.com:192.168.0.1:443
It also works well for HTTP. Suppose, if your HTTP server was serving on port 8080, you can use either the --resolve flag or set the Host header and the port manually, like so:
curl http://192.168.0.1:8080/ -H 'Host: example1.com:8080' curl http://example.com/ --resolve example1.com:192.168.0.1:8080
The two commands mentioned above are equivalent.
Resolve domains to IPv4 and IPv6 addresses
Sometimes, you may want to check if a site is reachable over both IPv4 or IPv6. You can force cURL to connect to either the IPv4 or over IPv6 version of your site by using the -4 or -6 flags.
Please bear in mind that a website can be reached over IPv4 and IPv6 only if:
- There are appropriate DNS records for the website that links it to IPv4 and IPv6 addresses.
- You have IPv4 and IPv6 connectivity on your system.
For example, if you want to check if you can reach the website icanhazip.com over IPv6, you can use:
curl -6 https://icanhazip.com/
If the site is reachable over HTTPS, you should get your own IPv6 address in the output. This website returns the public IP address of any client that connects to it. So, depending on the protocol used, it displays an IPv4 or IPv6 address.
You can also use the -v option along with -4 and -6 to get more details.
Disabling cURL’s certificate checks
By default, cURL checks certificates when it connects over HTTPS. However, it is often useful to disable the certificate checking, when you are trying to make requests to sites using self-signed certificates, or if you need to test a site that has a misconfigured certificate.
To disable certificate checks, use the -k certificate. We will test this by making a request to expired.badssl.com, which is a website using an expired SSL certificate.
curl -k https://expired.badssl.com/
With the -k option, the certificate checks are ignored. So, cURL downloads the page and displays the request body successfully. On the other hand, if you didn’t use the -k option, you will get an error, similar to the one below:
Troubleshooting website issues with “cURL timing breakdown”
You may run into situations where a website is very slow for you, and you would like to dig deeper into the issue. You can make cURL display details of the request, such as the time taken for DNS resolution, establishing a connection etc. with the -w option. This is often called as a cURL “timing breakdown”.
As an example, if you want to see these details for connecting to https://www.booleanworld.com/, run:
curl https://www.booleanworld.com/ -sSo /dev/null -w 'namelookup:\t%{time_namelookup}\nconnect:\t%{time_connect}\nappconnect:\t%{time_appconnect}\npretransfer:\t%{time_pretransfer}\nredirect:\t%{time_redirect}\nstarttransfer:\t%{time_starttransfer}\ntotal:\t\t%{time_total}\n'
(If you are running this from a Windows system, change the /dev/null to NUL).
You will get some output similar to this:
Each of these values is in seconds, and here is what each value represents:
- namelookup — The time required for DNS resolution.
- connect — The time required to establish the TCP connection.
- appconnect — This is the time taken to establish connections for any layers between TCP and the application layer, such as SSL/TLS. In our case, the application layer is HTTP. Also, if there is no such intermediate layer (such as when there is a direct HTTP request), this time will always be 0.
- pretransfer — This is the time taken from the start to when the transfer of the file is just about to begin.
- redirect — This is the total time taken to process any redirects.
- starttransfer — Time it took from the start to when the first byte is about to be transferred.
- total — The total time taken for cURL to complete the entire process.
As an example, say, you are facing delays connecting to a website and you notice the “namelookup” value was too high. As this indicates a problem with your ISP’s DNS server, you may start looking into why the DNS lookup is so slow, and switch to another DNS server if needed.
cURL configuration files
Sometimes, you may want to make all cURL requests use the same options. Passing these options by hand isn’t a feasible solution, so cURL allows you to specify options in a configuration file.
The default configuration file is located in ~/.curlrc in Linux/MacOS and %appdata%\_curlrc in Windows. Inside this file, you can specify any options that you need, such as:
# Always use IPv4 -4 # Always show verbose output -v # When following a redirect, automatically set the previous URL as referer. referer = ";auto" # Wait 60 seconds before timing out. connect-timeout = 60
After creating the above file, try making a request with curl example.com. You will find that these options have taken effect.
If you want to use a custom configuration file instead of the default one, then you can use -K option to point curl to your configuration file. As an example, if you have a configuration file called config.txt, then you can use it with:
curl -K config.txt example.com
Conclusion
In this article, we have covered the most common uses of the cURL command. Of course, this article only scratches the surface and cURL can do a whole lot of other things. You can type man curl in your terminal or just visit this page to see the man page which lists all the options.
The cURL command, short for “Client for URLs“, is a versatile command-line tool for transferring data across various network protocols. Supporting an extensive array of protocols such as HTTP, HTTPS, FTP, SMTP, and many more, it has found a central place in the toolkit of many network and system administrators.
Starting with Windows 10, Microsoft has included cURL by default in the Command Prompt. This means that Windows 11 also supports cURL commands without needing to install additional software. This comprehensive guide will focus on using cURL in the Command Prompt on Windows 11, covering various examples such as downloading and uploading files, sending an email, managing cookies, handling HTTP headers, dealing with SSL certificates, and more.
Verifying cURL Installation in Windows 11
First, let’s verify that cURL is installed on your system. Open Command Prompt and type in the following command:
curl --version
This command should display the installed version of cURL on your system, confirming its installation.
Also see: How to use FTP via Command Line (CMD) on Windows 11
Downloading Files with cURL
One of the most common uses of cURL is downloading files from the web. The process is simple and intuitive. To download a file, use the -O (or --remote-name) option followed by the URL of the file:
curl -O https://example.com/myfile.txt
This command will download the file myfile.txt from example.com and save it in the current working directory.
To save the downloaded file with a different name, use the -o (or --output) option:
curl -o renamedfile.txt https://example.com/myfile.txt
The -o option followed by a filename allows you to specify a different name for the downloaded file.
Related resource: How to Download an FTP File Using CMD in Windows 11/10
Uploading a File using cURL
cURL enables file uploads to servers using several protocols, such as FTP, SFTP, and HTTP/HTTPS. Here’s an example using FTP:
curl -T localfile.txt ftp://example.com/ --user username:password
This command will upload localfile.txt from your local machine to example.com via FTP. Replace username:password with your actual username and password for the FTP server.
Sending an Email with cURL
cURL’s support for the SMTP protocol means you can use it to send emails. This is particularly useful for sending automated notification emails from scripts. Here’s an example:
curl --url smtps://smtp.example.com --ssl-reqd --mail-from [email protected] --mail-rcpt recei[email protected] --upload-file email.txt --user username:password
In this example, email.txt is a text file containing the email’s subject, headers, and body. Replace smtp.example.com, [email protected], [email protected], and username:password with your SMTP server details and email information.
Managing Cookies with cURL
Cookies can be managed in cURL commands by using the -b (or --cookie) option to send cookies, and the -c (or --cookie-jar) option to store cookies. Here’s an example of sending a cookie:
curl -b "name=value" https://example.com
And here’s an example of storing the server’s cookies into a file:
curl -c cookies.txt https://example.com
Setting HTTP Headers with cURL
In many instances, particularly when working with APIs, you may need to set specific HTTP headers in your request. This can be achieved with cURL using the -H (or --header) option:
curl -H "Content-Type: application/json" -H "Authorization: Bearer your_token" https://api.example.com
Working with SSL Certificates
cURL allows you to troubleshoot SSL certificates, check their validity, and even download them. To check the validity of a certificate, you can use the following command:
curl -vI https://example.com 2>&1 | findstr "expire date"
In this example, the command uses cURL to check the validity of an SSL certificate by retrieving the server’s response headers and searching for the “expire date” string. It allows for quick verification of SSL certificate expiration dates.
Linked issue: Fixing Website’s Security Certificate Error on Windows 11/10
Making POST Requests with cURL
A POST request is a way to send data to a server. With cURL, making POST requests is straightforward. Let’s say we want to send some JSON data to an API. We can do so like this:
curl -d "{'key1':'value1', 'key2':'value2'}" -H "Content-Type: application/json" -X POST https://api.example.com
The -d (or --data) option specifies the data to be sent, while the -H (or --header) option sets the Content-Type header to application/json. The -X (or --request) option specifies that this is a POST request.
cURL and REST APIs
cURL is also invaluable when working with REST APIs, which use standard HTTP methods like GET, POST, PUT, DELETE, etc. We have already seen examples of GET (downloading files) and POST (sending data) operations. Now let’s look at how to use cURL for a PUT request, typically used for updating resources:
curl -d "{'key1':'value1', 'key2':'value2'}" -H "Content-Type: application/json" -X PUT https://api.example.com/resource/1
And here’s how to make a DELETE request, used to delete a resource:
curl -X DELETE https://api.example.com/resource/1
Concluding Thoughts
Running cURL from the Command Prompt in Windows 11 is not only a simple process, but also a powerful tool that allows you to interact with web services directly from your machine, automating tasks, testing web servers, and much more. It’s a tool that is as powerful as it is ubiquitous – cURL is essentially available on every platform, making your scripts portable and universal.
If you’re ever unsure about a cURL command or need to check how to write one, don’t forget you have a handy helper: the curl --help command. This is like a mini instruction book right at your fingertips, giving you a fast rundown of what cURL can do. It’s the first place to go if you have questions or need help with cURL.
CURL — утилита командной строки для Linux или Windows, поддерживает работу с протоколами: FTP, FTPS, HTTP, HTTPS, TFTP, SCP, SFTP, Telnet, DICT, LDAP, POP3, IMAP и SMTP. Она отлично подходит для имитации действий пользователя на страницах сайтов и других операций с URL адресами. Поддержка CURL добавлена в множество различных языков программирования и платформ.
Для начала скачаем саму утилиту, для этого переходим на официальный сайт утилиты, в раздел Download. После скачивания архива для своей платформы (у меня это Windows 64 bit), распаковываем архив. Чтобы иметь возможность работать с HTTPS и FTPS, устанавливаем сертификат безопасности url-ca-bundle.crt, который находится в папке curl/bin.
Запускаем командную строку, переходим в директорию curl/bin и пытаемся скачать главную страницу Google:
> cd C:/curl/bin > curl google.com <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="http://www.google.com/">here</A>. </BODY></HTML>
Опция -X позволяет задать тип HTTP-запроса вместо используемого по умолчанию GET. Дополнительные запросы могут быть POST, PUT и DELETE или связанные с WebDAV — PROPFIND, COPY, MOVE и т.п.
Следовать за редиректами
Сервер Google сообщил нам, что страница google.com перемещена (301 Moved Permanently), и теперь надо запрашивать страницу www.google.com. С помощью опции -L укажем CURL следовать редиректам:
> curl -L google.com <!doctype html> <html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"> <head> <meta content="Поиск информации в интернете: веб страницы, картинки, видео и многое другое." name="description"> <meta content="noodp" name="robots"> <meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"> <meta content="origin" name="referrer"> <title>Google</title> ..........
Сохранить вывод в файл
Чтобы сохранить вывод в файл, надо использовать опции -o или -O:
-o(oнижнего регистра) — результат будет сохранён в файле, заданном в командной строке;-O(Oверхнего регистра) — имя файла будет взято из URL и будет использовано для сохранения полученных данных.
Сохраняем страницу Google в файл google.html:
> curl -L -o google.html google.com % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 219 100 219 0 0 2329 0 --:--:-- --:--:-- --:--:-- 2329 100 14206 0 14206 0 0 69980 0 --:--:-- --:--:-- --:--:-- 69980
Сохраняем документ gettext.html в файл gettext.html:
> curl -O http://www.gnu.org/software/gettext/manual/gettext.html % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1375k 100 1375k 0 0 800k 0 0:00:01 0:00:01 --:--:-- 800k
Загрузить файл, только если он изменён
Опция -z позволяет получить файлы, только если они были изменены после определённого времени. Это будет работать и для FTP и для HTTP. Например, файл archive.zip будет получен, если он изменялся после 20 августа 2018 года:
> curl -z 20-Aug-18 http://www.example.com/archive.zip
Команда ниже загрузит файл archive.zip, если он изменялся до 20 августа 2018 года:
> curl -z -20-Aug-18 http://www.example.com/archive.zip
Прохождение аутентификации HTTP
Опция -u позволяет указать данные пользователя (имя и пароль) для прохождения базовой аутентификаци (Basic HTTP Authentication):
> curl -u evgeniy:qwerty -O http://www.example.com/archive.zip
Получение и отправка cookie
Cookie используются сайтами для хранения некой информации на стороне пользователя. Сервер сохраняет cookie на стороне клиента (т.е. в браузере), отправляя заголовки:
Set-Cookie: PHPSESSID=svn7eb593i8d2gv471rs94og58; path=/ Set-Cookie: visitor=fa867bd917ad0d715830a6a88c816033; expires=Mon, 16-Sep-2019 08:20:53 GMT; Max-Age=31536000; path=/ Set-Cookie: lastvisit=1537086053; path=/
А браузер, в свою очередь, отправляет полученные cookie обратно на сервер при каждом запросе. Разумеется, тоже в заголовках:
Cookie: PHPSESSID=svn7eb593i8d2gv471rs94og58; visitor=fa867bd917ad0d715830a6a88c816033; lastvisit=1537086053
Передать cookie на сервер, как будто они были ранее получены от сервера:
> curl -b lastvisit=1537086053 http://www.example.com/
Чтобы сохранить полученные сookie в файл:
> curl -c cookie.txt http://www.example.com/
Затем можно отправить сохраненные в файле cookie обратно:
> curl -b cookie.txt http://www.example.com/catalog/
Файл cookie.txt имеет вид:
# Netscape HTTP Cookie File # https://curl.haxx.se/docs/http-cookies.html # This file was generated by libcurl! Edit at your own risk. www.example.com FALSE / FALSE 0 lastvisit 1537085301 www.example.com FALSE / FALSE 1568621304 visitor 60f7c17ba4b5d77975dfd020f06ac8ca www.example.com FALSE / FALSE 0 PHPSESSID p23cr2d14rlgj5kls58kd7l6a6
Получение и отправка заголовков
По умолчанию, заголовки ответа сервера не показываются. Но это можно исправить:
> curl -i google.com HTTP/1.1 301 Moved Permanently Location: http://www.google.com/ Content-Type: text/html; charset=UTF-8 Date: Sun, 16 Sep 2018 08:28:18 GMT Expires: Tue, 16 Oct 2018 08:28:18 GMT Cache-Control: public, max-age=2592000 Server: gws Content-Length: 219 X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="http://www.google.com/">here</A>. </BODY></HTML>
Если содержимое страницы не нужно, а интересны только заголовки (будет отправлен HEAD запрос):
> curl -I http://www.example.com/ HTTP/1.1 200 OK Date: Sun, 16 Sep 2018 08:20:52 GMT Server: Apache/2.4.34 (Win64) mod_fcgid/2.3.9 X-Powered-By: PHP/7.1.10 Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate Pragma: no-cache Set-Cookie: PHPSESSID=svn7eb593i8d2gv471rs94og58; path=/ Set-Cookie: visitor=fa867bd917ad0d715830a6a88c816033; expires=Mon, 16-Sep-2019 08:20:53 GMT; Max-Age=31536000; path=/ Set-Cookie: lastvisit=1537086053; path=/ Content-Length: 132217 Content-Type: text/html; charset=utf-8
Посмотреть, какие заголовки отправляет CURL при запросе, можно с помощью опции -v, которая выводит более подробную информацию:
> curl -v google.com
- Строка, начинающаяся с
>означает заголовок, отправленный серверу - Строка, начинающаяся с
<означает заголовок, полученный от сервера - Строка, начинающаяся с
*означает дополнительные данные от CURL
* Rebuilt URL to: http://google.com/ * Trying 173.194.32.206... * TCP_NODELAY set * Connected to google.com (173.194.32.206) port 80 (#0)
> GET / HTTP/1.1 > Host: google.com > User-Agent: curl/7.61.1 > Accept: */* > < HTTP/1.1 301 Moved Permanently < Location: http://www.google.com/ < Content-Type: text/html; charset=UTF-8 < Date: Mon, 17 Sep 2018 15:11:49 GMT < Expires: Wed, 17 Oct 2018 15:11:49 GMT < Cache-Control: public, max-age=2592000 < Server: gws < Content-Length: 219 < X-XSS-Protection: 1; mode=block < X-Frame-Options: SAMEORIGIN < <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <TITLE>301 Moved</TITLE></HEAD><BODY> <H1>301 Moved</H1> The document has moved <A HREF="http://www.google.com/">here</A>. </BODY></HTML>
* Connection #0 to host google.com left intact
Если этой информации недостаточно, можно использовать опции --trace или --trace-ascii.
А вот так можно отправить свой заголовок:
> curl -H "User-Agent: Mozilla/5.0" http://www.example.com/
Отправка данных методом POST
Команда ниже отправляет POST запрос на сервер аналогично тому, как пользователь, заполнив HTML форму, нажал бы кнопку «Отправить». Данные будут отправлены в формате application/x-www-form-urlencoded.
> curl -d "key1=value1&key2=value2" http://www.example.com
> curl --data "key1=value1&key2=value2" http://www.example.com
Параметр --data аналогичен --data-ascii, для отправки двоичных данных необходимо использовать параметр --data-binary. Для URL-кодирования полей формы нужно использовать --data-urlencode.
> curl --data-urlencode "name=Василий" --data-urlencode "surname=Пупкин" http://www.example.com
Если значение опции --data начинается с @, то после него должно быть имя файла с данными (или дефис — тогда будут использованы данные из стандартного ввода). Пример получения данных из файла для отправки POST-запроса:
> curl --data @data.txt http://www.example.com
Содержимое файла data.txt:
key1=value1&key2=value2
Массив $_POST, который будет содержать данные этого запроса:
Array
(
[key1] => value1
[key2] => value2
)
Пример URL-кодирования данных из файла перед отправкой POST-запроса:
> curl --data-urlencode name@username.txt http://www.example.com
Содержимое файла username.txt:
Иванов Иван Иванович
Массив $_POST, который будет содержать данные этого запроса:
Array
(
[name] = Иванов Иван Иванович
)
Загрузка файлов методом POST
Для HTTP запроса типа POST существует два варианта передачи полей из HTML форм, а именно, используя алгоритм application/x-www-form-urlencoded и multipart/form-data. Алгоритм первого типа создавался давным-давно, когда в языке HTML еще не предусматривали возможность передачи файлов через HTML формы.
Со временем возникла необходимость через формы отсылать еще и файлы. Тогда консорциум W3C взялся за доработку формата POST запроса, в результате чего появился документ RFC 1867. Форма, которая позволяет пользователю загрузить файл, используя алгоритм multipart/form-data, выглядит примерно так:
<form action="/upload.php" method="POST" enctype="multipart/form-data"> <input type="file" name="upload"> <input type="submit" name="submit" value="OK"> </form>
Чтобы отправить на сервер данные такой формы:
> curl -F upload=@image.jpg -F submit=OK http://www.example.com/upload.php
Скрипт upload.php, который принимает данные формы:
<?php print_r($_POST); print_r($_FILES); move_uploaded_file($_FILES['upload']['tmp_name'], 'image.jpg');
Ответ сервера:
Array
(
[submit] => OK
)
Array
(
[upload] => Array
(
[name] => image.jpg
[type] => image/jpeg
[tmp_name] => D:\work\temp\phpB02F.tmp
[error] => 0
[size] => 2897
)
)
Работа по протоколу FTP
Скачать файл с FTP-сервера:
> curl -u username:password -O ftp://example.com/file.zip
Если заданный FTP путь является директорией, то по умолчанию будет выведен список файлов в ней:
> curl -u username:password -O ftp://example.com/public_html/
Выгрузить файл на FTP-сервер
> curl -u username:password -T file.zip ftp://example.com/
Получить вывод из стандартного ввода и сохранить содержимое на сервере под именем data.txt:
> curl -u username:password -T - ftp://example.com/data.txt
Дополнительно:
- Утилита curl. Обмен данными с сервером
- Пользуемся curl для отладки HTTP
Поиск:
CLI • CURL • Cookie • FTP • GET • HTTP • Linux • POST • URL • Web-разработка • Windows • Форма
Каталог оборудования
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Производители
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Функциональные группы
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.