Requests is the only Non-GMO HTTP library for Python, safe for human consumption.
- Keep-Alive & Connection Pooling
- International Domains and URLs
- Sessions with Cookie Persistence
- Browser-style SSL Verification
- Automatic Content Decoding
- Basic/Digest Authentication
- Elegant Key/Value Cookies
- Automatic Decompression
- Unicode Response Bodies
- HTTP(S) Proxy Support
- Multipart File Uploads
- Streaming Downloads
- Connection Timeouts
- Chunked Requests
- .netrc Support
Let’s start with loading Drakma and Dexador libraries. Additionally we load a few useful libraries for our demo code.
Simple GET request, note that Dexador uses multiple return values to return request status, headers, etc.
(dexador:get "https://api.github.com/events")This is how you make an HTTP POST request:
(dexador:post "https://httpbin.org/post" :content '(("key" . "value")))
Other HTTP methods
(dexador:put "https://httpbin.org/put" :content '(("key" . "value")))
(dexador:delete "https://httpbin.org/delete")
(multiple-value-bind (body status headers uri connection)
(dexador:head "https://httpbin.org/get")
(alexandria:hash-table-alist headers))(multiple-value-bind (body status headers uri connection)
(dexador:request "https://httpbin.org/get" :method :options)
(alexandria:hash-table-alist headers))If you wanted to pass key1=value1 and key2=value2 to httpbin.org/get, you would use the following code:
(let ((payload '(("key1" . "value1") ("key2" . "value2"))))
(multiple-value-bind (body status headers uri connection)
(dexador:get (quri:make-uri :defaults "https://httpbin.org/get" :query payload))
uri))You can also pass a list of items as a value:
(let ((payload '(("key1" . "value1") ("key2" . "value2") ("key2" . "value3"))))
(multiple-value-bind (body status headers uri connection)
(dexador:get (quri:make-uri :defaults "https://httpbin.org/get" :query payload))
uri))We can read the content of the server’s response. Consider the GitHub timeline again:
(dexador:get "https://api.github.com/events")Dexador will automatically decode content from the server. Most unicode charsets are seamlessly decoded.
It is possible to get the guessed charset:
(multiple-value-bind (body status headers uri connection)
(dexador:get "https://api.github.com/events")
(dexador.encoding:detect-charset (gethash "content-type" headers) body))To manually fix encoding issues you can resort to geting raw binary data for further processing.
(dexador:get "https://api.github.com/events" :force-binary t)You can also access the response body as bytes, for non-text requests:
(dexador:get "http://httpbin.org/image/jpeg")The gzip and deflate transfer-encodings are automatically decoded for you.
For example, to create an image from binary data returned by a request, you can use the following code:
(ql:quickload 'opticl)
(opticl:read-image-stream
(flexi-streams:make-in-memory-input-stream
(dexador:get "http://httpbin.org/image/jpeg"))
"jpeg")
Dexador doesn’t provide built-in support for decoding JSON. Please use other libraries to handle parsing i.e. https://github.com/madnificent/jsown
(jsown:parse
(dexador:get "https://api.github.com/events"))Dexador doesn’t provide access to raw socket streams. But you can get binary stream for decompressed body data.
(dexador:get "https://api.github.com/events" :force-binary t :want-stream t)If you’d like to add HTTP headers to a request, simply pass in an alist to the headers parameter.
For example, let’s specify user-agent:
(dexador:get "http://httpbin.org/headers" :headers '(("user-agent" . "my-app/0.0.1") (:foo . :bar)))
Note how Dexador automatically converts header names to capitalised kebab case.
Typically, you want to send some form-encoded data — much like an HTML form.
To do this, simply pass an alist to the content argument.
Your alist of data will automatically be form-encoded when the request is made:
(dexador:post "http://httpbin.org/post" :content '(("key1" . "value1") ("key2" . "value2")))
The content argument can also have multiple values for each key.
This is particularly useful when the form has multiple elements that use the same key:
(dexador:post "http://httpbin.org/post" :content '(("key1" . "value1") ("key1" . "value2") ("key2" . "value3")))
There are times that you may want to send data that is not form-encoded. If you pass in a string instead of an alist, that data will be posted directly.
(dexador:post "http://httpbin.org/post"
:content (jsown:to-json '(:OBJ ("key" . "value")))
:headers '((:content-type . "application/json")))
Dexador directly supports sending Multipart-encoded files.
(dexador:post "http://httpbin.org/post"
:content '(("hello.txt" . #p"hello.txt")))
Status code is returned as one of the multiple values from Dexador request call:
(multiple-value-bind (body status headers url connection) (dexador:get "http://httpbin.org/get")
status)Bad requests will signal a http-request-failed condition
(handler-case (dex:get "https://httpbin.org/status/404")
(dex:http-request-failed (e)
(format nil "The server returned ~D" (dex:response-status e))))You can handle more specialized conditions
(handler-case (dex:get "https://httpbin.org/status/400")
(dex:http-request-bad-request (e)
(format nil "Bad reqest was sent to server: ~D" (dex:response-status e)))
(dex:http-request-failed (e)
(format nil "The server returned ~D" (dex:response-status e))))(handler-case (dex:get "https://httpbin.org/status/404")
(dex:http-request-not-found (e)
(format nil "Page not found: ~D" (dex:response-status e)))
(dex:http-request-failed (e)
(format nil "The server returned ~D" (dex:response-status e))))You can ignore specific conditions
(handler-bind ((dexador:http-request-not-found #'dexador:ignore-and-continue))
(dexador:get "https://httpbin.org/status/404"))Or retry the request.
(let ((retry-request (dex:retry-request 5 :interval 3)))
(handler-bind ((dex:http-request-failed retry-request))
(dex:get "https://httpbin.org/status/404"))))
This will result in condition afer about 15 seconds.
An HTTP request to "https://httpbin.org/status/404" returned 404 not found.
[Condition of type DEXADOR.ERROR:HTTP-REQUEST-NOT-FOUND]
Restarts:
0: [RETRY-REQUEST] Retry the same request.
1: [IGNORE-AND-CONTINUE] Ignore the error and continue.
2: [RETRY] Retry SLIME evaluation request.
3: [*ABORT] Return to SLIME's top level.
4: [ABORT] abort thread (#<THREAD "worker" RUNNING {10017C1793}>)
Backtrace:
0: (DEXADOR.ERROR:HTTP-REQUEST-FAILED 404 :BODY "" :HEADERS #<HASH-TABLE :TEST EQUAL :COUNT 7 {1001AF01D3}> :URI #<QURI.URI.HTTP:URI-HTTPS https://httpbin.org/status/404> :METHOD :GET)
1: (DEXADOR.BACKEND.USOCKET:REQUEST #<unavailable argument> :METHOD :GET)
2: ((LAMBDA ()))
We can view the server’s response headers:
(multiple-value-bind (body status headers uri connection)
(dexador:head "https://httpbin.org/get")
(alexandria:hash-table-alist headers))Since header names are case insensitive keys in the headers hash table are converted to lower case.
Dexador adopts https://github.com/fukamachi/cl-cookie for its cookie management. All functions takes a cookie-jar instance at :cookie-jar.
(defvar *cookie-jar* (cl-cookie:make-cookie-jar))
;; setting cookies
(dex:head "https://mixi.jp" :cookie-jar *cookie-jar*);; getting cookies
(dex:head "https://mixi.jp" :cookie-jar *cookie-jar*)
*cookie-jar*
Dexador automatically follows redirects on GET and HEAD requests.
You can limit the count of redirection by specifying :max-redirects with an integer. The default value is 5.
(multiple-value-bind (body status headers uri connection)
(dex:get "http://httpbin.org/redirect/2")
(list status uri body))
(multiple-value-bind (body status headers uri connection)
(dex:get "http://httpbin.org/redirect/3" :max-redirects 2)
(list status uri body))You can use forth returned parameter to get the URL of the final redirect location.
Dexador doesn’t track the history of responses.
You can tell Dexador to stop waiting for a connection after connect-timout and waiting to read a response after read-timeout number of seconds.
(dex:get "http://httpbin.org/delay/5")
(handler-case (dex:get "http://httpbin.org/delay/5" :read-timeout 3)
(error (c)
c))