Skip to content

RutledgePaulV/clj-okhttp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

51 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

travisci clojars codecov cljdoc

Intro

clj-okhttp is a http and websocket client library built on OkHttp, Muuntaja, and Jsonista that supports both synchronous and asynchronous access patterns.

Philosophies

it's just data

clj-okhttp follows in the tradition of other Clojure http clients by accepting ring-style request maps and returning ring-style response maps.

data instead of exceptions

clj-okhttp does not throw exceptions on non-2xx status codes. it just returns data and you're responsible for checking status codes and expectations prior to using the response data. if you really want exceptions you can write your own middleware to do that.

no viral asynchronous constructs

clj-okhttp uses respond/raise callbacks just like asynchronous ring instead of forcing you into an opinionated construct like futures / manifold / core.async.

composable transformations

clj-okhttp uses ring-style middleware internally and allows for custom middleware functions to be used per-client instance and per-request.

do the right thing by default

clj-okhttp tries to just do the right thing for you instead of requiring configuration options for common cases.

  • connection pooling is automatic
  • you can supply pem-encoded keys/certificates instead of messing with java crypto classes
  • it decodes response data for you based on the content type of the response (by default)
  • it has default read/write/connection timeouts instead of infinite timeouts like some other http clients

no globals or dynamic vars

clj-okhttp does not provide a default / global http client instance. You need to manage your client instances and pass them into clj-okhttp functions as appropriate.

be efficient

clj-okhttp uses jsonista because it is faster than cheshire. clj-okhttp does not buffer entire responses into a byte array like some other http clients and it supports lazy streaming serialization of request data.

Quickstart

The main HTTP client functionality is provided by the clj-okhttp.core namesapce.

First, require it in the REPL:

(require '[clj-okhttp.core :as http])

Or in your application:

(ns my-app.core
  (:require [clj-okhttp.core :as http]))

This client supports simple head, options, get, put, post, delete, and patch requests.

You must create your own instance of an OkHttpClient to make requests. This can be done via the create-client function:

(def client (http/create-client opts))

You are responsible for managing these clients and reusing them. By reusing them you get things like connection pooling for free for added performance!

Usage

(require '[clj-okhttp.core :as http])

(def opts {:read-timeout 1000})

; clients contain a connection pool
(def client (http/create-client opts))

; synchronous GET that returns a data map and 
; a parsed body of data if the body was a content 
; type understood by the default muuntaja instance
(http/get client "https://httbin.org/get")

; supply a request map to customize other aspects of the request
(http/get client "https://httbin.org/get" {:query-params {:a 1}})
(http/get client "https://httpbin.org/anything" {:query-params {:a "foo" "b" ["1" "2"] :c 3}})

;; custom headers (can mix keywords and strings)
(http/get client "https://httpbin.org/anything" {:headers {"content-type" "text/plain" :accept :json}})

; make asynchronous calls by using callbacks
(def request {})
(def respond (fn [response] (println "response" response)))
(def raise   (fn [exception] (.printStackTrace exception)))
(http/get client "https://httbin.org/get" request respond raise)

Creating a Client

For a full list of client options see clj-okhttp.core/create-client. Explanation of the parameters can be found here.

With that being said there are a few common options to look at:

Connection Pool

You may want to adjust the default connection pool to increase the number of connections or adjust the keep alive.

{:connection-pool {:max-idle-connections 5 :keep-alive-duration 5 :time-unit java.util.concurrent.TimeUnit/MINUTES}}

Redirects

By default the client will follow redirects and ssl redirects. This behavior can be adjusted if desired.

{:follow-redirects true 
 :follow-ssl-redirects true}

Timeouts

By default the client will have a connect/read/write timeout of 10s and no call timeout.

{:call-timeout 0 :connect-timeout 10000 :read-timeout 10000 :write-timeout 10000}

Cookies

A custom CookieJar may be supplied. Below is an example of an in-memory CookieJar

(def MemoryCookieJar
  (let [cache (atom [])]
    (letfn [(cookie-expired? [^Cookie cookie]
              (< (.expiresAt cookie) (System/currentTimeMillis)))
            (cookies-equal? [^Cookie cookie ^Cookie other]
              (and (= (.name cookie) (.name other))
                   (= (.domain cookie) (.domain other))
                   (= (.path cookie) (.path other))
                   (= (.secure cookie) (.secure other))
                   (= (.hostOnly cookie) (.hostOnly other))))]
      (reify CookieJar
        (^List loadForRequest [_ ^HttpUrl url]
          (->> (swap! cache #(remove cookie-expired? %))
               (filterv (fn [^Cookie cookie]
                          (.matches cookie url)))))
        (saveFromResponse [_ _ response-cookies]
          (swap! cache (fn [cookies]
                         (->> cookies
                              (remove (fn [^Cookie cookie]
                                        (some #(cookies-equal? cookie %) response-cookies)))
                              (concat response-cookies)))))))))
{:cookie-jar MemoryCookieJar}

Decompression

By default OkHttp will perform transparent GZIP. It will add the appropriate accept-encoding header (if not already supplied) and automatically handle decompression for you.

Caching

OkHttp implements an optional, off by default, Cache. OkHttp aims for RFC correct and pragmatic caching behaviour, following common real-world browser like Firefox/Chrome and server behaviour when ambiguous. This can be configured via:

{:directory  "/tmp"
 :max-size    123
 :file-system okhttp3.internal.io.FileSystem/SYSTEM}

Proxy

To configure a proxy, see the :proxy/:proxy-selector/:proxy-authenticator options. An example of using a socks5 proxy below:

(let [proxy-host "proxy-host"
      proxy-port 8888
      username   "user"
      password   "pass"]
  (create-client
    {:proxy               (Proxy. Proxy$Type/SOCKS (InetSocketAddress. proxy-host proxy-port))
     :proxy-authenticator (reify Authenticator
                            (authenticate [this route response]
                              (when (and (= (str/lower-case (.getRequestingHost this))
                                            (str/lower-case proxy-host))
                                         (= (.getRequestingPort this)
                                            proxy-port))
                                (PasswordAuthentication. username (.toCharArray password)))))}))

DNS Resolution

Users may add their own DNS resolver function to override the default DNS Resolver. This is useful in situations where you are unable to change the name to IP Address mapping. It is analogous to the --resolve flag present in curl. Below is an example of how to configure your own DNS resolver.

(create-client
  {:dns (reify Dns
          (lookup [this hostname]
            ;; Or do custom logic here
            (.toList (InetAddress/getAllByName hostname))))})

Output Coercion

;; By default, decoding performed on the body is determined by the content type returned by the server
(http/get client "http://example.com/foo.txt")

;; decode as json
(http/get client "http://example.com/foo.json" {:as :json})

;; decode as Transit encoded JSON or MessagePack
(http/get client "http://example.com/foo" {:as :transit+json})
(http/get client "http://example.com/foo" {:as :transit+msgpack})

;; Leave the body as a stream
(http/get client "http://example.com/bigrequest.html" {:as :stream})
;; Note that the connection to the server will NOT be closed until the stream has been read!

Link Headers

clj-okhttp parses any link headers returned in the response, and adds them to the :links key on the response map. This is particularly useful for paging RESTful APIs:

(:links (http/get client "https://api.github.com/gists"))
=> {:next {:href "https://api.github.com/gists?page=2"}
    :last {:href "https://api.github.com/gists?page=22884"}}

Raw Requests

A more general request function is also available which is useful as a primitive for building higher-level interfaces:

(defn api-action [method path & [opts]]
  (let [final-req (merge {:request-method method :url (str "https://httpbin.org" path)} opts)]
  (http/request client final-req)))

Middleware

clj-okhttp uses ring-style middleware internally and allows for custom middleware functions to be used per-client instance and per-request.

(let [my-middleware [(fn [handler]
                       (fn wrap-hello
                         ([request]
                          (println "hello")
                          (handler request))
                         ([request respond raise]
                          (println "hello")
                          (handler request respond raise))))]]
  ;; Per Client
  (get (create-client {:middleware my-middleware}) "https://google.com")
  ;; Per Request
  (get client "https://google.com" {:middleware my-middleware}))

Interceptors

OkHttp provides two different types of interceptors: Application and Network. You can read abouch which one to choose here. For example, you can roll your own logging interceptor

{:interceptors [(fn [^Interceptor$Chain chain]
                     (let [request (.request chain)
                           t1  (System/nanoTime)]
                       (printf "Sending request %s on %s%n%s"
                               (.url request)
                               (.connection chain)
                               (.headers request))
                       (let [response (.proceed chain request)
                             t2       (System/nanoTime)]
                         (printf "Received response for %s in %.1fms%n%s"
                                 (.url (.request response))
                                 (/ (- t2 t1) 1e6)
                                 (.headers response))
                         response)))]}

You can read more here when deciding between application and network interceptors.

Generally speaking, it's preferred to use middleware for application interceptors.

Headers

By default, clj-okhttp will convert all headers to lower-case strings for both the request and the response.

  • Request headers keys can be strings/keywords and request header values can be Dates/Instanst/strings/keywords
  • Response headers will always be a map of a string to a string.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Packages

No packages published