Http.fs

A gloriously functional HTTP client library for F#! NuGet name: Http.fs.

.Net build (AppVeyor): Mono build (Travis CI): NuGet package:

How do I use it?

In it's simplest form, this will get you a web page:

open Hopac
open HttpFs.Client

let body =
  Request.createUrl Get "http://somesite.com"
  |> Request.responseAsString
  |> run

printfn "Here's the body: %s" body

To get into the details a bit more, there are two or three steps to getting what you want from a web page/HTTP response.

1 - A Request (an immutable record type) is built up in a Fluent Builder style as follows:

open System.IO
open System.Text
open Hopac
open HttpFs.Client

let pathOf relativePath =
  let here = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location)
  Path.Combine(here, relativePath)

let firstCt, secondCt, thirdCt, fourthCt =
    ContentType.parse "text/plain" |> Option.get,
    ContentType.parse "text/plain" |> Option.get,
    ContentType.create("application", "octet-stream"),
    ContentType.create("image", "gif")

let httpClientWithNoRedirects () =
    let handler = new HttpClientHandler(UseCookies = false)
    handler.AllowAutoRedirect <- false
    let client = new HttpClient(handler)
    client.DefaultRequestHeaders.Clear()
    client

// we can trivially extend request to add convenience functions for common operations
module Request =
    let autoFollowRedirectsDisabled h = 
        { h with httpClient = httpClientWithNoRedirects () }

let request =
    Request.createUrl Post "https://example.com"
    |> Request.queryStringItem "search" "jeebus"
    |> Request.basicAuthentication "myUsername" "myPassword" // UTF8-encoded
    |> Request.setHeader (UserAgent "Chrome or summat")
    |> Request.setHeader (Custom ("X-My-Header", "hi mum"))
    |> Request.autoFollowRedirectsDisabled
    |> Request.cookie (Cookie.create("session", "123", path="/"))
    |> Request.bodyString "This body will make heads turn"
    |> Request.bodyStringEncoded "Check out my sexy foreign body" (Encoding.UTF8)
    |> Request.body (BodyRaw [| 1uy; 2uy; 3uy |])
    |> Request.body (BodyString "this is a greeting from Santa")

    // if you submit a BodyForm, then Http.fs will also set the correct Content-Type, so you don't have to
    |> Request.body (BodyForm 
        [
            // if you only have this in your form, it will be submitted as application/x-www-form-urlencoded
            NameValue ("submit", "Hit Me!")

            // a single file form control, selecting two files from browser
            FormFile ("file", ("file1.txt", ContentType.create("text", "plain"), Plain "Hello World"))
            FormFile ("file", ("file2.txt", ContentType.create("text", "plain"), Binary [|1uy; 2uy; 3uy|]))

            // you can also use MultipartMixed for servers supporting it (this is not the browser-default)
            MultipartMixed ("files",
              [ "file1.txt", firstCt, Plain "Hello World" // => plain
                "file2.gif", secondCt, Plain "Loopy" // => plain
                "file3.gif", thirdCt, Plain "Thus" // => base64
                "cute-cat.gif", fourthCt, Binary (File.ReadAllBytes (pathOf "cat-stare.gif")) // => binary
          ])
    ])
    |> Request.responseCharacterEncoding Encoding.UTF8    
    |> Request.proxy {
          Address = "proxy.com";
          Port = 8080;
          Credentials = Credentials.Custom { username = "Tim"; password = "Password1" } }

(with everything after createRequest being optional)

2 - The Http response (or just the response code/body) is retrieved using one of the following:

job {
  use! response = getResponse request // disposed at the end of async, don't
                                      // fetch outside async body
  // the above doesn't download the response, so you'll have to do that:
  let! bodyStr = Response.readBodyAsString response
  // OR:
  //let! bodyBs = Response.readBodyAsBytes

  // remember HttpFs doesn't buffer the stream (how would we know if we're
  // downloading 3GiB?), so once you use one of the above methods, you can't do it
  // again, but have to buffer/stash it yourself somewhere.
  return bodyStr
}

3 - If you get the full response (another record), you can get things from it like so:

response.StatusCode
response.Body // but prefer the above helper functions
response.ContentLength
response.Cookies.["cookie1"]
response.Headers.[ContentEncoding]
response.Headers.[NonStandard("X-New-Fangled-Header")]

So you can do the old download-multiple-sites-in-parallel thing:

[ "http://news.bbc.co.uk"
  "http://www.wikipedia.com"
  "http://www.stackoverflow.com"]
|> List.map (createRequestSimple Get)
| > List.map (Request.responseAsString) // this takes care to dispose (req, body)
|> Job.conCollect
|> Job.map (printfn "%s")
|> start

If you need direct access to the response stream for some reason (for example to download a large file), you need to write yourself a function and pass it to getResponseStream like so:

open Hopac
open System.IO
open HttpFs.Client

job {
  use! resp = Request.createUrl Get "http://fsharp.org/img/logo.png" |> getResponse
  use fileStream = new FileStream("c:\\bigImage.png", FileMode.Create)
  do! resp.Body.CopyToAsync fileStream
}

Note because some of the request and response headers have the same names, to prevent name clashes, the response versions have 'Response' stuck on the end, e.g.

response.Headers.[ContentTypeResponse]

Building

1. Download the source code
2. Execute the build.sh (linux & macos) or build.cmd (windows)

Examples

Check out HttpClient.SampleApplication, which contains a program demonstrating the various functions of the library being used and (to a limited extent) unit tested.

SamplePostApplication shows how you can create a post with a body containing forms.

Version History

Http.fs attempts to follow Semantic Versioning, which defines what the different parts of the version number mean and how they relate to backwards compatability of the API. In a nutshell, as long as the major version doesn't change, everything should still work.

• 0.X.X - Various. Thanks for code and suggestions from Sergeeeek, rodrigodival, ovatsus and more
• 1.0.0 - First stable API release. Changed how 'duplicated' DUs were named between request/response.
• 1.1.0 - Added withProxy, thanks to vasily-kirichenko
• 1.1.1 - Handles response encoding secified as 'utf8' (.net encoder only likes 'utf-8')
• 1.1.2 - Added utf16 to response encoding map
• 1.1.3 - Added XML comments to public functions, made a couple of things private which should always have been (technically a breaking change, but I doubt anybody was using them)
• 1.2.0 - Added withKeepAlive
• 1.3.0 - Added getResponseBytes, thanks to Sergeeeek
• 1.3.1 - Added project logo, thanks to sergey-tihon
• 1.4.0 - Added getResponseStream, with thanks to xkrt
• 1.5.0 - Added support for Patch method with help from haf, and xkrt fixed an issue with an empty response.CharacterSet
• 1.5.1 - Corrected the assembly version
• 2.0.0 - Production hardened, major release, major improvements
• 3.0.3 - Async -> Job, withXX -> Request.withXX

FAQ

• How does it work?

Http.fs currently uses HttpClient under the hood.

• Does it support proxies?

Yes. By default it uses the proxy settings defined in IE, and as of 1.1.0 you can specify basic proxy settings separately using withProxy.

• Can I set KeepAlive?

Yes, as of version 1.2.0. This actually sets the Connection header (to 'Keep-Alive' or 'Close'). Note that if this is set to true (which is the default), the Connection header will only be set on the first request, not subsequent ones.

Why?

Simplicity for F# programmers. An abstract, immutable API that you can build better abstractions beneath (if needed).

What other kick-ass open source libraries are involved?

The only thing that's used in the HttpClient module itself is AsyncStreamReader.fs, a source file taken directly from the Fsharpx library.

However, for testing a couple of other things are used:

• Suave to create a web server for integration testing
• FsUnit for unit testing
• NancyFX to create a web server for integration testing

That's about it. Happy requesting!

Henrik Feldt – @haf

Originally built by Grant Crofton.

Post Scriptum

• https://www.youtube.com/watch?v=_1rh_s1WmRA