Module HighwaySource

Highway is a simple, framework-agnostic HTTP router. Its goal is to provide a highly abstract API based on the concepts of response and request so that it can, broadly speaking, be (probably) adapted to any framework in the OCaml ecosystem.

Big Picture

The library allows you to define typed routes, enabling you to generate links using an API similar to the Format module, and to attach controllers to them, similar to the Scanf module.

Highway is primarily used to describe route and associate them with services.

# open Highway ;;

Describing routes

To begin with, Highway provides a DSL for describing routes: the combination of an HTTP method, a path and a set of query parameters. For now, we won’t concern ourselves with query parameters.

Let's define a set of routes:

# module Routes = struct
    let home = get [] ignore_params
    let hello = get [s "hello"] ignore_params
    let hello_to = get [s "hello"; string] ignore_params
  end ;;
module Routes :
  sig
    val home : (local, [> `GET ], something, unit, Void.t) route
    val hello : (local, [> `GET ], something, unit, Void.t) route
    val hello_to :
      (local, [> `GET ], something, unit, string -> Void.t) route
  end

As you can see, road types contain a lot of information:

• scope (can be local or global), that describe the scope of a route. If it is local, it refers to a route that is "internal" to the application, and later, it can be associated with a service. If the route is global, it is associated with a domain and is external to the application (and therefore cannot be associated with a service).

• met, describes the HTTP method of the route. This tracking is useful because it allows you, for example, to prevent certain link-generation functions from being used on specific routes. (For example, html_href, which generates a link usable for <a> tags, works only for routes associated with the GET method).

• constraints Specifies whether the route disallows query parameters (the router will continuously reject requests containing query parameters). It can have two values: something, which allows query parameters, and nothing, which disallows query parameters.

• param_type the type of all (valid) query parameters. If the constraint is nothing, it will always be unit. Here, in our examples, we ignore the query parameters (we don't prohibit them), but their value is also unit.

• path_params describes a function that returns void parameters extracted from the route path.

Here is an other example with a lot of path parameters (and prohibiting query parameters):

# post
    [int; string; float; bool; s "foo"; s "bar"; int]
    discard_params ;;
- : (local, [> `POST ], nothing, unit,
     int -> string -> float -> bool -> int -> Void.t)
    route
= <abstr>

You can define your own hole using Hole and Pattern.

Computing links from routes

The separation of a route's definition from its association with a service is heavily inspired by Ocsigen/Eliom; it allows routes to be used within services to describe connections between different services (and to manage form actions).

Link generation is based on three functions:

• html_href which generates a link to be used in a <a> tag for the href attribute (among others), and this function only applies to GET routes.

• html_action which generates a link to be used in a <form> tag for the method attribute (among others), and this function only applies to GET and POST routes.

• target which generates the arbitrary route link (which can be used, for example, to implement an ambitious fetch function).

These three functions fill in the gaps in the routes described by the path and the query params, and allow you to associate additional query parameters (using the extra_params parameter, it is useful since a service can extract other query params after the routing, for example) and an optional anchor (using the anchor parameter).

Functions have "quoted images", which reject extra parameters (traditional functions require routes with something as a constraint).

Here is some example of generating links for routes:

# html_href Routes.home [] ();;
- : string = "/"

    # html_href Routes.hello [] ();;
    - : string = "/hello"

    # html_href Routes.hello_to ["Xavier"] ();;
    - : string = "/hello/Xavier"

And with extra parameters and anchor:

    # html_href
         ~extra_params:["foo", "true"; "bar", "hello"]
         ~anchor:"top-content"
         Routes.hello_to ["Xavier"] ();;
    - : string = "/hello/Xavier?foo=true&bar=hello#top-content"

Or here's a broader example that illustrates the heterogeneous nature of args:

let a_route =
  post
    [ int; string; float; bool; s "foo"; s "bar"; int; s "a-long-url" ]
    discard_params
;;

If we use html_action' since it is a POST route that disallows query params. First, we can see the error if we did not give a proper list of path fragment. The error indicates that the list ends prematurely and that some fragments are missing during compilation.

# html_action'
    a_route
    [42; "foo"; 3.14; false; 42]
    ~anchor:"a-specific-part-of-the-document"
    () ;;
- : string =
"/42/foo/3.14/false/foo/bar/42/a-long-url#a-specific-part-of-the-document"

Query Params

Even though query parameters can be used within the body of a service, it is sometimes convenient to extract some of them directly during the routing phase so they can be used immediately in the body of a service. That is why a route description allows you to define a strategy for extracting query parameters.

Since Highway makes no assumptions about the framework being used, the library assumes that query parameters are represented as an associative list of string * string. So, for example, the following query string: ?foo=bar&x=10&foo=message produces the following list:

[ "foo", "bar"; "x", "10"; "foo", "message" ]

That is the choice Dream has made. However, if you are using a framework that makes a different choice — such as Uri's, which uses an associative list of string * string list — you can use the Param.from_nested_list function to convert to that representation.

Unlike path fragments, where order matters, we'd like to be able to process query parameters independently. To do this, the param type defines a validation rule on a set of query parameters to produce a specific value.

Behind the scenes, validation uses the Pidgin library to validate query parameters as if they were records. By using the make_params function, you can define a query parameter validator. As for pattern, you need to give a validation function and a projection function (for link generation).

An example

Let's imagine we want to describe a filtering strategy by author and category:

module Filter = struct
  type t =
    { author : string
    ; category : string
    ; limit : int option
    }

  let param =
       make_params
         ~from_query:(fun fields ->
           let open Pidgin.Check in
           let+ author = req fields "author" string
           and+ category = req fields "category" string
           and+ limit = opt fields "limit" (int & Int.is_positive) in
           { author; category; limit })
         ~to_query:(fun { author; category; limit } ->
           let all = [ "author", author; "category", category ] in
           match limit with
           | None -> all
           | Some x -> ("limit", string_of_int x) :: all)
           (* This is just for testing reason, you do not need, obviously,
              to disable ocamlformat here... *)
           [@@ocamlformat "disable"]
end

We define from_query, which uses a Pidgin record validator (you can use the full Pidgin API for making fine-grained validators), and a to_query function that returns an associative array. (We do not return a Pidgin expression because that would be too expressive for query parameters.)

Now, we can define a route that will use the filter:

let another_route = get [ s "books"; s "filter" ] Filter.param

And when we try to generate the corresponding link:

# html_href another_route []
     {author = "John Doe"; category = "Novel"; limit = None} ;;
- : string = "/books/filter?author=John Doe&category=Novel"

# html_href another_route []
     {author = "John Doe"; category = "Novel"; limit = Some 42} ;;
- : string = "/books/filter?limit=42&author=John Doe&category=Novel"

And during the routing phase, query parameters will be validated and given to the controller.

Simple API

Although the manual definition of a validator is very flexible and allows you to describe a wide variety of scenarios, sometimes you might want a more direct and straightforward approach. For that, there is a slightly simpler (and composable) API available.

In fact, there are functions that allow you to process query parameters one by one, for example:

• string_param which takes a string as an argument—the key—and checks whether a string associated with the given key exists.

• string_opt_param which takes a string as an argument—the key—and checks whether a string associated with the given key exists (and wrap it into an option).

• along with int_param, float_param, char_param, bool_param.

• and int_opt_param, float_opt_param, char_opt_param, bool_opt_param.

By default, these validators support only one query parameter, for example:

  # html_href
     (get [ s "books"; s "filter" ] (string_param "author"))
     [] "Xavier" ;;
  - : string = "/books/filter?author=Xavier"

However, using the & operator, they can be combined. In fact, & composes two arbitrary validators. For example, the original example could be reproduced only in this way:

  # html_href
     (get [ s "books"; s "filter" ]
     (string_param "author"
        & string_param "category"
        & int_opt_param "limit"))
     [] ("Xavier", ("novel", Some 43)) ;;
  - : string = "/books/filter?author=Xavier&category=novel&limit=43"

Similarly, there is / that uses Either to construct a sum.

External routes

In our view, route description tools provide a robust and well-defined approach to describing access points to resources. It would therefore be a shame to limit ourselves to internal links.

Fortunately, the global function allows you to convert a local route into a global route. For example, here is a very small, minimalist (and partial) binding for the JsonPlaceholder API:

module Json_api = struct
  open Highway

  let base_url = "https://jsonplaceholder.typicode.com"

  let all_posts =
    global base_url (get [ s "posts" ] ignore_params)

  let one_post =
    global base_url (get [ s "posts"; int ] ignore_params)

  let one_post_with_comments =
    global base_url (get [ s "posts"; int; s "comments" ] ignore_params)

  let comments =
    global base_url (get [ s "comments" ] (int_opt_param "postId"))

end
(* This is just for testing reason, you do not need, obviously,
   to disable ocamlformat here... *)
[@@ocamlformat "disable"]

You cannot associate global routes with services (which makes sense), but you can still use them to generate links:

# [ html_href Json_api.all_posts [] ()
  ; html_href Json_api.one_post [1] ()
  ; html_href Json_api.one_post_with_comments [1] ()
  ; html_href Json_api.comments [] None
  ; html_href Json_api.comments [] (Some 1)
  ] ;;
- : string list =
["https://jsonplaceholder.typicode.com/posts";
 "https://jsonplaceholder.typicode.com/posts/1";
 "https://jsonplaceholder.typicode.com/posts/1/comments";
 "https://jsonplaceholder.typicode.com/comments";
 "https://jsonplaceholder.typicode.com/comments?postId=1"]

There is therefore no particular reason not to describe your external endpoints using this API, which provides typed functions.

Describing services

Now that we’ve looked at routes, let’s see how to associate behaviour with them. In other words, how to associate a controller with a route. In Highway terminology (also inspired by Ocsigen), this involves describing a service.

Creating a service is done using the service function:

# service ;;
- : ?middleware:('request, 'response) middleware ->
    ?precondition:('request -> bool) ->
    ?postcondition:('args args -> 'param_ty -> 'request -> bool) ->
    context:('ctx, 'request, 'response) context ->
    route:(local, meth, 'cstr, 'param_ty, 'args) route ->
    ('args args -> 'param_ty -> 'ctx -> ('request, 'response) handler) ->
    ('request, 'response) service
= <fun>

For now, we won’t worry about the specific settings; we’ll get straight to the point by using a pre-configured version of service:

# let simple_service ~route handler =
    service ~context:no_context ~route handler ;;
val simple_service :
  route:(local, meth, 'a, 'b, 'c) route ->
  ('c args -> 'b -> unit -> ('d, 'e) handler) -> ('d, 'e) service = <fun>

So the main idea of Services is to associate a route with an handler. As we can see in the signature of simple_service, a handler is a function that takes the following form:

fun [ args_from_route_path ] query_parameter context request -> a_response

Here, since our simple_service is fixed to no_context the context parameter will be unit.

As we have said on numerous occasions, Highway is HTTP server-agnostic, so for the purposes of this tutorial, let’s assume that a response is a string and define a dummy request type, and let’s create our first service:

type request =
  { user : string option
  ; query : (string * string) list
  }

let req ?user ?(query = []) () = { user; query }

let a_first_service =
  simple_service ~route:Routes.home (fun [] () () _req ->
    "Welcome to my website")
;;

Let's write an other service with a more complicated route:

let an_other_service =
  simple_service ~route:Routes.hello_to (fun [ name ] () () _req ->
    "Hello world, Hello" ^ name)
;;

And let’s write one final service that utilises query parameters:

let yet_another_service =
  simple_service
    ~route:another_route
    (fun [] { author; category; limit } () _req ->
       [ "Author: " ^ author
       ; "Category: " ^ category
       ; ("Limit: "
          ^ Option.(value ~default:"none" (map string_of_int limit)))
       ]
       |> String.concat "\n")
;;

As we can see, the callback function – the handler – is heavily dependent on the route. This allows us to be guided by the type system.

Middleware

A middleware is a composable function that wraps a web handler to process a request before it reaches the handler and/or a response after it returns. They can provide capabilities, guards etc.

They are applied once routing has been completed (and therefore do not allow the user to proceed to the next page). Let’s imagine, for example, that we want to have services that are only accessible if the user is logged in:

(* An helper for errors *)
let error_response ?message code _req =
  "Error "
  ^ string_of_int code
  ^
  match message with
  | None -> ""
  | Some message -> "\n" ^ message
;;

let user_required next_handler ({ user; _ } as req) =
  match user with
  | None -> error_response ~message:"You need to be logged" 401 req
  | Some _ -> next_handler req
;;

Now, our function is a middleware; if the user is present in the request, the programme continues; otherwise, an error response is returned. We can now use it in a service definition:

let yet_another_service =
  service
    ~middleware:user_required
    ~context:no_context
    ~route:another_route
    (fun [] { author; category; limit } () _req ->
       [ "Author: " ^ author
       ; "Category: " ^ category
       ; ("Limit: "
          ^ Option.(value ~default:"none" (map string_of_int limit)))
       ]
       |> String.concat "\n")
;;

Now, if the service is running but the user is not logged in, the application will return an error response. It is possible to combine several middleware components sequentially using middleware_list.

Context

A context is a “type of middleware” that is applied and allows you to retrieve a value to pass to the handler function. For example, if instead of just verifying, via middleware, that a user is registered (using our test request), we also want to “pass it to our controller,” we could describe a context provider this way:

let provide_user handler ({ user; _ } as req) =
  match user with
  | None -> error_response ~message:"You need to be logged" 401 req
  | Some user -> handler user req
;;

It's roughly the same as our user_required middleware, except that this time, the user is “sent” to the next handler (which is our service's controller), which will change the nature of the slot (previously ()) located between the query parameter and the request..

let yet_another_service =
  service
    ~middleware:user_required
    ~context:provide_user
    ~route:another_route
    (fun [] { author; category; limit } provided_user _req ->
       [ "Author: " ^ author
       ; "Category: " ^ category
       ; ("Limit: "
          ^ Option.(value ~default:"none" (map string_of_int limit)))
       ; "Hello " ^ provided_user
       ]
       |> String.concat "\n")
;;

If you don't need context, you can simply use the no_context function, which returns unit as the context (as in our definition of simple_service).

Conditions

In addition, a service can “hold” two conditions: precondition and postcondition, which are two functions that return Boolean values. If either of these functions returns false, the router moves on to analyzing the route for the next service. The conditions therefore allow the router to skip the route currently being analyzed during the routing process.

This is convenient because once a service has booted (i.e., once it has been selected in the routing), it is not possible to move on to the next route from the controller—but why are there two levels of conditions?

• precondition is a function 'request -> bool, It takes action immediately after verifying that the methods match, which is why it uses only the request as the subject of observation.

• postcondition is a function 'args args -> 'param -> 'request -> bool It runs after the path fragments have been parsed and after the query parameters have been validated. It provides additional context to help determine whether to validate the route. It runs immediately before the middleware (and the context) are applied.

By default, both conditions always evaluate to true.

Performing routing

Now that we've seen how to describe route and service (and how to constrain them using middleware, context, and conditions), we'll take a broad look at how routing works.

To route, we use the dispatch function, which has the following type:

# let dispatch = Service.dispatch ;;
val dispatch :
  given_method:meth ->
  given_path:string list ->
  given_query_params:(string * string) list ->
  ('a, 'b) service list -> ('a, 'b) middleware = <fun>

For reasons of generality (once again), the function assumes that query parameters have been normalized as described in the previous sections and that the path is a list of strings; for example, a/b/foo becomes [“a”; ‘b’; “foo”]. It then takes a list of services as an argument and returns a middleware. In other words, if the function doesn't find a route, it passes control to the next handler (which could be another piece of middleware).

Analysis of a candidate service

Here is a general overview of how the router determines whether a service is a candidate or not:

• Checks whether the service's method is the same as the one provided by given_method.
• Checks the precondition.
• Checks the path using given_path (and extract path values into an args).
• Checks the param using given_query_params.
• Checks the postcondition.

At this stage, the service is validated as a result and the router apply middleware and context:

• Apply the context of the service.
• Apply the middleware (if it exists).

At this time, Highway does not “intelligently” sort service routes.

There you go—you've just gotten a quick overview of the various features offered by Highway, just like a generic router. Here is the full API.

Types

Re-exporting utility types to simplify the API.

Describing patterns

Literal Pattern

Hole Pattern

You can define your own patterns using Hole and Pattern.

Query parameters

Allowing query parameters to be taken into account is a potentially debatable choice because, unlike the placeholders introduced in a route's patterns, the order of the parameters is of little importance. For this reason, all parameters observable in the router are processed by a parameter validation function.

You can describe and compose more Query Param description using the module Param.

Building Param description

From holes

For "unique" query parameters, you can use Hole to describe them "on the fly."

Prebuilt params on top of holes

A set of pre-built query parameters based on Hole. All of these parameters take a string (the query parameter key) as an argument.

Routes

Building internal routes

Building global routes

Link Generation

Routes are used to generate links and generally follow this pattern: function ?anchor ?extra_params route args.

They are backed by quoted versions that prevent extra parameters from being passed to routes that do not accept query parameters.

Middleware

A middleware is a composable function that wraps a web handler to process a request before it reaches the handler and/or a response after it returns.

Context

A context is a specific type of middleware that allows data to be injected arbitrarily into a service handler associated with a route.

Service

A service is a controller associated with a route. Along with route, it is the main component of Highway.

Routing services

Now that we can describe services, the final step is to choose the right service from a given list.

Infix

A set of infix operators to simplify the use and composition of some Highway objects.

Internal modules

Re-exporting internal modules (if functions are not re-exported in the main module).