Queues are useful for running jobs in the background. Typical use cases are exporting tables as CSV files or sending emails over SMTP that take too long to perform during an HTTP request.
In essence, you create a
job and register it with a job worker. In your app, use
Sihl.Contract.Queue.Sig.dispatch to put the
job onto a queue for later processing.
sihl-queue ships with 3 backend implementations.
First, choose a backend in
module Queue = Sihl_queue.PostgreSql
Register the service in
let services = [ Service.Queue.register ~jobs: () ]
make sihl migrate to run pending migrations.
The service API is documented in
Sihl_queue.create you can create a job in
let cook_pizza = Sihl_queue.create (fun pizza_name -> Pizza.create_pizza pizza_name  |> Lwt.map ignore |> Lwt.map Result.ok) "cook-pizza" ;;
Don't forget to register the job with the queue service. The queue service comes with queue workers which need to know about the jobs available. In
let services = [ Sihl.Database.register () ; Service.Migration.(register ~migrations:Database.Migration.all ()) ; Service.Queue.register ~jobs:[ Job.cook_pizza; Job.order_ingredient ] () ] ;;
You can only dispatch jobs that have been registered.
Service.Queue.dispatch ~input:"funghi" cook_pizza
Service.Queue.dispatch ~input:"funghi" ~delay:(Sihl.Time.Span.minutes 2) Job.cook_pizza
Lwt.t resolves as soon as the job is queued.
You can also dispatch multiple jobs of the same type with different inputs. Following dispatches the same job 3 times with different inputs.
Service.Queue.dispatch_all ~input:["funghi"; "salami"; "prosciutto"] ~delay:(Sihl.Time.Span.hours 1) Job.cook_pizza
sihl-queue comes with a built-in dashboard. The dashboard is packaged as a router
Sihl.Contract.Queue.Sig.router and can be mounted into any existing app.
In order to only allow authenticated users to use the dashboard you can add your custom authentication and authorization middlewares. In
let router_admin_queue = Service.Queue.router ~back:"/" ~prefix:"/path" "/admin/queue"
let services = [ Sihl.Database.register () ; Service.Migration.(register ~migrations:Database.Migration.all ()) ; Sihl.Web.Http.register ~middlewares:Routes.Global.middlewares ~routers: [ Routes.Api.router ; Routes.Site.router_admin_queue ] () ; Service.Queue.register ~jobs:[ Job.cook_pizza; Job.order_ingredient ] () ] ;;
The dashboard has built-in support for HTMX. However, it is not requird to use HTMX and the dahsboard remains usable without. HTMX is used for dynamic features like auto-refreshing parts of the job instance list.
In order to use HTMX set
public or by a CDN.