joetjen.net
EN DE
Web framework

Core concepts

The contract, and the libraries built on it — each shown with code from Webglue’s own sources, tests and demo application.

In development · v0.4

The contract

A request is a map with :method (a lowercase keyword), :path (still percent-encoded), :query, :scheme, :host, :port, :remote, :version, :headers and :body. A response is a map with :status, :headers and :body. That is the whole of webglue_spec.

  • Headers are a map from lowercase name to a list of values, on both sides — a list is the only shape that is right for Set-Cookie.
  • A body is a string, a stream that can be read once, or nil.
  • Middleware adds its own keys — :session, :path-params, :current-user, :repo — and each is documented by the library that sets it.

Short-circuiting needs no marker. A middleware that should stop the request simply does not call the handler; a :halted flag was considered and rejected as a security hazard.

(defn wrap-auth [handler]
  (fn [request]
    (if (authorised? request)
      (handler request)
      {:status 403 :headers {} :body nil})))

Handlers and middleware

Middleware is written by hand the same way the libraries write it. The demo’s own puts the signed-in user on the request:

(defn wrap-user
  [handler] :spec [function -> function]
  (fn [request]
    (let [named (or (:user (:session request)) (spec/header request "x-demo-user"))]
      (handler (if (nil? named)
                   request
                   (map/assoc request :current-user named))))))

Routing

A route table is data: a vector of [method pattern handler], tried in order, first match wins. In a pattern a string matches itself, a keyword binds one segment and :* binds the rest; "/users/:id" is shorthand for ["users" :id]. Matches land on :path-params, decoded.

(defn routes [] :spec [-> vector]
  [(router/get "/socket/tickets" watch)
   (router/post "/session" sign-in)
   (router/delete "/session" sign-out)
   (router/group "/tickets"
                 [(router/post "/" raise)
                  (router/get "/" index)
                  (router/get "/:id" show)
                  (router/post "/:id/assign" assign)
                  (router/post "/:id/hold" hold)
                  (router/post "/:id/resume" resume)
                  (router/post "/:id/close" close)])])
(router/segments "/users/:id")                       ; => ["users" :id]
(router/group "/users" [[:get [] :index]])           ; => [[:get ["users"] :index]]
(router/allowed [[:get ["users"] :index]] "/users")  ; => [:get :head :options]

HEAD falls through to GET, and OPTIONS is answered from the table with an Allow header. router/handler answers 404 when nothing matches; router/wrap falls through to the next handler instead. prx routes webglue.demo.web prints the table.

Servers

A server adapter is passed in as a function, not named by an atom — a server the project does not have is a name that does not compile. An adapter has three verbs: start, stop, and child, which returns the supervision child spec an application actually uses.

webglue_demo/apps/web/src/webglue/demo.prax
(defn init [_argument] :spec [any -> tuple]
  (supervisor/tree (supervisor/flags :one-for-one)
                   [(pubsub/child)
                    (presence/child)
                    (supervisor/child webglue.session.memory nil)
                    (supervisor/child webglue.demo.store.memory nil)
                    (server/child (adapter) (web/app) {:port 4000})]))

(defn- adapter [] :spec [-> function]
  (if (= (system/env "WEBGLUE_ADAPTER") "wrangler")
      wrangler/child
      bandit/child))

There are two adapters. webglue_bandit runs on Bandit and Plug, and gives HTTP/1.1, HTTP/2 and WebSockets; all of Plug is confined to that one library. webglue_wrangler is an HTTP/1.1 server written in Praxis that speaks the contract directly, with keep-alive, pipelining, streamed bodies and WebSockets. It becomes the default only once it has earned it: a parity suite sends the same raw bytes through both adapters and expects the same answer — on its first run it found real bugs in Wrangler.

Sessions

A handler reads :session from the request — always a map, never nil. To write a session it sets :session on the response; nil ends it; leaving the key out changes nothing. Every write mints a new token, which rules out session fixation. The cookie defaults to HttpOnly, SameSite=Lax and Path=/; storage sits behind a Store behaviour.

(defn- sign-in [request] :spec [map -> map]
  (let [who (map/get (form-of request) "user")]
    (if (praxis.string/blank? (or who ""))
        (text 400 "a session needs a user")
        (map/assoc (text 200 (str/join (list "signed in as " who) ""))
                   :session {:user who}))))

(defn- sign-out [_request] :spec [map -> map]
  (map/assoc (text 200 "signed out") :session nil))

Static files

static/wrap answers GET and HEAD for files under a directory, with the content type taken from the extension. Anything else falls through — a missing file becomes the application’s own 404. Path traversal is refused by checking where a path resolves, not which spellings it uses:

(deftest and-a-path-climbing-out-of-the-directory-is-refused
  (assert= (:status (serve (test/request :get "/assets/../../project.pxn"))) 404)
  (assert= (:status (serve (test/request :get "/assets/css/../../../../etc/passwd"))) 404))

Validation

One function, check, and rules as data. String keys go in, keyword keys come out, values are coerced from text, and every failure is returned — not just the first.

(v/check {:page {:type :integer}} {"page" "2"})     ; => #(:ok {:page 2})
(v/check {:debug {:type :boolean}} {"debug" "on"})  ; => #(:ok {:debug true})

(def- raising
  {:subject  {:type :string :required true :max 200}
   :body     {:type :string :default ""}
   :reporter {:type :string}})

(defn- raise [request] :spec [map -> map]
  (with [#(:ok asked) (v/check raising (form-of request))]
        (text 201 (rendered (announced :raised (store/raise (db/repo-of request)
                                                            (:subject asked)
                                                            (:body asked)
                                                            (:reporter asked)))))
        [#(:error failures) (text 400 (complaint failures))]))

The data layer

db/wrap puts a repository on the request as :repo, with optional checkout and transaction; db/repo-of reads it back. Rolling back is explicit — (db/rollback response) — and never guessed from a status code. Webglue calls whatever module it is handed, so Ecto is the application’s dependency, not the framework’s. The demo uses the seam for an in-memory store behind a behaviour.

Sockets

A socket is a response. A handler answers (socket/upgrading handlers state), a 101 response carrying a :socket key — so it has already passed through the session and every other middleware, which is how socket authentication is settled. There are four optional callbacks, :connected, :received, :informed (a message from elsewhere in the system) and :closed, and three answers: socket/keeping, socket/pushing, socket/closing.

(defn- watch [request] :spec [map -> map]
  (socket/upgrading {:connected joined :informed told :received asked}
                    {:topic "tickets" :who (:current-user request) :sent 0}))

(defn- joined [state] :spec [map -> tuple]
  (do
    (pubsub/subscribe (:topic state))
    (presence/track (:topic state) (or (:who state) "anonymous") {})
    (socket/pushing (fragment/frame
                      (fragment/data (:topic state)
                                     {:watching (:topic state)
                                      :whom     (praxis.map/keys
                                                  (presence/who (:topic state)))})
                      on-the-wire)
                    state)))

Pub/sub and presence

A subscriber is a process. (pubsub/subscribe "topic") and (pubsub/broadcast "topic" message) run on Erlang’s pg, so they span the cluster as soon as nodes are connected; a Redis adapter is a start option away. (presence/track topic key meta) records the calling process and monitors it — no heartbeat — and joins and leaves are broadcast on the same topic as {:presence-diff {:joined … :left …}}. The demo’s socket received presence diffs without a line of code for it.

(cluster/child {:strategy (cluster/static (list "web@10.0.0.4" "web@10.0.0.5"))})
(cluster/child {:strategy (cluster/by-dns "app.svc.cluster.local" "web")})

Clustering discovers nodes through a strategy, which is just a function returning node names.

Server push

webglue_fragment pushes two kinds of envelope to connected clients: data, and HTML fragments addressed to a target. A fragment is a markup tree of plain vectors and maps, not a string; the serializer escapes by position, so a ticket titled <script>alert(1)</script> stays text. Frames go out as binary DXN by default, or as DXN text or JSON.

(defn- row [given] :spec [map -> any]
  [:li {:class (str/str (:status given)) :id (str/join (list "ticket-" (str/str (:id given))) "")}
   [:span {:class "id"} (str/join (list "#" (str/str (:id given))) "")]
   [:span {:class "subject"} (:subject given)]
   (if (nil? (:assignee given))
       nil
       [:span {:class "who"} (:assignee given)])])

(defn- announced [event ticket] :spec [keyword map -> map]
  (do
    (fragment/push-data "tickets" {:event event :ticket ticket})
    (fragment/push-html "tickets" "#tickets" (row ticket))
    ticket))

Testing

Because an application is a function, testing it needs no server. test/request builds a complete request map, test/answering is a stand-in handler with a fixed response, and test/echoing hands back the request it received — the way to see what a middleware did.

(deftest echoing-is-how-you-see-what-a-middleware-did
  (let [wrap (fn [handler]
               (fn [request] (handler (praxis.map/assoc request :locale "de"))))
        app  (wrap (test/echoing))]
    (assert= (:locale (:request (app (test/request :get "/")))) "de")))

Configuration

Configuration is CASC. The main file imports every file of the current environment’s directory — last, so the environment wins — and each concern gets a file of its own:

config/config.casc
#@version = 1.0

webglue {
}

# Last, so what an environment says wins over what is above it.
import "${PRX_ENV:dev}/*.casc"
config/dev/logger.casc
#@version = 1.0

praxis.logger {
  level = debug
  formatter = "praxis.logfmt"
}