Pablo Fernandez

Tag: ClojureScript

  • Pre-signed S3 URLs with Clojure and ClojureScript

    , , ,

    Jacob Emcken, the author of aws-simple-sign, reached out to me to share that his library can probably help with this, so if you need to achieve this goal, you might want to check it out.

    Disclaimer: I haven’t tried it or looked at its code.

    Dashman uses S3 for storing the encrypted screenshots. They can go directly from the renderer application to S3 and from there to the displayer application as there’s no need for the server application to ever see them. They are end-to-end encrypted anyway.

    My needs in this regard were a bit unique. I have a server component that can generate whatever tokens and signatures are needed and then send it to both applications. The applications are written in ClojureScript, so, essentially, JavaScript, but unlike most direct-to-S3 upload solutions, I don’t have a web site with a form, so, I don’t need to fiddle with CORS or forms.

    The two libraries

    I found two libraries that could have helped me. The big one is amazonica, a Clojure library to interact with AWS. It looks very complete but when I tried to use the com.amazonaws.services.s3/generate-presigned-url function, I got this error:

    The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

    There are two versions of signatures, version 2 and 4. Version 2 is deprecated, it might or might not work on some data centers but it definitely doesn’t work on all. I refused to believe it at first because I was using very recent version of all libraries, but yup, it was generating version 2 of the signature with signed URLs that look something like this:

    https://bucket-name.s3.amazonaws.com/key-name?AWSAccessKeyId=AKI...OA&Expires=1494190327&Signature=ct...3D

    Digging further I found this snippet of code:

    (System/setProperty SDKGlobalConfiguration/ENFORCE_S3_SIGV4_SYSTEM_PROPERTY "true")

    But for me, it made no difference in my case. For the record I reported the issue and it is supposed to work, but it doesn’t for me.

    Then there’s s3-beam, that looked promising, but it seems targeted towards forms, not just plain PUT requests from the clients. On top of that, I didn’t want to mess with async, I didn’t need it and it would be over-engineering for my case. Lastly, it uses it’s own custom way of creating the signatures instead of Amazon’s SDK and that code looks like it’s generating the old signatures (sha1 instead of sha256). I ended up not even trying it.

    Instead, I decided to do my own custom option, which was not hard and worked straight away. This post is about sharing that code.

    Custom solution using the AWS Java SDK

    Starting on the server/Clojure side of things, I added the dependency:

    [com.amazonaws/aws-java-sdk "1.11.126"]

    With only that I was getting this unexpected error when trying to use the AWS SDK:

    ERROR com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
ERROR Could not initialize class com.amazonaws.partitions.PartitionsLoader

    which I resolved by adding these other dependencies as well (mostly about getting the latest version):

    [com.fasterxml.jackson.core/jackson-core "2.8.8"]
[com.fasterxml.jackson.core/jackson-databind "2.8.8"]
[com.fasterxml.jackson.core/jackson-annotations "2.8.8"]

    I wrote a function to generate the S3 client:

    (defn aws-s3-client []
  (let [region (Regions/fromName (config/env :aws-s3-region))
        credentials (BasicAWSCredentials. (config/env :aws-access-key-id)
                                          (config/env :aws-secret-access-key))]
    (-> (AmazonS3ClientBuilder/standard)
        (.withRegion region)
        (.withCredentials (AWSStaticCredentialsProvider. credentials))
        .build)))

    config/env just fetches configuration values, powered by cprop and mount. This is just standard stuff from Luminus. With

    (defn- pre-signed-url [key & [verb]]
  (let [expiration (tc/to-date (-> 24 t/hours t/from-now))
        generate-presigned-url-request (doto (GeneratePresignedUrlRequest. (config/env :aws-s3-bucket-name) key)
                                         (.setMethod (if (= verb :put)
                                                       HttpMethod/PUT
                                                       HttpMethod/GET))
                                         (.setExpiration expiration))]
    (.toString (.generatePresignedUrl (aws-s3-client) generate-presigned-url-request))))

    The function generates either a URL for reading, :get, or for writing, :put, depending on the parameter verb. key is just the path part of the URL (remember, S3 is actually a key/value storage). All URLs will last for 24hs, something you can tweak as well.

    And that’s it, generating the URL is done, now we have to use it.

    Reading and writing from ClojureScript

    If you have a form, then, you might need to deal with CORS and other issues. I suggest you look at s3-beam at least for inspiration. For me, it was just a matter of generating the requests from the ClojureScript app, which uses cljs-ajax. Reading was straightforward enough:

    (ajax/GET get-url
          {:handler (fn [value]
                      (do-something value))})

    although the handler might have to deal with encoding issues. Writing is similar, but again, you may need to fiddle with encoding issues. Since what I was sending is a JSON payload (an encrypted screenshot), this worked for me:

    (ajax/PUT put-url
          {:format :json
           :params payload})

    If you are uploading binary data, I recommend you play with :format :raw and the :body attribute.

  • How to work with a private library in ClojureScript

    ,

    Dashman is composed of many components  including three desktop apps written in ClojureScript using Electron that share code through a private library (as in, it’s not open source).

    To have continuous integration and a set up that is easy to boostrap, that library should be deployed to a private repository. I achieved that using the plug in s3-wagon-private so that the project.clj of the common library looks like this:

    [code lang=”clojure”]
    :repositories [["private" {:url "s3p://s3-bucket-for-my-project/releases/"
    :username "not-telling-you"
    :passphrase "not-telling-you-either"
    :sign-releases false}]]

    :plugins [;…
    [s3-wagon-private "1.3.0"]]
    [/code]

    You should never have production or even staging credentials on your source code repository; but those credentials are neither. They allow to compile the project so, pretty much everybody that has access to the source code also needs access to that private repository. Having them in project.clj simplifies CI, on-boarding developers, etc.

    One you have that you can deploy the library by executing:

    [code lang=”bash”]
    lein deploy private
    [/code]

    The project of the apps using the common library look similar:

    [code lang=”clojure”]
    :repositories [["private" {:url "s3p://s3-bucket-for-my-project/releases/"
    :username "not-telling-you"
    :passphrase "not-telling-you-either"
    :sign-releases false}]]
    :dependencies [;…
    [tech.dashman/clientcommon "0.1.0-SNAPSHOT"]]

    :plugins [;…
    [s3-wagon-private "1.3.0"]]
    [/code]

    Yes… very similar.

    The next issue is compiling and re-loading code. You could modify the common library, install it, re-compile the apps, and re-run the apps. And you could do it while wearing a mullet and watching MacGyver (the original one). This is not the 80s! You want code reloading and all the goodies.

    If you are using Clojure, you should look into checkouts and maybe lein-checkout. In ClojureScript you can just add the path of the source code to your project even if it’s outside the project, with one gotcha. You might be used to a config like this:

    [code lang=”clojure”]
    :cljsbuild {:builds {:app {:source-paths ["src/app"]}}}
    [/code]

    which would imply this might work:

    [code lang=”clojure”]
    :cljsbuild {:builds {:app {:source-paths ["src/app" "../clientcommon/src"]}}}
    [/code]

    It might on non-Windows platforms, on Windows it doesn’t. On Windows you want this:

    [code lang=”clojure”]
    :cljsbuild {:builds {:app {:source-paths ["src/app" "..\\clientcommon\\src"]}}}
    [/code]

    Since you should probably never commit that anyway, it’s not a big deal.

  • ClojureScript debouncer

    ,

    I’m developing Dashman in ClojureScript (and Clojure) and as I was developing the cookie handler I found sites that constantly modify the cookies. As in, many times per second. Obviously I don’t want to encrypt the cookies and send them to the server that often, but even without taking that into account, just storing them to keep track of them in an atom, as in:

    (reset! cookies new-cookies)

    was slowing down the app considerably. I found myself needing to debounce so many calls as there’s no value in the middle calls, only the last one. Let’s backtrack a little bit here.

    The term to debounce comes from electronics:

    To remove the small ripple of current that forms when a mechanical switch is pushed in an electrical circuit and makes a series of short contacts.

    In programming it refers to a signal that gets emitted often and you want to process it but you don’t care about processing every intermediate value. To tie into electronics imagine you have a program that receives a message every time the temperature changes and the message contains all the temperature changes for the day. Clearly each message doesn’t have to be processed, only one, only the latest.

    Here’s where a software debouncer can help. It’s a function that gets a function and returns another function. When you call the resulting function 10 times, it will only call the original function once, with the last value it received.

    I found this debouncer written by David Nolan: https://gist.github.com/swannodette/5888989 and in the comments you can start to see that writing a debouncer might not be a trivial as you may think. One of the comments point to this other version: https://gist.github.com/scttnlsn/9744501. The comment implies this is a more correct version but I haven’t verified it myself. In that last page you can see someone found a bug and posted an improved version.

    For me, when a supposedly simple code snippet generates so many comments and versions, it’s better to find a library that implements it than to copy and paste and never maintain it again. Unfortunately, I couldn’t find a ClojureScript debouncer library but I got pointed to goog.function.debounce which you can use this way;

    (ns whatever
  (:require [goog.functions]))

(def debounced-println (goog.functions/debounce println))

  • Free-form version 0.2.0 released

    , , , , , , ,

    We are very happy to announce version 0.2.0 of our form building library Free-form. This version includes:

    The Bootstrap 3 support means that you can have whole fields defined as succinctly as:

    [:free-form/field {:type        :email
                   :key         :email
                   :label       "Email"}]]

    Enjoy!

     

  • Prerenderer 0.2.0 released

    , , , , ,

    We are proud to announce the release of version 0.2.0 of our ClojureScript library Prerenderer, a library to do server side pre-rendering of single page applications. In this release, we include:

    The two first items in the changelog came hand in hand and they are the biggest changes to keep in mind if you are upgrading. We are very happy that we no longer need a fork of re-frame and we would like to extend our gratitude to Mike Thompson for working with us on having the appropriate API to make this happen.

    The change in API means that your Prerenderer module now would look something like this:

    (ns projectx.node
  (:require [prerenderer.core :as prerenderer]))

(defn render-and-send [page-path send-to-browser]
  (send-to-browser (render page-path)))

(set! *main-cli-fn* (prerenderer/create render-and-send "ProjectX"))

    instead of:

    (ns projectx.node
  (:require [cljs.nodejs :as nodejs]
            [prerenderer.core :as prerenderer]))

(defn render [req res]
  (let [page-path (.-path (.parse url (.-url (.-query req))))]
    (.send res (render page-path))))

(set! *main-cli-fn* (prerenderer/create render "ProjectX"))

    Enjoy!

  • Don’t forget to clear your client side state when logging a user out

    , , , ,

    When a user logs out from our web site, we are used to clearing the session and that’s it. When you are developing a single page application, you are likely to keep a lot of state on the client side, and that should be cleared too.

    For Ninja Tools, that meant going from the traditional re-frame initialize-db:

    (re-frame/register-handler
  :initialize-db
  (fn [_ _]
    (re-frame/dispatch [:get-current-user])
    {:current-route        nil
     :alerts               (sorted-map)
     :current-user         nil
     :log-in-form          {}
     :registration-form    {}
     :reset-password-form  {}
     :change-password-form {}
     :tools                nil
     :used-tools           nil}))

    to having the initial-db in a re-usable value:

    (def initial-db {:current-route        nil
                 :alerts               (sorted-map)
                 :current-user         nil
                 :log-in-form          {}
                 :registration-form    {}
                 :reset-password-form  {}
                 :change-password-form {}
                 :tools                nil
                 :used-tools           nil})

(re-frame/register-handler
  :initialize-db
  (fn [_ _]
    (re-frame/dispatch [:get-current-user])
    initial-db))

    and our logged-out handler to use it instead of modifying the current estate, which meant going from:

    (re-frame/register-handler
  :logged-out
  (fn [db [_]]
    (routing/redirect-to :home)
    (-> db
        (assoc :current-user nil)
        (alerts/add-alert :success "You are now logged out."))))

    to:

    (re-frame/register-handler
  :logged-out
  (fn [db [_]]
    (routing/redirect-to :home)
    (-> db/initial-db
        (alerts/add-alert :success "You are now logged out."))))

    Since we care so much about security, for us, it’s important to go back to initial-db, and if there’s some state that should survive, we’ll pass it on manually. That is, we’ll be doing whitelisting vs blacklisting.

    Something that we haven’t decided is whether we clear the state on log-out, when the user just clicked the log out link, or logged-out, when when the server has cleared the session.

    The advantage of the former is that we clear all the state as soon as possible, the advantage of the later is that should the log out procedure fail for some reason, the app still has the state and it should still be usable, which might be required for re-trying logging out.

    Recapping that, clearing state immediately behaves better when everything goes well, clearing state later behaves better when something goes wrong during the log out process. We can’t comment one being safer that the other, as clearing the session earlier and failing to log out might leave the user under the impression they successfully logged out when it’s not the case.

    Picture by Ken Hawkins.

  • Isomorphic JavaScript (with ClojureScript) for pre-rendering single-page-applications, part 3

    , , , , ,

    I was not expecting there to be a part 3 to this series and this third part is also going to be quite different to the first two. In parts 1 and 2 I walked you through an exploration of server side pre-rendering with Nashorn. My naive example worked fine with Nashorn but it didn’t survive encountering the real world.

    Nashorn is not a headless browser, it’s a plain JavaScript engine. It doesn’t implement document or window for example, which were easy to workaround, but it also doesn’t implement setTimeout, setInterval or XMLHttpRequest which are much harder to workaround.

    When I started to look for alternatives I focused on NodeJS because I knew if implemented those things I was missing and V8‘s speed is attractive. Also, the fact the ClojureScript has it as a compilation target made me feel it was well supported, a first class citizen.

    At this point someone might interject and exclaim: What about nodyn.io? nodyn.io is an attempt to bring all the NodeJS goodness to Nashorn and I think it’s a great idea. Sadly, on GitHub we can find this notice:

    This project is no longer being actively maintained. If you have interest in taking over the project, please file an issue.

    I’m not sure if the project got far before being abandoned either.

    Implementing the missing bits of Nashorn in Clojure was tempting. It looks like fun and it also looks like something that might be popular amongst Java users and thus good for the Clojure echo system. I exercised some restrain and moved on.

    In the process of experimenting with NodeJS my code quickly took the form of a library and without further ado, let me introduce you to Prerenderer. The ultimate solution for all your server side pre-rendering needs. I’m not going to show you how to use it here because its page go into a lot of detail already.

    My big concern about prerendering is performance and stability. As I showed in part 2, a naive implementation can behave horribly while in production, sometimes taking up to 17 seconds to serve a page. Prerenderer was not developed with a sample project, like I did with Nashorn, but with a project we are working on called Ninja Tools that uses re-frame and AJAX. Before any modifications to it, this was its performance:

    After enabling Prerenderer, this is how it looks like:

    The average response time went up from 51ms to 362ms. This would generally be a very bad thing. The reason for this is explained in Prerenderer’s documentation:

    […] SPAs are never done. Imagine a SPA that has a timer and every second sends a request to the server, and the server replies with the current time, which then the application displays. When is it done rendering? Never. But Prerenderer needs to, at some point, decide that the page is done enough and ship it to the browser. […]

    […] Prerenderer will watch for events and once nothing happened for a period of time (300ms by default) it’ll consider the application done and if a certain amount of time went by (3s by default) even if the application is still active, it’ll stop and send it to the browser.

    That’s where the jump in 300ms is coming from and it’s constant. It’s not linear and definitely not exponential. It’s a constant number that can be tuned and tweaked. There are also some potential optimizations to reduce it or remove all together.

    The important thing is that all other values remained more or less the same and that the performance characteristics where quite stable. For me, this feels good enough to move on and start producing SPAs and with a bigger codebase we’ll be able to improve this library and make it better.

    Picture by Ian Farrel

  • Bidi vs Silk

    , , , ,

    In previous blog posts I mention that Bidi and Silk are essentially equivalent. I don’t believe this anymore. I now prefer Silk and I can show you why with a little example. First, let’s define some routes:

    (def silk-routes (domkm.silk/routes [[:home-page [[]]]
 [:about [["about"]]]]))

(def bidi-routes ["/" {"" :home-page
 "about" :about-page}])

    When it comes to defining routes, I find both a bit cryptic. Bidi feels a bit easier to read but I found it was harder to write in some scenarios.

    Continue reading →

  • Bidi vs Silk

    , , , ,

    In previous blog posts I mention that Bidi and Silk are essentially equivalent. I don’t believe this anymore. I now prefer Silk and I can show you why with a little example. First, let’s define some routes:

    (def silk-routes (domkm.silk/routes [[:home-page [[]]]
                                     [:about [["about"]]]]))

(def bidi-routes ["/" {""      :home-page
                       "about" :about-page}])

    When it comes to defining routes, I find both a bit cryptic. Bidi feels a bit easier to read but I found it was harder to write in some scenarios.

    Let’s parse some paths:

    (bidi.bidi/match-route bidi-routes "/")
; => {:handler :home-page}

(domkm.silk/arrive silk-routes "/")
; => {:domkm.silk/name :home-page, :domkm.silk/pattern {:path []}, :domkm.silk/routes #object[domkm.silk.Routes 0x7a46e3be "domkm.silk.Routes@7a46e3be"], :domkm.silk/url #domkm.silk.URL{:scheme nil, :user nil, :host nil, :port nil, :path [], :query {}, :fragment nil}}

(bidi.bidi/match-route bidi-routes "/about")
; => {:handler :about-page}

(domkm.silk/arrive silk-routes "/about")
; => {:domkm.silk/name :about, :domkm.silk/pattern {:path ["about"]}, :domkm.silk/routes #object[domkm.silk.Routes 0x7a46e3be "domkm.silk.Routes@7a46e3be"], :domkm.silk/url #domkm.silk.URL{:scheme nil, :user nil, :host nil, :port nil, :path ["about"], :query {}, :fragment nil}}

    So far I would consider them equivalent. Silk gives you more information about the route but is also more noisy. I personally dislike the namespaced keywords, but that’s easily solved with:

    (defn sanitize-silk-keywords [matched-route]
  (rename-keys matched-route {:domkm.silk/name    :name
                              :domkm.silk/pattern :pattern
                              :domkm.silk/routes  :routes
                              :domkm.silk/url     :url}))

    The real difference, for me, comes when I try to parse /about?,which should be the same as /about and some lazy URL handling libraries emit the former rather than the latter. Silk first:

    (domkm.silk/arrive silk-routes "/about?")
; => {:domkm.silk/name :about, :domkm.silk/pattern {:path ["about"]}, :domkm.silk/routes #object[domkm.silk.Routes 0x7a46e3be "domkm.silk.Routes@7a46e3be"], :domkm.silk/url #domkm.silk.URL{:scheme nil, :user nil, :host nil, :port nil, :path ["about"], :query {}, :fragment nil}}

    No surprised there. What about Bidi:

    (bidi.bidi/match-route bidi-routes "/about?")
; => nil

    Oops, what happened here? A bit of digging around lead me to the issue Why are query parameters not supported anymore? Essentially, Bidi’s design is that you shouldn’t route on query arguments (which I would agree in principle) and thus it relays on others to separate the two, something I don’t like at all. In a ClojureScript application it would requiring some pre-parsing of the URL, while Silk does it for you.

    Silk also seems to support, if not routing, at least reporting port, host, user, scheme and even fragment. This can come in handy at some point. If you want to learn more about using Silk and Pushy, take a look at the blog post No-hashes bidirectional routing in re-frame with Silk and Pushy.

    Picture by MartialArtsNomad.com

  • Isomorphic JavaScript (with ClojureScript) for pre-rendering single-page-applications, part 2

    , , , , , , , , ,

    In part 1 I covered the basic problem that SPA (single page applications) face and how pre-rendering can help. I showed how to integrate Nashorn into a Clojure app. In this second part, we’ll get to actually do the rendering as well as improving performance. Without further ado, part 2 of isomorphic ClojureScript.

    Rendering the application

    Now to the fun stuff! It would be nice if we had a full browser running on the server where we could throw our HTML and JS and tell it go! but unfortunately I’m not aware of such thing. What we’ll do instead is call a JavaScript function that will do the rendering and we’ll inject that into our response HTML.

    The function to convert a path into HTML will be called render-page and it’ll be in core.cljs:

    (defn ^:export render-page [path]
  (reagent/render-to-string [(parse-path path)]))

    We need to mark this function as exportable because JavaScript optimizations can be very aggressive even removing dead code and since this code is called dynamically from Clojure, it’ll look like it’s unused and it’ll be removed.

    render-page  is similar to mount-root but instead of causing the result to be displayed to the user, it just returns it. The former takes the path as an argument while the latter reads it from the local state which is in turn set by Pushy by reading the current URL.

    To invoke that function, we’ll go back to handler.clj, just after we define js-engine we’ll define a function called render-page:

    render-page (fn [path]
              (.invokeMethod
                ^Invocable js-engine
                (.eval js-engine "projectx.core")
                "render_page"
                (object-array [path])))

    and instead of sending a message about the application is loading, we just call it:

    [:div#app [:div (render-page path)]]

    That extra div is not necessary, it’s there only because projectx.core/current-page adds it and without it you’ll get a funny error in the browser:

    Aside from that little trip into the internals of React, which is interesting, we now have a snappy, pre-rendered application… that is… if you can wait 3 seconds or so for it to load:

    That is not good, not good at all. We have a serious performance problem here, we need to get serious about fixing it.

    Performance

    The first step to fix any performance problems is making sure you have one, as premature optimization is the root of all evil. I think we are at this point with this little project. The second step is measuring the problem: we need a good repeatable way of measuring the problem that allows us to actually locate it and and verify it was fixed.

    To measure the performance behaviour of this app I’m going to use one of Heroku’s bigger instances, the Performance-L, which is a dedicated machine with 14GB of RAM. The reason is that I don’t want out of memory or my virtual CPU affected by other instances to muddy my measurements. That unacceptable 3 seconds load time was measured in that type of server.

    To perform the load and the measurement of the response I’m going to use the free version of BlazeMeter, an web application to trigger load testing which I’m falling in love with. The UI is great. I’m going to hit the home and the about page with their default configuration which includes up to 20 virtual users:

    In all the tests I’m going to make a few requests to the application manually after any restart to make sure the application is not being tested in cold. Ok… go!

    That is terrible! Under load it behaves so much worst! 17.1s response time. Now that we have a way to measure how horrendous our application is behaving, we need to pin-point which bit is causing this. The elephant in the room is of course server-side JavaScript execution.

    Disabling the server side JavaScript engine causes load times to go down:

    but what we really care about is the load testing:

    40ms vs 17000ms, that’s a big difference! The scripting engine is definitely the problem, so, what now?

    Optimizing time

    Now it’s time to find optimizations. Poking around Nashorn it seems the issue is that it has a very slow start. We already know that browsers spend a lot of time parsing and compiling JavaScript and the way we are using Nashorn, we are parsing and compiling all our JavaScript in every request. Clearly we should re-use this compiled JavaScript.

    Re-using Nashorn is not straightforward because it’s not thread safe while our server is multi-threaded. JavaScript just assumes that there’s one and only thread and when developing Nashorn they decided to respect that and not make any other assumptions, which leads to a non-thread-safe implementation. We need to re-use Nashorn engines, but never at the same time by two or more threads.

    Nashorn does provides a way to have binding sets, that is, the state of a program, separate from the Nashorn script engine, so that you could use the same engine with various different states. Unfortunately this is very poorly documented. Fortunately, ClojureScript is immutable, so we don’t have much to worry about breaking state.

    After a lot of experimentation and poking, I came up with an acceptable solution using a pool. My choice was to use Dirigiste through Aleph‘s Flow. To do that, we extract the creation of a JavaScript engine into its own function:

    (defn create-js-engine []
  (doto (.getEngineByName (ScriptEngineManager.) "nashorn")
    (.eval "var global = this")
    (.eval (-> "public/js/server-side.js"
               io/resource
               io/reader))))

    Then we define the pool. In Dirigiste, each object in the pool is associated to a key, so that effectively it’s a pool of pools. We don’t need this functionality, so we’ll have a single constant key:

    (def js-engine-key "js-engine")

    and without further ado, the pool:

    (def js-engine-pool
  (flow/instrumented-pool
    {:generate   (fn [_] (create-js-engine))
     :controller (Pools/utilizationController 0.9 10000 10000)}))

    flow is aleph.flow and Pools is io.aleph.dirigiste.Pools. In this pool you can have different controllers which create new objects in different ways. The utilization controller will attempt to have the pool at 0.9, the first arg, so that if we are using 9 objects, there should be 10 in the pool. The other two args is the maximum per key and the total maximum and they are set two numbers that are essentially infinite.

    The reason for such a big pool is that you should never run out of JavaScript engines. If your server is getting too many requests for the amount of RAM, CPU or whatever limit you find, it should be throttled by some other means, not by an arbitrary pool inside it. Normally you’ll throttle it by limiting the amount of worker threads you have or something like that.

    The function render-page was promoted to be top level and now takes care of taking a JavaScript engine from the pool and returning it when done:

    (defn render-page [path]
  (let [js-engine @(flow/acquire js-engine-pool js-engine-key)]
    (try (.invokeMethod
           ^Invocable js-engine
           (.eval js-engine "projectx.core")
           "render_page"
           (object-array [path]))
         (finally (flow/release js-engine-pool js-engine-key js-engine)))))

    The function to render the app now doesn’t create any engines, it just uses the previous method:

    (defn render-app [path]
  (html
    [:html
     [:head
      [:meta {:charset "utf-8"}]
      [:meta {:name    "viewport"
              :content "width=device-width, initial-scale=1"}]
      (include-css (if (env :dev) "css/site.css" "css/site.min.css"))]
     [:body
      [:div#app [:div (render-page path)]]
      (include-js "js/app.js")]]))

    Let’s load test this new solution:

    That is a big difference. It’s almost as fast as no server side scripting! You can find this change in GitHub: https://github.com/carouselapps/isomorphic-clojurescript-projectx/… as well as the full final project: https://github.com/carouselapps/isomorphic-clojurescript-projectx/tree/nashorn

    Future

    There are a few problems or potential problems with this solution that I haven’t addressed yet. One of those is that at the moment I’m not doing anything to have Nashorn generate the same cookies or session as we would have in the real browser.

    This pool works well when it’s under constant use, but for many web apps that do not see than level of usage, the pool will kill all script engines which means every request will have to create a fresh one. Solving this might require creating a brand new controller, a mix between Dirigiste’s Pools.utilizationController  and Pools.fixedController.

    A big thanks to DomKM for his Omelette app, that was a source of inspiration.

    Another approach worth considering is to implement the rendering system in portable Clojure (cljc), the common language between Clojure and ClojureScript and have it run natively on the server, without the need of a JavaScript engine. I’m very skeptical of this working in the long run as it means none of your rendering function can ever use any JavaScript or if they do, you need to implement Clojure(non-Script) equivalents.

    This approach is being explored by David Tanzer and he wrote a blog post about it: Server-Side and Client-Side Rendering Using the Same Code With Re-Frame. David’s approach is to use Hiccup to do the rendering on the server side, where React and Reagent are not available. I personally prefer to steer clear of template engines that are not safe by default, like Hiccup at the time of  this writing, as they make XSS inevitable. The only reason why I’m using it in projectx is because that’s what the template provided and I wanted to do the minimum amount of changes possible.

    Another optimization I briefly explored is not doing the server side rendering for browsers that don’t need it, that is, actual browser being used by people, like Chrome, Firefox, Safari, even IE (>10). The problem is that many bots do identify themselves as those types of browsers and Google gets very unhappy when its bots see a different page than the browsers, so it’s dangerous to perform this optimization except, maybe, for pages that you can only see after you log in.

    In conclusion I’m happy enough with this solution to start moving forward and using it, although I’m sure it’ll require much tweaking an improvement. Something I’m considering is turning it into a library, but this library would make quite a bit of assumptions about your application, how things are rendered, compiled, etc. What’s your opinion, would you like to see this code expressed as a library or are you happy to just copy and paste?

    Update

    There’s now a part 3 for this post.

    Photo by Jared Tarbell