camsaul.com

My Experience Using Prolog in 2024 to Solve Advent of Code

2025-02-04T18:37:00+00:00

This was my first year doing Advent of Code. Previous years I had other stuff I wanted to do with my free time besides sit at a computer even more than I already do… after seeing some of my coworkers post epic solutions I got FOMO and joined the challenge.

I decided to try to solve things in Prolog to “kill two pigs with one bird” as the Gen Zs say. Green pig #1 was learning Prolog at more than just a really superficial level… I’m a big fan of logic programming languages and Prolog has been on my list for a while. Green pig #2 was I thought it would be a good fit for solving some of these challenges since I’d (in theory) mostly be encoding the rules and letting the computer do the work for me. (Foreshadowing: it was not really this simple in practice.) A bonus pig was flexing on everyone with 1337 skillz which I think I did actually achieve a little bit, at least I like to think so.

I made it all the way to Day 18 before I got sidetracked with family get-togethers and what not. I still think I’m going to go back and finish the last week one of these days tho. At least before AoC 2025 hopefully, if the world still exists by then. Checc out my code on GitHub

Someone convinced me to Twitch stream myself solving the challenges. I really didn’t believe anyone would want to sit around and watch someone program but I guess on the off chance any of the youfs in college might find it useful to see how the “pros” (legends?) work I went ahead and did it… I actually had a lot of people tune in, and no hecklers. I might be the first person to ever live stream coding in Prolog on Twitch. Will I be doing more of this in the future? Maybe. My dream is to be a content creator/influencer and get free merch and a blue checkmark next to my name. Every day I wake up and hang my head in disgust because I am only a minor local celebrity and my invite to the Met Gala gets lost in the mail most years. Gotta stay on that grind!!!!

Anyways you can checc out my Twitch here. Will I be doing more streaming in the future? Was the moon landing faked? Scientists don’t have answers to life’s two biggest questions yet, but keep your eyes peeled.

I also posted videos of some of the live streams on my YouTube for your viewing pleasure.

TL;DR How Was Using Prolog Anyway?

Some of the challenges this year were super easy to solve in Prolog, but others really were not. There was an excessive amount of 2-D grid stuff this year, and Prolog is just really not great at this stuff. (I used SWI Prolog, and maybe one of the other implementations is better, but as a beginner I found this easy to get started and it had a great Emacs integration.) There is no native array type in Prolog, so the idiomatic way of representing a grid is probably a linked list of linked lists… this is fine for small grids but the polynomial lookup/modification times really bite you in the butt when the grid is something larger like 2000 ✕ 2000 and you have to do thousands or even millions of lookups.

At the end of it I spent way more time trying to get my Prolog code to run quickly than I would have with a language that had a better selection of data types and libraries. I wanted to change my grid code to use bit sets instead of lists-of-lists so I could use bit arithmetic for O(1) lookup and modification, but there was no bit set data type or library (that I know of) in SWI prolog… so I had to implement a hacky version myself using big integers. A lot faster than lists-of-lists for sure, but if I was writing this in Clojure I could have just used something like java.util.BitSet. This story kept repeating itself… I had to implement A* myself

Almost every challenge had me coming up with an elegant and straightforward working solution (for the examples) relatively quickly that was just way too slow for the actual challenge input. I then had to spend time reworking my pretty code and making it faster and uglier in order to complete the actual input in a reasonable amount of time.

I think like the semantics and power of Prolog a lot better than miniKanren/core.logic. I really enjoyed wrapping my head around Prolog, and really wish it was more practical to use for more things.

Pits

I think the biggest missing things for me are:

A real lack of built-in data types… if Clojure can have immutable arrays then why can’t Prolog?
A lack of libraries – why am I wasting time implementing A* myself?
Fractured ecosystem – libraries and code written for SWI Prolog are incompatible with GNU Prolog or SICStus or Scryer.
A lack of easy escape hatches to drop to lower-level stuff – I wish I could have dropped to something java.util.BitSet or a raw byte array to implement my bitset code. I don’t even know how I would go about implementing a custom data type, I’m sure it would require C code but I don’t have a good sense of how well supported extensions like that are.
Questionable performance. I think it’s way to easy to shoot yourself in the foot here and write code that is too slow for real-world use… I spent way to much time performance tuning my code.
Some basic things like lambdas (anonymous functions/closures) are missing from the core language, and you can use various competing libraries that utilize operator overloading to add support for them… the syntax is ugly and sad. Prolog syntax is not terrible but I would have preferred writing it in S-expressions.
I wish constraint logic programming was more generalized and not limited to specific libraries that apply constraints to specific data types. What if this was the way things worked normally instead of being something you have to opt in to?
Thinking in terms of relations instead of functions really is harder, and made my head hurt sometimes. Generally not too bad once I got used to it, but it was a little sad when I had to rework some of my relations so they would work efficiently if say the first parameter was ungrounded versus the second – this sort of “extra-logical” code needed to make things work efficiently takes away from the magic of pure logic programming

Peaks

The promise of writing declarative rules-based code (tell the computer what you want instead of how you want it to do something) is so alluring it still makes up for the harsh reality of actually using things in a lot of cases.
Pattern matching is amazing – this makes me want to use something like Clojure defun all over the place
SWI Prolog’s Emacs integration, Sweep, is really good, and debugging my code was really easy – especially compared to something like core.logic, which in my experience was a real nightmare to debug.
I really like Prolog’s built-in definite clause grammar parsing capabilities. They worked really well for parsing the input for most of the challenges.
I used Microsoft Copilot to tell me how to do basic stuff in Prolog, and it generally gave me pretty good answers – better than I expected given how niche Prolog is. I guess 50 years of Prolog code to train on worked well here. And while more JS code probably gets written in a given day than all the Prolog code ever written, because Prolog is so niche the AIs are probably training mostly on good code.
I am happy Prolog is not as dogmatic about pure logic programming as miniKanren. To me it seemed like the Common Lisp vs. Scheme – the former is for actually solving real problems and the latter is more for writing research papers and admiring the elegance of your solutions.
There are lots of good books about learning Prolog from “back in the day”… they don’t all cover a lot of modern stuff like CLP but they did a pretty good job of helping me wrap my head around thinking about things in a Prolog way (as opposed to “writing C in any language”).

Other thoughts

I “learned” Prolog in a programming languages course in college, and wow did they really sell the language short. I really did not fully understand the power until I spent a month doing self-study
The Mercury language looks really interesting and makes bold promises about performance but it doesn’t seem to have a lot of resources available for learning it.
I would really love if there was a nice implementation of Prolog in Clojure that let me use Clojure goodness like S-expressions and macros and drop-down to low-level Java when needed. I think there are a few JVM implementations of Prolog, but I’m not sure what state they’re in. I spent a little time playing around with things and got a basic Prolog implementation working in Clojure… it’s honestly not rocket science. If I get more free time I might try to play around with this more and see if I can turn it into something actually useful. Maybe I can use it for AoC 2025.
I know back in the day a lot of the Common Lisp people played around with implementing Prolog in CL – I even have a few books on my bookshelf that walk thru how to write an implementation. Time to do some reading.

👇 Like and subscribe below! 👇

I gave an epic talk at Clojure/conj 2024

2025-02-04T18:35:00+00:00

What’s up everyone. I gave an epic talk at Clojure/conj 2024 a few months back, you can watch it on YouTube here: “Getting 50,000 Companies on Board with Clojure” by Cam Saul. Enjoi!

👇 Like and subscribe below! 👇

Debugging Custom Kondo Hooks and Clojure LSP Jump-to-Definition Functionality

2024-09-03T15:13:00+00:00

At Metabase I wrote a macro called defendpoint for defining (you guessed it!) REST API endpoints with a typical usage that looks something like

(api/defendpoint POST "/:id/copy"
  "Copy a `Card`, with the new name 'Copy of _name_'"
  [id]
  ;; Malli schemas defined after the argslist are used for validation
  {id [:maybe ms/PositiveInt]}
  (let [orig-card (api/read-check Card id)
        new-name  (str (trs "Copy of ") (:name orig-card))
        new-card  (assoc orig-card :name new-name)]
    (-> (card/create-card! new-card @api/*current-user*)
        (assoc :last-edit-info (last-edit/edit-information-for-user @api/*current-user*)))))

I was spending a lot of time scratching my head wondering why Clojure LSP jump-to-definition functionality wasn’t working as expected inside defendpoint. I wrote a custom clj-kondo hook for it, but moving my cursor to something like api/read-check and doing something M-x lsp-find-definition would just take me to the start of the defendpoint form rather than what I was expecting.

Debugging this issue was pretty hard – while there was clearly something wrong with my custom Kondo hook, but I couldn’t figure out what it was.

In the end I finally managed to figure out how to debug these sorts of things so I’m writing down my steps in case anyone else runs into the same problem.

Add Clojure LSP as a dependency

I added this to my deps.edn:

{:aliases
 {:lsp
  {:extra-deps {com.github.clojure-lsp/clojure-lsp {:mvn/version "2024.08.05-18.16.00"}
                dev.weavejester/cljfmt             {:git/url "https://github.com/weavejester/cljfmt"
                                                    :git/sha "434408f6909924f524c8027b37422d32bb49622d"}}}}}

Note that at the time of this writing because of clojure-lsp/clojure-lsp#1867 I had to add a fake dependency for cljfmt to make this work as well – I just used the latest Git SHA at the time of this writing.

Launch REPL with the `:lsp` alias

In Emacs I used

C-u M-x cider-jack-in

to let me edit the command it used to launch the nREPL to add the :lsp alias. Note that I launched the REPL from the same project I was debugging.

Debug from the REPL

This is the code I came up with while poking thru Kondo and LSP internals to debug things:

(require 'clj-kondo.hooks-api
         'clojure-lsp.kondo
         'clojure-lsp.queries
         'clojure.java.io)

(defn find-element [path line col]
  (binding [clj-kondo.hooks-api/*reload* true]
    (letfn [(analysis [path]
              (let [db*              (atom {})
                    config           nil
                    file-analyzed-fn identity]
                (clojure-lsp.kondo/run-kondo-on-paths! [path] db* config file-analyzed-fn)))
            (db-with-analysis [path]
              (clojure-lsp.kondo/db-with-analysis {} (analysis path)))]
      (let [db  (db-with-analysis path)
            uri (str "file://" (.getAbsolutePath (clojure.java.io/file path)))]
        (clojure-lsp.queries/find-element-under-cursor db uri line col)))))

Binding clj-kondo.hooks-api/*reload* is important so Clojure LSP will reload your Kondo config (e.g., custom hooks) after you make changes to it.

The basic idea here is to run Kondo on a specific file path and then pass that analysis along to LSP which can find the element at a given position in a file based on that analysis.

All you need is a file name to debug and a line number and column number that positions the cursor over the symbol you want to debug.

Example usage:

(let [path "src/metabase/api/card_2.clj"
      line 17
      col  21]
  (some-> (find-element path line col)
          (select-keys [:name :alias :from :from-var :to])))
;;=> {:name read-check, :alias api, :from metabase.api.card-2, :from-var POST_:id_copy, :to metabase.api.common}

If your Kondo hook is broken then this will either return nothing or return the wrong information. If the information looks something like the example result above then things should work as expected.

What was broken?

My original version of the hook returned when I used the find-element code above.

{:name POST_:id_copy}

My hook basically looked like this (simplified a bit):

(ns hooks.metabase.api.common
  (:require
   [clj-kondo.hooks-api :as api]
   [clojure.string :as str])

(defn route-fn-name
  "route fn hook"
  [method route]
  (let [route (if (vector? route) (first route) route)]
    (-> (str (name method) route)
        (str/replace #"/" "_")
        symbol)))

;;; Original custom Kondo hook -- BROKEN!!
(defn defendpoint
  [arg]
  (letfn [(update-defendpoint [node]
            (let [[_defendpoint method route & body] (:children node)]
              (api/list-node
               (list
                (api/token-node 'do)
                (api/token-node (symbol "compojure.core" (str (api/sexpr method))))
                (-> (api/list-node
                     (list*
                      (api/token-node 'clojure.core/defn)
                      (api/token-node (route-fn-name (api/sexpr method) (api/sexpr route)))
                      body))
                    (with-meta (meta node)))))))]
    (update arg :node update-defendpoint)))

which means the example at the beginning would be transformed in a node representing a form like this:

;;; Output of custom Kondo hook
'(do
   compojure.core/POST
   (clojure.core/defn POST_:id_copy
     "Copy a `Card`, with the new name 'Copy of _name_'"
     [id]
     {id [:maybe ms/PositiveInt]}
     (let [orig-card (api/read-check Card id)
           new-name (str (trs "Copy of ") (:name orig-card))
           new-card (assoc orig-card :name new-name)]
       (-> (card/create-card! new-card @api/*current-user*)
           (assoc :last-edit-info (last-edit/edit-information-for-user @api/*current-user*))))))

We’re generating a do form here so we can capture the usage of compojure.core/POST so Kondo doesn’t think it’s unused if it’s in our :require form in the ns declaration.

Side note: this is mentioned in the Kondo docs but you can verify your Kondo hook output looks correct on the surface level by Kondo itself as a dep and using clj-kondo.hooks-api/parse-string and clj-kondo.hooks-api/sexpr:

;;; debugging custom Kondo hooks
(def form
  '(api/defendpoint POST "/:id/copy"
     "Copy a `Card`, with the new name 'Copy of _name_'"
     [id]
     {id [:maybe ms/PositiveInt]}
     (let [orig-card (api/read-check Card id)
           new-name  (str (trs "Copy of ") (:name orig-card))
           new-card  (assoc orig-card :name new-name)]
       (-> (card/create-card! new-card @api/*current-user*)
           (assoc :last-edit-info (last-edit/edit-information-for-user @api/*current-user*))))))

(-> {:node (-> form
               pr-str
               api/parse-string)}
    defendpoint
    :node
    api/sexpr)

I’ve actually started putting stuff like these in tests to make sure I don’t break my dozens of custom Kondo hooks.

Anyways “looks correct at a surface level” is key here. Because while this output looks correct, what you do with the metadata (specifically stuff like start and end line and column numbers) from the original nodes is actually super important to Clojure LSP. (They’re important to Kondo too to make sure warnings pop up in the right place.)

If you want to see what this metadata looks like in your REPL you can

(set! *print-meta* true)

or just bind *print-meta* and use something like clojure.pprint/pprint e.g.

(binding [*print-meta* true]
  (-> {:node (-> form
                 pr-str
                 api/parse-string)}
      hooks.metabase.api.common/defendpoint
      :node
      api/sexpr
      clojure.pprint/pprint))

The output is actually super noisy so I’ll leave it out here.

The fixed version of the hook actually has the exact same output aside from some changes to metadata.

Here’s the fixed version of the hook above:

;;; custom Kondo hook -- FIXED
(defn defendpoint
  [arg]
  (letfn [(update-defendpoint [node]
            (let [[_defendpoint method route & body] (:children node)]
              (-> (api/list-node
                   (list
                    (api/token-node 'do)
                    (-> (api/token-node (symbol "compojure.core" (str (api/sexpr method))))
                        (with-meta (meta method)))
                    (api/list-node
                     (list*
                      (api/token-node 'clojure.core/defn)
                      (-> (api/token-node (route-fn-name (api/sexpr method) (api/sexpr route)))
                          (with-meta (meta route)))
                      body))))
                  (with-meta (meta node)))))]
    (update arg :node update-defendpoint)))

The important change here was to give the generated function name symbol POST_:id_copy the metadata (line and column start and end numbers) from the original "/:id/copy" string. It seems like without knowing where a function name is defined in the text file (or rather, where it is restricted to) LSP for whatever reason was assuming the entire defendpoint form served as a the function name which meant any attempt to find the definition under the cursor anywhere in that form would assume you were trying jump to the function definition itself.

Other changes I made which didn’t seem to fix anything but seem more correct are:

compojure.core/POST now gets the metadata (line and column start and end numbers) from the original POST symbol (the method parameter) from the original node
The overall new top-level list node gets the metadata from the original top-level node rather than putting it on the generated defn node. Not sure this makes a huge difference TBH.

Conclusion

Custom Kondo hooks are powerful and worth the time it takes to wrap your head around rewrite-clj. Debugging them is tricky, and debugging them when used by LSP is even trickier. I haven’t seen it documented anywhere before, but it is possible. I hope you find this useful!

👇 Like and subscribe below! 👇

Humane Are Just Dropped!

2022-09-01T20:40:00+00:00

Hello, world. It’s me, your friendly neighborhood #cam.

You may have been concerned about my whereabouts for the last 2 years, but I was doing the HUMANE thing and working on a new library for you all… In my free time while I’m not being a full time #birddad I have been working on a fun new project that will help you write tests. Let me break it down.

I’ve been using some version of this or another for a while now; I thought it was finally time to spin it out into a separate library, in case anyone else found it useful. clojure.test/are is great for writing lots of assertions quickly, but it has two big problems that prevent me from using it everywhere:

Failing assertions give no indication as to which set of arguments failed if you’re using anything that pretty prints test output, such as Humane Test Output, CIDER, or eftest
are lets you shoot yourself in the foot by writing expressions that include is or testing, and wraps them in another is without complaining

You can learn more about it on GitHub.

I hope everyone finds this as useful as I hope I will.

Peace, love, and bird seed.

xoxo CamSaul

My Clojure/north 2020 talk about Methodical is now available on YouTube

2020-08-19T20:07:00+00:00

I gave a talk about a library I wrote called Methodical at Clojure/north 2020. Methodical ports the “generic method” parts of the Common Lisp Object System (CLOS) to Clojure in a Clojurey way. As far as I can tell, it’s the best/most complete port of the CLOS to Clojure out there, even though it only ports a subset of CLOS.

This talk is now available to watch on YouTube:

If you enjoyed the talk and have money burning a hole in your pocket, consider buying me a cup of coffee at GitHub Sponsors.

Intro to Emacs Lisp: Adding Live Previews when Editing Markdown Files (Part 3 of 3)

2020-06-10T02:12:00+00:00

This post is part three of a series of 3 posts. View the other parts:

In this post, we’ll make the command more useful by:

Having it prompt us for a file to preview when running interactively.
Adding documentation
Having Emacs run it automatically whenever we save a Markdown file

We’ll discuss the following Emacs Lisp concepts:

Optional arguments
interactive code characters
progn forms
if forms
Docstrings
Hooks
Buffer-local variables
What it means that Emacs Lisp is a Lisp-2

Prompting for a file to preview

Here we’ve broken out cam/preview-markdown a bit further.

This version of the command works the same as before when run programmatically, but when executed interactively (e.g. via M-x) it will prompt you for the file to preview, complete with autocomplete; it defaults to the current file.

(defun cam/-scroll-percentage ()
  (/ (float (line-number-at-pos (window-start)))
     (float (line-number-at-pos (point-max)))))

(defun cam/-set-window-start-to-percentage (scroll-percentage)
  (goto-char (point-min))
  (let ((target-line-number (truncate (* (line-number-at-pos (point-max)) scroll-percentage))))
    (forward-line (1- target-line-number)))
  (set-window-start nil (point)))

(defun cam/-render-markdown-preview-current-buffer ()
  (message "Rendering Markdown preview of %s" buffer-file-name)
  (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*")
  (switch-to-buffer-other-window "*Preview Markdown Output*")
  (let ((document (libxml-parse-html-region (point) (point-max))))
    (erase-buffer)
    (shr-insert-document `(base ((href . ,url)) ,document))
    (setq buffer-read-only t)))

(defun cam/-preview-markdown-file (filename)
  (save-selected-window
    (find-file filename)
    (let ((url (concat "file://" filename))
          (scroll-percentage (cam/-scroll-percentage)))
      (cam/-render-markdown-preview-current-buffer)
      (cam/-set-window-start-to-percentage scroll-percentage))))

(defun cam/preview-markdown (&optional filename)
  "Render a markdown preview of FILENAME (by default, the current file) to HTML and display it with `shr-insert-document'."
  (interactive "fFile: ")
  (if filename
      (progn
        (cam/-preview-markdown-file filename)
        (switch-to-buffer (current-buffer)))
    (cam/-preview-markdown-file buffer-file-name)))

There are a couple of new concepts to introduce here:

&optional arguments
arguments to the (interactive) declaration
progn
Emacs Lisp docstrings

Optional arguments

In function definitions, &optional is used to denote optional positional arguments. If an optional argument is not passed, its value will be nil in the body of the function:

(defun my-filename (&optional filename)
  filename)

(my-filename "x.txt") ; -> "x.txt"
(my-filename)         ; -> nil

(defun cam/preview-markdown (&optional filename)
  ...)

we are making the filename argument optional, because we’d like to be able to call this function with a specific file to preview, but default to the current file when it is called with no argument (such as when it is called as part of a hook, as discussed below).

Specifying a prompt in the `interactive` declaration

The interactive declaration can optionally take an argument that tells Emacs how to prompt for values to use as the command’s arguments:

(interactive "fFile: ")

The f code character tells Emacs to prompt for an existing filename, defaulting to the name of the file in the current buffer. The rest of the string is the text of the prompt to show the user in the minibuffer ("File: "). Note that we have to include the space after the prompt ourselves.

The string name of the file the user chooses will be passed in as the filename argument. When the function is invoked programmatically, Emacs will not prompt the user for a value of filename.

`if` forms

In Emacs Lisp, if forms have the syntax

(if condition
    then-form
  else-forms...)

For example:

;; t means true and nil means null/false
(defun my-> (x y)
  (if (> x y)
      t
    nil))

(my-> 3 2)
;; -> t

The if form in the example is roughly equivalent to this Algol-style if form:

if (x > y) {
  true;
} else {
  false;
}

`progn` forms

progn can be used to execute multiple statements as a single form. (If you’re familiar with Clojure, this is the classic Lisp equivalent of do). Each form is evaluated in order, and the result of a progn form is the result of the last form contained by it:

(progn
  (message "We are adding some numbers.")
  (+ 3 4))
;; -> 7

Because only the value of the last form is returned, forms other than the last are usually executed for side effects.

Understanding the updated `cam/preview-markdown`

Now that we’ve discussed the new concepts, Let’s take a deeper look at our simplified cam/preview-markdown function:

(defun cam/preview-markdown (&optional filename)
  "Render a markdown preview of FILENAME (by default, the current file) to HTML and display it with `shr-insert-document'."
  (interactive "fFile: ")
  (if filename
      (progn
        (cam/-preview-markdown-file filename)
        (switch-to-buffer (current-buffer)))
    (cam/-preview-markdown-file buffer-file-name)))

If a filename argument was passed:
1. Call cam/-preview-markdown-file with that filename
2. Switch back to the current buffer (save-selected-window, called in cam/-preview-markdown-file, also restores the current buffer, but doesn’t necessarily bring it to the top).
If no filename argument was passed:
1. Call cam/-preview-markdown-file with the filename of the current buffer.

Opening a file/switching to existing buffer for a file

In cam/-preview-markdown-file we’ve added a call to find-file.

(find-file filename)

switches to a buffer containing filename. In cases where we’re already looking at that file, find-file doesn’t change anything, so this code continues to work normally when rendering the file in the current buffer. For other files, it will switch to a buffer for that file, creating a buffer (opening the file) if needed.

Adding a docsting

(defun cam/preview-markdown (&optional filename)
  "Render a markdown preview of FILENAME (by default, the current file) to HTML and display it with `shr-insert-document'."
  (interactive "fFile: ")
  ...)

Emacs Lisp docstrings come after the argument list but before the interactive declaration, if there is one. Emacs Lisp docstrings conventionally mention arguments in all capital letters (e.g. FILENAME). When viewing the documentation for a function, arguments mentioned this way are highlighted automatically.

You can add hyperlinks to other Emacs Lisp functions or variables using the

`symbol-name'

syntax. These days you can also use curved single quotes instead, but figuring out how to type them involves too much effort, so I stick with the backtick-single-quote convention, which is the one you’ll encounter most commonly in the wild.

Try it yourself! C-h f cam/preview-markdown to view the documentation for the function cam/preview-markdown.

This page of the Emacs Lisp documentation has a very good explanation of docstrings for further reading.

Adding an `after-save-hook`

Now all that’s left to do is telling Emacs to automatically run our command whenever we save a Markdown file.

(add-hook 'markdown-mode-hook
  (lambda ()
    (add-hook 'after-save-hook #'cam/preview-markdown nil t)))

Whenever we open a Markdown file, Emacs will set the major mode to markdown-mode. Major modes and most minor modes have hooks that run whenever the mode is entered. When we enter markdown-mode, Emacs will run any functions in markdown-mode-hook.

To markdown-mode-hook we add a lambda (anonymous function) that itself adds the command #'cam/preview-markdown to after-save-hook. Any functions in after-save-hook will run after a file is saved.

A hook is just a variable that contains a list of functions that get ran at a specific point in time. markdown-mode-hook is a list of functions to run when entering markdown-mode and after-save-hook is a list of functions to run after saving a file.

Add-hook and buffer-local variables

add-hook adds a function to a hook (which, again, is just a list), creating the variable if it doesn’t already exist. add-hook has two optional args, append (default nil) and local (default nil). append tells it to add the function at the end of the list, meaning it gets ran last. In some cases, it is preferable to run a certain function after others in the hook have ran. In our case, it doesn’t really matter if our function runs before or after others, so we’ll pass the default value of nil. local tells it to add it to the buffer-local version of the hook rather than the global version. We’ll explore the difference more in the future, but for now suffice to know that variables can have global values as well as values specific to a buffer. Buffer-local values overshadow global values.

(add-hook 'after-save-hook #'cam/preview-markdown)

without the optional args would add the function #'cam/preview-markdown to the global after-save-hook, which means it would run after saving any file, which is not what we want. By adding the local option, the function is only added to the hook for the current buffer, meaning it only runs for the current buffer.

To sum it up: markdown-mode-hook gets ran once for every new Markdown file we open, and we use that to add cam/preview-markdown to the after-save-hook for the newly created buffer. Files that aren’t opened in markdown-mode aren’t affected at all.

Lisp-1s and Lisp-2s

Emacs Lisp is a Lisp-2, which means that variables and functions live in separate “namespaces”. For example, length could refer to both a variable named length and a function named length.

;; Emacs Lisp
(length '(1 2 3))
;; -> 3

;; the length variable doesn't overshadow the length function
(let ((length 4))
  (length '(1 2 3)))
;; -> 3

Contrast this to Clojure, a Lisp-1, where variables and functions share a “namespace”:

;; Clojure
;; count is the Clojure equivalent of length
(count '(1 2 3))
;; -> 3

;; let-bound count overshadows *any* usage of the symbol
(let [count 4]
  (count '(1 2 3)))
;; -> [100 line stacktrace: Integer cannot be invoked as a function]

Using functions passed as arguments is much simpler in Lisp-1s, however:

;; Clojure
(defn call-f [f]
  (f 100))

(call-f (fn [n] (inc n)) 100)
;; -> 101

With Lisp-2s, you have to use funcall to call a function bound to a variable:

;; Emacs Lisp
;; the symbol f refers only to variable, so to use it as a function contained in
;; f, you have to use funcall
(defun call-f (f)
  (funcall f 100))

(call-f (lambda (n) (1+ n)) 100)
;; -> 101

There are pros and cons to both Lisp-1s and Lisp-2s. Lisp-1s lend themselves more elegantly to passing around functions since you don’t need to use special forms like funcall. However you have to be much more careful not to unintentionally overshadow functions in Lisp-1s, which usually means intentionally misspelling function parameter names. You’ll often see nonsense like this in Lisp-1s:

;; clojure
;; so as to not overshadow the "type" function, we have to give our type
;; parameter a different name, such as "typ"
(defn type=
  "True if the type of `x` is equal to `typ`."
  [x typ]
  (= (type x) typ))

There are other differences to explore in the a future post, but let’s get back to tweaking our command!

Quoting function names

Attempting to evaluate cam/preview-markdown will result in an error:

cam/preview-markdown
*** Eval error ***  Symbol’s value as variable is void: cam/preview-markdown

Instead, we can quote the symbol name so the reader doesn’t attempt to evaluate it. Emacs offers an alternative syntax for quoting symbols that refer to functions:

#'cam/preview-markdown

Just as 'x is shorthand for (quote x), #'x is shorthand for (function x).

In many cases, using quote for function names will still work correctly:

(add-hook 'after-save-hook 'cam/preview-markdown)

But function is preferred over quote for symbols that name functions for a couple of reasons: it’s clearer and more explicit, and the compiler is better able to optimize function forms.

Final Thoughts

Complete source for the final version can be found at this GitHub Gist. Please feel free to leave comments or suggestions there, or on this Reddit thread. If there’s enough positive feedback from these posts, I’ll be sure to add more!

If you enjoyed these posts and have money burning a hole in your pocket, consider buying me a cup of coffee at GitHub Sponsors.

Intro to Emacs Lisp: Adding Live Previews when Editing Markdown Files (Part 2 of 3)

2020-06-09T23:12:00+00:00

This post is part two of a series of 3 posts. View the other parts:

In this post we’ll make a series of improvements to the cam/preview-markdown command, including:

Rendering in a way that images and other files relative to the directory containing the Markdown source work correctly
Preserving the current window
Making *Preview Markdown Output* read-only
Scrolling the preview so it’s at approximately the same location as the Markdown buffer

We’ll also look at the following Emacs Lisp concepts:

Lists, cons cells, and association lists (briefly)
Backquote-splicing
Macros

Viewing Source

A side note about how I figured out which functions to use to parse and render HTML: SHR isn’t terribly well documented. I wasn’t sure how to get it to do what I wanted until I poked around the source for EWW (the Emacs Web “Wowser”).

I strongly recommend you install a package like elisp-slime-nav that makes it easy to jump to the source code for functions. When editing Emacs Lisp, elisp-slime-nav binds M-. to jump to the source of a function or variable; if your cursor is over a symbol, it will jump straight to its definition.

Improving the `preview-markdown` command

Rendering relative to the current directory

Now that the meat and potatoes of our command are finished it’s time to polish it up and fix some of the annoyances. One issue with the preview is that links and images with paths relative to the markdown document don’t work correctly in our preview. We can fix this by prepending the parsed document with a another list that contains information as the base URL.

(defun cam/preview-markdown ()
  (interactive)
  (let ((filename buffer-file-name))
    (message "Rendering Markdown preview of %s" filename)
    (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*")
    (save-selected-window
      (switch-to-buffer-other-window "*Preview Markdown Output*")
      (let ((document (libxml-parse-html-region (point) (point-max)))
            ;; [1] Create the base URL
            (url (concat "file://" filename)))
        (erase-buffer)
        ;; [2] wrap `document` in a <base> element
        (shr-insert-document `(base ((href . ,url)) ,document))
        (goto-char (point-min))))))

First, we need to create a file:// URL that SHR can use as the base URL for images or other elements with a relative path. If our original file is /home/cam/README.markdown, our URL should be file:///home/cam/README.markdown. We can use concat to concatenate multiple arguments together into a single string:

(concat "file://" "/home/cam/README.markdown")
;; -> "file:///home/cam/README.markdown"

Next, we need to wrap document in a list that includes the base URL.

document itself is simply an Emacs Lisp representation of an HTML/XML data structure. It has the basic format

(element-name attributes child-elements...)

So for example HTML like

<p>This is <span class="important">IMPORTANT</span></p>

would be represented as

(p nil "This is " (span ((class . "important")) "IMPORTANT"))

Try it yourself. M-: and then type or paste the following form

(shr-insert-document '(p nil "This is " (span ((class . "important")) "IMPORTANT")))

to render it and insert it into the current buffer.

Quoting

Quoting a form tells Emacs Lisp not evaluate it, and to instead return it as-is:

(+ 1 2)
;; -> 3

'(+ 1 2)
;; -> (+ 1 2)

buffer-file-name
;; -> "/home/cam/README.md"

'buffer-file-name
;; -> buffer-file-name

This is especially useful for passing in lists, which would otherwise get interpreted as function calls, or symbols, which would otherwise get interpreted as function names or variables. The ' syntax is shorthand for the (quote) form:

(quote (+ 1 2))
;; -> (+ 1 2)

Understanding Lists, Cons Cells, and Association Lists

attributes are an association list, which is a classic Lisp way to store a map/dictionary as a list of key/value pairs. A deep dive into association lists is outside the scope of this article, but we can go over them briefly. But first, we should take a step back and look at how lists work in Lisp:

A Lisp list is composed of cons cells, which is Lisp-speak for linked list nodes. A cons cell has two pointers: one to an item, and one to the next cons cell. If a cons cell holds the last item in a list, its pointer to the next cell is nil. A cons cell is written with the dotted pair syntax (x . y):

'(1 . nil)
;; -> (1)

cons can also be used to create cons cells and is equivalent to the dotted pair syntax.

(cons 1 nil)
;; -> (1)

Very important! Note that (1 . nil) or (cons 1 nil) are the same thing as the list (1). A list with a single item is just a cons cell whose first pointer is to the item, and whose second pointer is nil.

You can chain together dotted pair forms or calls to cons to create lists if you like doing things the hard way.

'(1 . (2 . (3 . nil)))
;; -> (1 2 3)

(cons 1 (cons 2 (cons 3 nil)))
;; -> (1 2 3)

One more thing to note: to add an item to the front of an existing list, you can use cons.

(cons 1 '(2 3 4))
;; -> (1 2 3 4)

The newly created cons cell’s item and next-cons-cell pointers are to 1 and the list (2 3 4) respectively. This is not a destructive operation – the original list of (2 3 4) does not need to be modified in any way. Prepending items to a list via cons is a common operation in classic Lisp languages (not so much in Clojure).

Other uses of cons cells

In some cases, both pointers in a cons cell are used to point to items. Association lists are one such example: an association list is a list of cons cells, where each cell represents a (key . value) pair.

So a JSON dictionary like

{"a": 100, "b": 200}

could be translated to an association list like

((a . 100) (b . 200))

Wrapping our HTML `document`

With that big explanation out of the way, we need to take our <html> element stored in document and wrap it in a <base> form, so it looks like this:

<base href="file:///home/cam/README.markdown">
  <html>...</html>
</base>

In the Lisp representation, this means we need the list

(base ((href . "file:///home/cam/README.markdown")) document)

There are a few ways to create such a list. None are particularly convenient, because you need to quote symbols base and href so the reader doesn’t try to evaluate them, but you can’t quote the entire form, because you need to splice in the URL we created, as well as the value of document.

(list 'base (list (cons 'href url)) document)

is one option:

(let ((document "my doc")
      (url "my url"))
  (list 'base (list (cons 'href url)) document))
;; -> (base ((href . "my url")) "my doc")

Backquote-splicing

Luckily, we can use backtick or backquote-splicing, which is more fun. As mentioned before, we want something like (base ((href . url)) document), but quoting the entire form e.g. '(base ((href . url)) document) won’t work because then url or document would be quoted as well and thus wouldn’t get evaluated. Backquote splicing lets you quote things but selectively disable quoting:

(let ((document "my doc")
      (url "my url"))
  `(base ((href . ,url)) ,document))
;;-> (base ((href . "my url")) "my doc")

The backtick starts backquote-splicing, which you can think of as like switching on a “quote mode” that lasts for the entire form; commas nested temporarily inside disable quote mode for the next form. (For those familiar with Clojure: , is the traditional Lisp equivalent of Clojure’s ~.)

You can nest backticks forms as well – in other words, you can turn “quote mode” back on and off inside nested forms as you see fit:

(let ((x 1) (y 2))
  `(,x ,(cons y `(,x 0))))
;; -> (1 (2 1 0))

Splicing items into the parent form with `,@`

There’s another splicing operator: ,@, which lets you splice the items in a list directly into their parent form:

;; splice the list directly
(let ((my-list '(1 2 3)))
  `(,my-list 4 5))
;; -> ((1 2 3) 4 5)

;; splice the items in the list into the parent list
(let ((my-list '(1 2 3)))
  `(,@my-list 4 5))
;; -> (1 2 3 4 5)

Backquote splicing is essential when writing macros, so we’ll revisit it in a future post.

Preserving the current window

Right now, our command switches to the window containing the rendered HTML every time it runs, so you have to switch back to the window with the Markdown file to continue editing it. We eventually want this command to run automatically whenever we save a file, and this behavior will get pretty annoying.

We can use save-selected-window to restore the current window after evaluating a series of forms:

(defun cam/preview-markdown ()
  (interactive)
  (let ((filename buffer-file-name))
    (message "Rendering Markdown preview of %s" filename)
    (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*")
    (save-selected-window
      (switch-to-buffer-other-window "*Preview Markdown Output*")
      (let ((document (libxml-parse-html-region (point) (point-max)))
            (url (concat "file://" filename)))
        (erase-buffer)
        (shr-insert-document `(base ((href . ,url)) ,document))
        (goto-char (point-min))))))

To recap: switch-to-buffer-other-window switches to a different window and brings up the *Preview Markdown Output* buffer, and shr-insert-document renders the GUI widgets into the current buffer. After that, we’re free to switch back to the original window. save-selected-window will preserve the selected window and current buffer before the forms it wraps are executed; the forms are then executed normally, and save-selected-widnow restores the selected window and current buffer.

Making `Preview Markdown Output` read-only

Since *Preview Markdown Output* contains rendered HTML widgets derived from a separate source file, it’s probably less confusing if we make *Preview Markdown Output* a read-only buffer. Whether a buffer is read-only is determined by the variable buffer-read-only, which automatically becomes buffer-local if set. This means when you set this variable, you are setting it only for the current buffer; the current buffer’s value with overshadow the global value.

shell-command-on-region will “helpfully” clear the read-only flag when it writes its output to *Preview Markdown Output*, so we don’t have to worry about clearing it out if we end up reusing the buffer after running the command a second time.

(defun cam/preview-markdown ()
  (interactive)
  (let ((filename buffer-file-name))
    (message "Rendering Markdown preview of %s" filename)
    (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*")
    (save-selected-window
      (switch-to-buffer-other-window "*Preview Markdown Output*")
      (let ((document (libxml-parse-html-region (point) (point-max)))
            (url (concat "file://" filename)))
        (erase-buffer)
        (shr-insert-document `(base ((href . ,url)) ,document))
        (goto-char (point-min))
        ;; [1] Make buffer read-only
        (setq buffer-read-only t)))))

To change the value of a variable, you can use setq. It’s the equivalent of = assignment in Algol-style languages:

// javascript
x = 10;

;; emacs-lisp
(setq x 10)

The q in setq stands for quote. setq in the example above is actually equivalent to:

(set 'x 10)

'x is quoted because we’re setting the value named by the symbol x. We want

x = 10

rather than

[current value of x] = 10

which probably won’t make sense.

(set 'x 100)
;; -> 100
(set x 200)
*** Eval error ***  Wrong type argument: symbolp, 100

This error means that it expected a symbol (i.e., something that satisfied the symbolp predicate function) but got 100. 100 = 200 doesn’t really make sense.

Macros 101

So how does setq work if the variable name needs to be quoted? With macros, you can take the arguments you’re passed, before they are evaluated, and generate whatever code you want. This new code is used in place of the macro form.

setq could be implemented as a macro that quotes the variable name symbol. (setq is actually a built-in special form, but you can implement similar behavior with a macro). Here’s a simple setq-style macro:

(defmacro my-setq (symbol value)
  `(set ',symbol ,value))

(my-setq x 200)
;; -> 200

The important thing to know about Lisp macros is that they’re just functions that take arguments and return a new Lisp form to be used in their place. Macros and regular functions essentially differ only in when they’re evaluated, and what’s done with the result, but everything else works the same. Macros can call other functions, and more complex macros often have parts of their implementation split out into separate functions, defined with defun like any other function.

Evaluating a macro form and replacing it with the result is called macro “expansion”. When a form is evaluated, all macros are expanded first (top-level macro forms are expanded before ones nested inside them), and only then are variables and function calls evaluated.

(set 'x 100) is just a list with the elements set, 'x, and 100, so if we have a macro generate that sort of list for us our dreams will come true. In the example above, we use backquote splicing to construct our list.

Macroexpansion

You can use macroexpand to expand a macro form to see the Lisp form it expands to:

(macroexpand '(my-setq my-variable 200))
;; -> (set 'my-variable 200)

Note that you need to quote the form passed so it doesn’t get evaluated before macroexpand sees it.

Macros can of course expand to other macros, and Emacs Lisp will continue expanding things until there are no more macros to expand. To expand a macro form once instead of completely, you can use macroexpand-1 instead of macroexpand. In the example above, the result of my-setq is already fully expanded, so both functions give you the same result.

When writing macros, some tools you might you might find the built-in pp-macroexpand-last-sexp command useful; it pretty-prints the results of a macro expansion. I’m also a big fan of the macrostep package, which lets you view the expansions of macros directly in your source code.

Scrolling to approximate location in preview

It’s a little annoying to work on a giant Markdown file like this blog post and have the preview always scroll to the very top. Why not just have it scroll approximately the same position we’re currently looking at?

Here’s the quick and dirty solution we’ll implement:

When first running our command, while the Markdown buffer is still current, record how far we’ve scrolled thru the document (e.g. 40%).
After we’ve rendered the HTML as GUI widgets, scroll to the line that is approximately the same distance thru the document.

This will work even if the rendered output has a lot more or less lines than the Markdown source. For example, if our Markdown source file is 1000 lines, and the rendered output 800 lines, and the top of the window shows line 400, we’d record a scroll-percentage of 0.4 (40% scrolled); in the rendered output, we’ll scroll to line 320 (800 * 0.4).

This isn’t perfect, since there isn’t a 1:1 translation between Markdown text and rendered HTML lines, but in my experience it works well enough that I haven’t bothered creating a more sophisticated version.

For readability, I broke the command we’ve been working on out into a few separate functions. Since Emacs Lisp doesn’t have encapsulation features like private functions, functions intended to be private are often given a name with a dash after the “namespace” part of the name, such as package--function or package/-function. Thus I named our “private” functions with the pattern cam/-function-name.

(defun cam/-scroll-percentage ()
  (/ (float (line-number-at-pos (window-start)))
     (float (line-number-at-pos (point-max)))))

(defun cam/-set-window-start-to-percentage (scroll-percentage)
  ;; Move to the beginning of the rendered output buffer
  (goto-char (point-min))
  ;; target line number is floor(total-number-of lines * scroll-percentage)
  (let ((target-line-number (truncate (* (line-number-at-pos (point-max)) scroll-percentage))))
    ;; move to target line number
    (forward-line (1- target-line-number)))
  ;; now scroll the window so the line in question is at the top
  (set-window-start nil (point)))

(defun cam/preview-markdown ()
  (interactive)
  (let ((url (concat "file://" buffer-file-name))
        ;; record how far thru the Markdown source file we've scrolled
        (scroll-percentage (cam/-scroll-percentage)))
    (message "Rendering Markdown preview of %s" buffer-file-name)
    (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*")
    (save-selected-window
      (switch-to-buffer-other-window "*Preview Markdown Output*")
      (let ((document (libxml-parse-html-region (point) (point-max))))
        (erase-buffer)
        (shr-insert-document `(base ((href . ,url)) ,document))
        (cam/-set-window-start-to-percentage scroll-percentage)
        (setq buffer-read-only t)))))

This is pretty straightforward. To calculate the scroll percentage, we:

Use (window-start) to get the position of the first character visible in the current window
Use line-number-at-pos to convert the position to a line number.
Calculate the total number of lines by getting the line number of the last character in the buffer by calling (line-number-at-pos (point-max)).
Divide the window start line by the last line to get a percentage. As in C, integer division is truncated. By first casting the integers to floating-point numbers with (float) we can use floating-point division instead; the final result is a number like 0.4.
```
(/ 40 100)
;; -> 0

(/ (float 40) (float 100))
;; -> 0.4
```

Once the output is rendered, to scroll to the line the desired line, we:

Move the cursor to the beginning of the buffer by calling (goto-char (point-min))
Calculate the total number of lines using (line-number-at-pos (point-max))
Multiply the total number of lines by scroll-percentage
Call truncate to convert the resulting floating point number to an integer. The result of this is our target-line-number.
```
(truncate 20.5432)
;; -> 20
```
Next, we move the cursor to our target line. Since we already moved to line 1 in step 1, we need to move forward by target-line-number - 1 lines. forward-line is used to move forward a number of lines. For example, if we want to move from line 1 to line 20, we can call (forward-line 19). 1- function returns its argument minus one.
```
(1- 20)
;; -> 19
```
Now that the cursor is at the correct position, we can scroll the window so the first line is the one with the cursor. (point) gets the current position of the cursor (the point) and set-window-start scrolls the window so it shows that position in the first line (“start”) of the window. Nice!

Next Steps

In Part 3, we’ll have the command prompt us for a file to preview when running interactively, and have Emacs run it automatically whenever we save a Markdown file. We’ll discuss optional arguments, interactive code characters, progn and if forms, hooks, buffer-local variables, and Lisp-2s.

Intro to Emacs Lisp: Adding Live Previews when Editing Markdown Files (Part 1 of 3)

2020-06-09T20:12:00+00:00

This post is part one of a series of 3 posts. View the other parts:

This is the first post in what I hope is an ongoing series of posts that show you how to get started with Emacs Lisp by writing practical Emacs commands and customizing Emacs in other ways. For me, one of the best things about Emacs is that you can customize it to work however you want, but learning how to do so is easier said than done, and while the official Emacs Lisp documentation is extremely detailed, it can be hard to approach.

I’m hoping I can demonstrate that Emacs Lisp is fun to write and not as hard as you might think. I’m going to assume experience using Emacs. For example, you should have a basic understanding of things like windows and buffers, and know how to read and type keyboard shortcuts written in the Emacs style (C-x C-s the world).

I will also assume experience programming in other languages, but not necessarily Lisp ones; however, you should still have a very basic understanding of how Lisp languages work, for example what the result of (+ 1 2) is. If you don’t, the GNU Intro to Emacs Lisp is a good starting point.

In this series of posts I’m going to show you step-by-step how to set write a command to render a preview of a Markdown file and open it directly in Emacs. Then I’ll show you how to have it run automatically whenever you save a Markdown file.

For this tutorial you’ll want to install pandoc which we will use for rendering Markdown files, Emacs 25 or newer, and the markdown-mode package.

Evaluating forms and `ielm`

Before we get started, I want to briefly mention how to follow along with the code in this post. For the functions we’re working on, you’ll probably want to add them to your ~/.emacs.d/init.el file. You can evaluate function definitions by moving the cursor to some point in the function definition and then doing C-M-x. You can also evaluate the form immediately preceding your cursor with the key binding C-x C-e. Give them a try!

A big part of Lisp development is using a REPL. ielm is the Emacs Lisp REPL. You can start it with M-x ielm and use it to evaluate Emacs Lisp forms. This is especially useful for evaluating the smaller bits of code like (+ 1 2) that eventually make it in to larger functions.

Writing a `preview-markdown` command

Without further ado, let’s start writing our command.

Defining a function

Let’s start off with an empty function declaration:

(defun cam/preview-markdown ())

If you’re completely new to Emacs Lisp, especially if you come from a Clojure background, you might think the cam/ portion of the name of the function has some sort of special significance. It doesn’t! Emacs Lisp doesn’t have the notion of separate namespaces that other languages have, so every function and variable name is global; like Objective-C, it’s a best-practice to prefix all function and variable names with something that makes it clear they’re part of a particular package. Usually you’ll see names prefixed with <package-name>-, for example helm-mark-ring, but <package-name>/ and <package-name>: are reasonably common conventions as well. The / character itself has no special significance in symbol names. I prefer prefixing all of my own functions and macros with cam/, so I’ll use that in the code in this post.

Making it a command

Now this function itself doesn’t do anything, and you can’t invoke it with M-x. Emacs Lisp functions can either be invoked programmatically (e.g. by other Emacs Lisp functions, or the ielm REPL, or via M-:) or interactively (via M-x or a keybinding). A function that be invoked interactively is known as a command.

To make a function a command, you add an interactive declaration, which always goes right after the argument list (or docstring, if you have one):

(defun cam/preview-markdown ()
  (interactive))

Now the function can be invoked interactively. Try it out! M-x cam/preview-markdown. Cool! But it still doesn’t do anything.

Logging Messages

Let’s start by having our command log a message. message is the Emacs Lisp equivalent of printf, so this is sort of a “Hello World” example. Exciting! We’ll add a call message in the function body:

(defun cam/preview-markdown ()
  (interactive)
  (message "Rendering Markdown preview of %s" buffer-file-name))

The variable buffer-file-name is bound to the string name of the file associated with the current buffer (i.e., the one from which you invoked the command). Create a markdown file and switch to it in Emacs, then run M-x cam/preview-markdown again.

You’ll see something like

Rendering Markdown preview of /home/cam/README.markdown

appear in the minibuffer, and it will be appended to the *Messages* buffer as well. When writing any code, it’s important to be able to debug it, and until we talk about using the Emacs Lisp debugger being able to print stuff out is a good first step.

Calling external commands

Our plan is to:

Take the markdown content in the current buffer and render it as HTML to a buffer named *Preview Markdown Output*
Parse the HTML in *Preview Markdown Output*
Clear *Preview Markdown Output* and then render the pretty GUI widgets like you saw in the GIF at the start of this post.

So let’s add a command to render the Markdown to HTML using the external pandoc command:

(defun cam/preview-markdown ()
  (interactive)
  (message "Rendering Markdown preview of %s" buffer-file-name)
  (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*"))

shell-command-on-region runs a command (pandoc -f gfm) on a region. The simplest explanation of a region is that it’s all the text between two positions in a buffer – if you think of the current buffer as one giant string, then a region would be a substring.

Since we want to render the entire buffer as HTML rather than just part of it, we want our region to be the entire buffer. The functions point-min and point-max get the minimum possible position and maximum possible position in the current buffer respectively; together those positions represent the region we’re operating on.

The final argument to shell-command-on-region, "*Preview Markdown Output*", tells it the name of the buffer to write the output of the shell command to (creating it if it doesn’t already exist). If the buffer already exists, shell-command-on-region will clear any existing contents of the buffer before writing the command output.

Emacs Lisp buffers used for internally or for other special purposes are conventionally given names with asterisks (affectionately known as earmuffs), but you could name the buffer anything you want.

Go ahead and give it a try in your Markdown file buffer, and you’ll see a buffer called *Preview Markdown Output* pop up with the HTML output.

Before moving on, let’s take a quick break to discuss some essentials that you’ll really want to know when writing Emacs Lisp.

Looking up documentation

To look up documentation for a function, you can use C-h f <name-of-function>. If you want to write Emacs Lisp, commit that keybinding to memory right away – you’ll be using it all the time. Try using it to learn more about one of the functions we’re using.

Similarly, C-h v opens documentation for a variable, and C-h k will tell you the function that gets run for a given keybinding. Write these down on a Post-It and stick it to your computer until they are burned into your brain.

Evaluating Emacs Lisp from the minibuffer with `M-:`

Another keybinding you’ll want to commit to memory is M-:, which allows you to evaluate an Emacs Lisp form from the minibuffer. The code is evaluated in the context of the current buffer. If you want to look at certain values of variables or test how certain code works when evaluated in a certain buffer, M-: is one of the easiest ways to do so.

Try it from you markdown buffer – M-: and then type or paste buffer-file-name and hit return. The output is visible in the minibuffer and *Messages*. Try it with a different buffer – you’ll see a different result.

Viewing the HTML

We can use the built-in Simple HTML Renderer (SHR) to render HTML directly in Emacs. Change our function to the following and give this a try:

(defun cam/preview-markdown ()
  (interactive)
  ;; [1] bind the value of buffer-file-name to local variable filename
  (let ((filename buffer-file-name))
    (message "Rendering Markdown preview of %s" filename)
    (shell-command-on-region (point-min) (point-max) "pandoc -f gfm" "*Preview Markdown Output*")
    ;; [2] switch to a different window and bring up the *Preview Markdown Output* buffer
    (switch-to-buffer-other-window "*Preview Markdown Output*")
    ;; [3] Parse the HTML source in *Preview Markdown Output*, and store it in document
    (let ((document (libxml-parse-html-region (point-min) (point-max))))
      ;; [4] Clear the contents of *Preview Markdown Output*
      (erase-buffer)
      ;; [5] Render the parsed HTML into *Preview Markdown Output*
      (shr-insert-document document)
      ;; [6] Move back to the beginning of the buffer
      (goto-char (point-min)))))

We’re doing a few new things here, and I’ll explain each in turn:

Binding local variables with `let`

(let ((filename buffer-file-name))
  ...)

Not all buffers are associated with files. *Preview Markdown Output* isn’t actually associated with a file, so buffer-file-name will be nil if we evaluate it with *Preview Markdown Output* as the current buffer. To work around this, we can evaluate buffer-file-name before we switch buffers and save it for later as the local variable filename. (let ...) is used to introduce local bindings.

;; x = 100
(let ((x 100))
  (+ x x))
;; -> 200

You might be wondering why you need two sets of parentheses to set x in the example above. The syntax for let is

(let bindings
  ...)

where bindings is a list of bindings of the form (symbol value), e.g. (x 100). This means you can pass in multiple binding forms to bind several local variables at once:

;; x, y = 100, 200
(let ((x 100)
      (y 200))
  (+ x y))
;; -> 300

Bindings can also be of the form symbol, which is just shorthand for (symbol nil):

;; x = nil
(let (x)
  x)
;; -> nil

This is sort of like declaring a variable in an Algol-family language, e.g.

// JavaScript
var x;

You’d probably do this with the intention of (maybe) giving that variable a value at some later point, e.g.

;; x = nil
(let (x)
  ;; x = 100
  (setq x 100)
  x)
;; -> 100

You can use setq to set the value of a variable. setq is discussed more at length in Part 2. Note that Emacs Lisp is in many ways a traditional imperative language. Emacs Lisp is a functional language in the sense that functions are first-class objects that can be passed around as parameters and returned. But it’s not a functional language in the same pure function/immutable value sense that Clojure or Haskell are.

There’s one more thing you should know about let: bindings are done independently of one another, as if they were done in parallel.

(let ((x 100)
      (y 200))
  (+ x y))

is equivalent to the JavaScript

// JavaScript
[x, y] = [100, 200]

So something like this doesn’t work:

(let ((x 100)
      (y (+ x x)))
  y)
;; Error: x is not bound yet

Many times you do want to refer to the results of one binding form in a subsequent one. In this case, you can use let*, which binds things sequentially rather than in parallel:

(let* ((x 100)
       (y (+ x x)))
  y)
;; -> 200

This is the equivalent of the JavaScript

// JavaScript
x = 100;
y = x + x;

Switching to a different window

(switch-to-buffer-other-window "*Preview Markdown Output*")

It would be annoying to replace the window with your Markdown file with *Preview Markdown Output*, so it’s a nicer experience to show it in a window you’re not currently using. If *Preview Markdown Output* is already visible in a different window, it will switch to that window; otherwise it will pick another window (creating one if needed) and then switch to the *Preview Markdown Output* buffer in that window.

There are a lot of ways to create and split windows in Emacs. You might decide that you’d rather have *Preview Markdown Output* open a new window rather than replace the contents of an existing one; you can use functions like split-window-sensibly to split the current window instead. A full discussion of other options is outside the scope of this article, but hopefully I can include more examples in the future.

Parsing the HTML source

(libxml-parse-html-region (point-min) (point-max))

Before we can render the HTML, we have to parse it. libxml-parse-html-region parses HTML in the region. Again, we’re using point-min and point-max to tell it to parse the entire buffer; we’re saving the output of that function as document.

Clearing the contents of `Preview Markdown Output`

(erase-buffer)

We’re going to put the rendered HTML back into *Preview Markdown Output*, so we need to clear out the HTML source first. (erase-buffer) clears the contents of the current buffer.

Render the parsed HTML document with SHR

(shr-insert-document document)

Renders the parsed HTML document in the current buffer.

Moving back to the beginning of the buffer

(goto-char (point-min))

At this point, the cursor will be all the way at the end of the buffer. It’s nicer behavior if our preview shows us the beginning of the file rather than the end. goto-char moves the cursor, and we’re moving it to the beginning of the buffer, again using point-min to determine the beginning.

Next Steps

In Part 2, we’ll make improvements to the command, and discuss Emacs Lisp lists, cons cells, backquote-splicing, and macros.

Cam’s Secret Hot Sauce Recipe

2020-04-25T03:11:00+00:00

Not a day goes by without someone asking me the recipe for Cam’s House Sauce, widely considered one of the greatest hot sauces in the world. Surely you are familiar with its delicate and subtle blend of habanero with notes of cinnamon and cherry.

Now in this Coronavirus pandemic you can make it yourself from the comfort of your very own armchair. For1 a limited time only, here is the recipie:

Ingredients

11 oz of habaneros
11 oz of serrano peppers or equivalent (remove stems, keep everything else)
20 cloves of garlic
1 red onion (remove skin)
2 cups of fresh-squeezed ruby red grapefruit juice. Depending on the size of the grapefruits, this is probably about 3 grapefruits
4 fl oz of white wine vinegar
1 tablespoon of ghost pepper salt
1 bunch of cilantro (leaves only)

Needless to say, either grow all of these ingredients yourself or be sure to use the highest-quality organic free-range peppers and herbs.

Instructions

Throw everything in a blender and blend until it is a nice puree
Take resulting pulp and run thru a juicer.
Bring to a boil in a pot. Important! Make sure you use a non-reactive pot such as stainless steel. Don’t use aluminum unless you want it to taste weird.
Reduce heat and simmer 20 mins.

After that you can take it off the heat and bottle it with the hot fill method. I think this makes somewhere between 5 and 10 bottles the size of the ones in the picture above.

Final Thoughts

The lawyers advised me against giving away trade secrets like these for nothing, but I couldn’t resist the urge to lend a helping hand in this time of need. Also I wanted to type this recipe up in case I lose the envelope I wrote it down on that I’ve been using for the last 2 years:

Have fun and be sure to let me know if you make any tweaks or improvements to the recipe!

👇 Like and subscribe below! 👇

Methodical now supports partial default methods

2020-04-22T20:36:00+00:00

Methodical, my Clojure library that ports the “generic function” part of the Common Lisp Object System (CLOS) to Clojure (and makes multimethods way more powerful) just got even more powerful. Methodical now supports dispatch on partial defaults!

What does that even mean?

Take a look at this Clojure code:

(defmulti vanilla-multimethod
  identity)

(defmethod vanilla-multimethod [:x :y]
  [_]
  :x-y)

(defmethod vanilla-multimethod [:default :y]
  [_]
  :y-default)

(defmethod vanilla-multimethod :default
  [_]
  :default)

(vanilla-multimethod [nil :y]) ; -> :default

Vanilla multimethods support exactly one default method, and that’s it. If you want to have a method for any dispatch value where the second value is :y, you are completely out of luck. The best you can do is some hackery like this:

(defmethod vanilla-multimethod :default
  [[x y]]
  (let [default-method   (get-method vanilla-multimethod :default)
        x-default-method (get-method vanilla-multimethod [:default y])
        y-default-method (get-method vanilla-multimethod [x :default])]
    (condp not= default-method
      x-default-method (x-default-method [x y])
      y-default-method (y-default-method [x y])
      :default)))

(vanilla-multimethod [nil :y]) ; -> :y-default

That’s something I’ve actually done in real life, in production!

Why?

There’s some real-world use cases where you’d want partial defaults. When writing Metabase’s query processsing code for JDBC databases, I wanted to write a multimethod for reading columns in a ResultSet that dispatched on both the database and the JDBC type. For example, we have a default method for reading java.sql.Types/TIME_WITH_TIMEZONE columns like:

(defmethod read-column-thunk [:default java.sql.Types/TIME_WITH_TIMEZONE]
  ...)

For some databases, we override this method like:

(defmethod read-column-thunk [:postgres java.sql.Types/TIME_WITH_TIMEZONE]
  ...)

So what happens when we call read-column-thunk with the dispatch value [:mysql java.sql.Types/TIME_WITH_TIMEZONE]? By default, it doesn’t work – it will use the :default method rather than the [:default java.sql.Types/TIME_WITH_TIMEZONE] method as we would like.

Can’t I use hierarchies?

The astute reader will note that you can create a keyword hierarchy and have both :postgres and :mysql derive from a common parent keyword (e.g. :database). That’s certainly true! But creating a hierarchy (or using the global hierarchy with namespaced keywords) just for the sake of a multimethod is annoying, and it still wouldn’t handle cases like [nil java.sql.Types/TIME_WITH_TIMEZONE], where the first part of the dispatch value doesn’t derive from the common parent. It doesn’t work as a true fall-thru for any java.sql.Types/TIME_WITH_TIMEZONE.

Enter Methodical

Methodical now supports partial defaults! Here’s how to do it in Methodical:

(require '[methodical.core :as m])

(m/defmulti my-multimethod
  identity)

(m/defmethod my-multimethod [:x :y]
  [_]
  :x-y)

(m/defmethod my-multimethod [:default :y]
  [_]
  :y-default)

(m/defmethod my-multimethod :default
  [_]
  :default)

(my-multimethod [nil :y]) ; -> :y-default

The new multi-default-dispatcher is the default dispatcher, but with Methodical you can customize method dispatch and have it work any way you’d like!

How Can I Learn More?

I’m giving a talk about Methodical at Clojure/north this year, so stay tuned! As far as I know, Clojure/north is going to be a remote conference due to Coronavirus concerns, so you can attend from anywhere in the world!

👇 Like and subscribe below! 👇

camsaul.com

My Experience Using Prolog in 2024 to Solve Advent of Code

TL;DR How Was Using Prolog Anyway?

Pits

Peaks

Other thoughts

I gave an epic talk at Clojure/conj 2024

Debugging Custom Kondo Hooks and Clojure LSP Jump-to-Definition Functionality

Add Clojure LSP as a dependency

Launch REPL with the :lsp alias

Debug from the REPL

What was broken?

Conclusion

Humane Are Just Dropped!

Hello, world. It’s me, your friendly neighborhood #cam.

My Clojure/north 2020 talk about Methodical is now available on YouTube

Intro to Emacs Lisp: Adding Live Previews when Editing Markdown Files (Part 3 of 3)

Prompting for a file to preview

Optional arguments

Specifying a prompt in the interactive declaration

if forms

progn forms

Understanding the updated cam/preview-markdown

Opening a file/switching to existing buffer for a file

Adding a docsting

Adding an after-save-hook

Add-hook and buffer-local variables

Lisp-1s and Lisp-2s

Quoting function names

Final Thoughts

Intro to Emacs Lisp: Adding Live Previews when Editing Markdown Files (Part 2 of 3)

Viewing Source

Improving the preview-markdown command

Rendering relative to the current directory

Quoting

Understanding Lists, Cons Cells, and Association Lists

Other uses of cons cells

Wrapping our HTML document

Backquote-splicing

Splicing items into the parent form with ,@

Preserving the current window

Making *Preview Markdown Output* read-only

Macros 101

Macroexpansion

Scrolling to approximate location in preview

Next Steps

Intro to Emacs Lisp: Adding Live Previews when Editing Markdown Files (Part 1 of 3)

Evaluating forms and ielm

Writing a preview-markdown command

Defining a function

Making it a command

Logging Messages

Calling external commands

Looking up documentation

Evaluating Emacs Lisp from the minibuffer with M-:

Viewing the HTML

Binding local variables with let

Switching to a different window

Parsing the HTML source

Clearing the contents of *Preview Markdown Output*

Render the parsed HTML document with SHR

Moving back to the beginning of the buffer

Next Steps

Cam’s Secret Hot Sauce Recipe

Ingredients

Instructions

Final Thoughts

Methodical now supports partial default methods

What does that even mean?

Why?

Can’t I use hierarchies?

Enter Methodical

How Can I Learn More?

Launch REPL with the `:lsp` alias

Specifying a prompt in the `interactive` declaration

`if` forms

`progn` forms

Understanding the updated `cam/preview-markdown`

Adding an `after-save-hook`

Improving the `preview-markdown` command

Wrapping our HTML `document`

Splicing items into the parent form with `,@`

Making `Preview Markdown Output` read-only

Evaluating forms and `ielm`

Writing a `preview-markdown` command

Evaluating Emacs Lisp from the minibuffer with `M-:`

Binding local variables with `let`

Clearing the contents of `Preview Markdown Output`