As many projects use GitHub Actions for CI, there seems to be more resources for setting GitHub Actions up for Emacs Lisp, including purcell/setup-emacs and leotaku/elisp-check. There are also many examples to copy from.
On the other hand, I was only able to find one Emacs Lisp project using Gitlab CI: joewreschnig/gitlab-ci-mode.
To learn how to do this, I added automatic testing to tst.el, a tiny library hosted on Gitlab. This is the result.
default: before_script: - cask install test-26.3: image: silex/emacs:26.3-ci-cask script: - cask exec ert-runner test-27.1: image: silex/emacs:27.1-ci-cask script: - cask exec ert-runner
This tells Gitlab CI to automatically run tests (with ert-runner) under Emacs 26.3 and Emacs 27.1.
test-27.1are job names for Gitlab CI.
defaultis applied to every job. Only some properties can be specified here.
scriptisn’t one of them.
Alternative: you can also specify a
.testrule, then use
extends: .testto reduce duplication. This is what joewreschnig/gitlab-ci-mode does.
silex/emacsis a comprehensive set of Emacs Docker images. Some of them are great for interactive use, others are designed for use in CI.
There’s also the images from Flycheck (
flycheck/emacs-cask); those include Cask but not have Git, which undercover needs for reporting coverage.
The Cask file:
(source gnu) (source melpa) (package-file "tst.el") (development (depends-on "ert-runner") (depends-on "undercover"))
Cask lets you write down your project’s dependencies in a declarative way and install them in one command, instead of having to write a bunch of boilerplate in an ad-hoc init file.
The entire Cask file language is documented in this 700-word page.
cask installinstalls all dependencies, including development dependencies, in a local folder (
cask emacscan then be used to run Emacs with those local dependencies made available.
Tests are written like this:
(require 'ert) (require 'tst) (ert-deftest tst-get-test () ;; Empty (should (null (tst-get "abc"))) (should (null (tst-get "abc"))) ;; Not at end of string (should (null (tst-get "abc[a b]def"))))
ert-deftestdefines a test.
tst-get-testis the name of the test.
shouldmakes the test fail if its body is nil.
You can evaluate this then run
M-x ert to run the test.
Some articles that introduce ERT better:
abrochard - ERT: Emacs Lisp Regression Testing
Sean Miller - Refactoring “Beginning Emacs Lisp”: I: Adding Tests
EmacsWiki - Ert Test Library
Running tests with ERT involves a long command buried in its manual.
cask install cask emacs -batch -l ert -l my-tests.el -f ert-run-tests-batch-and-exit
cask emacs so that dependencies are available.)
ert-runner is designed to simplify this, and make running ERT tests less painful.
cask install cask exec ert-runner
Coverage means how much of your code is covered by unit tests.
Typically (as far as I know) one uses a coverage library for their language to compute it, then upload the results to a coverage tracking service.
The coverage library for Emacs Lisp is undercover.
Setting up undercover
Install it with Cask:
(development (depends-on "undercover"))
require the library and specify a wildcard that matches your source files before you load your package:
(when (require 'undercover nil t) (undercover "*.el")) (require 'ert) (require 'tst) (ert-deftest tst-get () ;; Empty (should (null (tst-get "abc"))))
Undercover will then automatically upload the results to Coveralls if a token has been given (through the
COVERALLS_REPO_TOKEN environment variable).
Setting up Coveralls
Log in with Github, Gitlab, or Bitbucket
Maybe connect with the other two services, so that you don’t accidentally create another account if you forget which service you logged in with.
Authorize its access
Connect your repository
Copy the repository token
Add a secret environment variable for your repository on Gitlab:
Go to your project → settings → CI / CD → Variables → Expand → Add Variable
Set Key to
COVERALLS_REPO_TOKEN, Value to the repository token you just copied
Make sure both Protect Variable and Mask Variable are checked.
Maybe add the badge to your README.