As many projects use GitHub Actions for CI, there seems to be more resources for setting GitHub Actions up for Emacs Lisp, including purcell/setup-emacs and leotaku/elisp-check. There are also many examples to copy from.
On the other hand, I was only able to find one Emacs Lisp project using Gitlab CI: joewreschnig/gitlab-ci-mode.
To learn how to do this, I added automatic testing to tst.el, a tiny library hosted on Gitlab. This is the result.
default: before_script: - cask install test-26.3: image: silex/emacs:26.3-ci-cask script: - cask exec ert-runner test-27.1: image: silex/emacs:27.1-ci-cask script: - cask exec ert-runner
This tells Gitlab CI to automatically run tests (with ert-runner) under Emacs 26.3 and Emacs 27.1.
test-27.1are job names for Gitlab CI.
defaultis applied to every job. Only some properties can be specified here.
scriptisn’t one of them.
- Alternative: you can also specify a
.testrule, then use
extends: .testto reduce duplication. This is what joewreschnig/gitlab-ci-mode does.
silex/emacsis a comprehensive set of Emacs Docker images. Some of them are great for interactive use, others are designed for use in CI. There’s also the images from Flycheck (
flycheck/emacs-cask); those include Cask but not have Git, which undercover needs for reporting coverage.
The Cask file:
(source gnu) (source melpa) (package-file "tst.el") (development (depends-on "ert-runner") (depends-on "undercover"))
Cask lets you write down your project’s dependencies in a declarative way and install them in one command, instead of having to write a bunch of boilerplate in an ad-hoc init file.
The entire Cask file language is documented in this 700-word page.
cask installinstalls all dependencies, including development dependencies, in a local folder (
cask emacscan then be used to run Emacs with those local dependencies made available.
Tests are written like this:
(require 'ert) (require 'tst) (ert-deftest tst-get-test () ;; Empty (should (null (tst-get "abc"))) (should (null (tst-get "abc"))) ;; Not at end of string (should (null (tst-get "abc[a b]def"))))
ert-deftestdefines a test.
tst-get-testis the name of the test.
shouldmakes the test fail if its body is nil.
You can evaluate this then run
M-x ert to run the test.
Some articles that introduce ERT better:
- abrochard - ERT: Emacs Lisp Regression Testing
- Sean Miller - Refactoring “Beginning Emacs Lisp”: I: Adding Tests
- EmacsWiki - Ert Test Library
Running tests with ERT involves a long command buried in its manual.
cask install cask emacs -batch -l ert -l my-tests.el -f ert-run-tests-batch-and-exit
cask emacs so that dependencies are available.)
ert-runner is designed to simplify this, and make running ERT tests less painful.
cask install cask exec ert-runner
Coverage means how much of your code is covered by unit tests.
Typically (as far as I know) one uses a coverage library for their language to compute it, then upload the results to a coverage tracking service.
The coverage library for Emacs Lisp is undercover.
Setting up undercover
Install it with Cask:
(development (depends-on "undercover"))
require the library and specify a wildcard that matches your source files before you load your package:
(when (require 'undercover nil t) (undercover "*.el")) (require 'ert) (require 'tst) (ert-deftest tst-get () ;; Empty (should (null (tst-get "abc"))))
Undercover will then automatically upload the results to Coveralls if a token has been given (through the
COVERALLS_REPO_TOKEN environment variable).
Setting up Coveralls
- Log in with Github, Gitlab, or Bitbucket
- Maybe connect with the other two services, so that you don’t accidentally create another account if you forget which service you logged in with.
- Authorize its access
- Connect your repository
- Copy the repository token
Add a secret environment variable for your repository on Gitlab:
- Go to your project → settings → CI / CD → Variables → Expand → Add Variable
- Set Key to
COVERALLS_REPO_TOKEN, Value to the repository token you just copied
- Make sure both Protect Variable and Mask Variable are checked.
- Maybe add the badge to your README.