Coding Dojo: Let's build a LISP lexer together - in Python

Over the course of the last 12 months, Lukas Prokop has written several lexers and parsers. He shares his experience and is going to introduce everyone to the idea of writing a basic lexer and parser. LISP (or its syntax which is based on S-expressions) is famously the simplest syntax, if you want to support nested structures. The syntax is based on a prefix notation where every instruction is wrapped by parentheses and the arguments are whitespace-separated: (function arg1 arg2).

In fact, Peter Norvig's article on writing a LISP interpreter in python is quite famous: (How to Write a (Lisp) Interpreter (in Python)).

In this coding dojo, we will join together in a session to write a lexer for LISP. First, Lukas is going to introduce us to the challenge in a small presentation. He will also provide utilities for error messages and depending on the time, the parser (which is trivial to write) is provided or will be developed together. The core effort is developing the lexer (= tokenizer) in a communal effort.