Name: Finite State Machines: A Tour of Python's Grammar Parser
Start: 2016-11-09T19:00:00-05:00
End: 2016-11-09T22:00:00-05:00
Location: Buzzfeed

Python's parser/parser generator is responsible for incorporating the rules defined in Python's grammar into a valid block of code.

We'll take an academic look at non-deterministic and deterministic finite automata. Next we'll look at how Python's team implemented this in python (see Parser/pgen.c) by walking through the code.

Hopefully you'll leave with the skills required to write your very own grammar parser.

I'm going to try and order burritos to go with our beer this time. Pizza is getting a little old, right?

UPDATE:

Hey guys, I've been doing a bit of research and have some homework reading for you all! If you want to brush up on parsing in Python you can get access to the low-level interface/parse tree with the parser module here:

https://docs.python.org/3.7/library/parser.html

Although they recommend using the ast module for interacting with the tree.

We'll just mention in passing the lexical analyzer; which feeds tokens into the parser. You can find documentation on it here:

https://docs.python.org/3.7/reference/lexical_analysis.html

Here's a pretty good run-through on finite state machines: https://en.wikipedia.org/wiki/Finite-state_machine

There was a kind of interesting conversation going on about peak memory usage by the parser lately. You can find the thread here:

https://mail.python.org/pipermail/python-dev/2016-August/145803.html

Guido shot it down with a comment to the affect of "sometimes things are too hard to be worth doing". I can't remember the exact words but I'll dig them up. I think I tweeted it recently.

I'm having a bit of trouble digging up cool stuff from the dev mailing list but I'll try to scrape together some more historical context soon.

I would also recommend checking out the "dragon book" on compilers for it's section on Finite State Automata:

https://www.amazon.com/Compilers-Principles-Techniques-Alfred-Aho/dp/0201100886/ref=sr_1_1?ie=UTF8&qid=1476137216&sr=8-1&keywords=Compilers%3A+Principles%2C+Techniques%2C+and+Tools

which is the basis for Python's algorithm.
See you all in a week!

Andrew Kelleher

Deep Dive into Python: Know Thy Interpreter

Technology

Internet Professionals

Open Source

Web Technology

Linux

Web Development

Software Development

Open Source Python

Programming Languages

Python Web Development

Python

Computer Programming

Finite State Machines: A Tour of Python's Grammar Parser

Buzzfeed

Share

Deep Dive into Python: Know Thy Interpreter

Finite State Machines: A Tour of Python's Grammar Parser

Deep Dive into Python: Know Thy Interpreter

Details

Members are also interested in