Progress on ELTN

2019-03-26

In an earlier post I introduced ELTN, the Extended Lua Table Notation.

Well, I decided to try writing the thing. In Lua. Which seems pointless except for the JSON parsers in JavaScript. I chose Lua since:

This has been a learning experience in more ways than one:

  1. I’ve never written anything significant in Lua before.

  2. I’m still a parsing newb. Sure, I wrote a JSON parser in Java once, and almost one in Eiffel. Mostly, though, I’ve been getting by on regular expressions and libraries.

  3. Not writing software for a while has taken its toll. I started the project by writing a bunch of code for parsing, deleting it, and then writing what I thought was better. What I should have done first, and what I finally did in a coffee shop, was write a program to test the eventual parser. I don’t know if Lua has a “standard” unit testing module or not. However, it wasn’t hard to code up a quick test program:

    • Generate a table from an ELTN “document”, catching any errors.
    • Compare it to a table that has the same structure the generated one should have, using a custom comparison function. (Lua only compares a table by reference, unless you set up a metamethod for each table, which seemed too strenuous.)
    • If the tables don’t match, or the parser signals an error (when it’s not supposed to), print out a message and the generated error message.
  4. Having a test harness, I proceeded to test the (third1) smallest possible ELTN document “a=1”. It was surprisingly hard, especially with the ill-considered gorp I’d previously written. I still hadn’t passed the test before the battery on my laptop cut out.

So far I haven’t gotten back to it. Probably the first thing I’ll do is move all the questionable code to another file, leaving just the nearly empty main parser function. Also I’ll read up on coroutines in Lua and experiment with some functions that return functions. In the code so far I seem to be carrying the string to parse and the current position in the string everywhere. Ideally only the lexer/tokenizer needs that. So maybe I need an initialization function that returns a function that simply spits out a token (or signals an error) on each invocation. (Again, trying function-oriented rather than object-oriented.)

Also, I’m not even going to worry about the “function call” syntax. So for now ELTN’s grammar is this:

document         ::= (stat)*
stat             ::=  ';' | var '=' exp
var              ::= NAME
exp              ::= constant | tableconstructor 
constant         ::= 'nil' | 'false' | 'true' | NUMERAL | STRING_LITERAL
arg              ::= tableconstructor | STRING_LITERAL
tableconstructor ::= '{' ( fieldlist )? '}'
fieldlist        ::= field ( fieldsep field )* ( fieldsep )?
field            ::= key '=' exp | exp
key              ::= '[' constant ']' | NAME
fieldsep         ::= ',' | ';'

where | separates alternatives, “()* means zero or more and ()? means optional, as before.

At least I’m not the Toml guy who specified a data description language without providing a reference parser.


  1. The second smallest is “;"; the smallest is “". ↩︎