(N.B. Note added 13 June 2021: This article is useful in terms of understanding how to add token hooks to code. However, in terms of solving the specific problem outlined, the article is obsolete. See here for the updated solution.) Since I picked up my work on the JavaCC codebase at the end of 2019, …
“You can’t get there from here!” — The Problem of Context-Sensitive Tokenization Read More »
The FAIL construct is really pretty dead simple. Here is an example: A. ( “foo” | “bar” | FAIL “Was expecting \”foo\” or \”bar\” here!” ) At first I thought the above was just syntactic sugar, since you can, of course, already write: B. ( “foo” | “bar” | {throw new ParseException(“Was expecting \”foo\”or “\bar\” …
A bit over a month ago, I wrote about my intention of tearing out support for “JAVACODE” productions. A JAVACODE production is really just a Java method that is like a pretend grammatical production. I could find very few JavaCC grammars in the wild that used this. I asked whether anybody would miss them and …
One of the very basic goals of this iteration of work on the JavaCC codebase, JavaCC 21, is to turn this into a capable tool for generating fault-tolerant parsers. Traditionally, not just JavaCC, but tools in this space generally, consider that it is their job to parse, i.e. to break down input into its various pieces …
A couple of weeks ago, I implemented a solution to a longstanding Dmitry Dmitriyevich problem with LOOKAHEAD specifications in JavaCC. Dmitry Dmitriyevich (alternatively Ivan Ivanovich or Vladimir Vladimirovich) is my own personal terminology for situations where you are required to write highly repetitive code. (Patriotic Irishmen are welcome to call this the Patrick Fitzpatrick or Gerald …
As a result of some private correspondence with one of the PMD developers (PMD makes extensive use of JavaCC) I started thinking about some issues that I had in the back of my mind to look into at a later point. Having recently ripped out the code in JavaCC 21 that supported putting LOOKAHEAD specifications …
Revisiting LOOKAHEAD in JavaCC, Attacking the Dmitry Dmitriyevich Problem Read More »
Programmatically generating code is a little world in itself, with its own little quirks and tricks. Consider the generation of the C-style switch-case with fall-through. You have your code generation that generates all your cases, and each case ends with a break. There is a problem, which is that the moment any of your case …