I recently completed a major refactoring of how the generated Token API works in JavaCC 21. Though this constitutes a quite revolutionary set of changes under the hood it may actually not be all that noticeable to many (or most) users. I think there are basically two groups of users (not mutually exclusive) who would …
JavaCC 21 now has assertions. There are actually two kinds of assertions: The assertion condition is expressed in Java code. The assertion condition is a lookahead expansion. The first kind of condition looks like this: ASSERT {someCondition()} Optionally, the assertion can have a message, as in: ASSERT {x!=y} : “At this point, x and y …
As the title says: as of this writing, the included grammars for Java and Python both support the latest versions of the language, JDK 17 and Python 3.10 respectively. Contextual Keywords It turns out that, from a language support perspective, the only new stable feature in JDK 17 is the concept of sealed classes. This …
Announcement: JavaCC 21 includes parsers for the latest Java and Python versions! Read More »
Sometimes, when you complete a major code cleanup, features that were previously pie in the sky become low-hanging fruit to pluck. The new feature that I describe here, the ability to activate and deactivate tokens is such a case. It resulted from my rewriting of the lexical code generation that I describe here. In an …
Context-Sensitive Tokenization, Next Installment, Activating and De-activating Tokens Read More »
Dear Readers… This blog has been dark for about two months now, but not because I was inactive in the project. Quite the opposite actually. What happened over the last couple of months is that I went into full-blown obsessive mode and managed to rewrite the remaining part of JavaCC that had been resisting my …
Major Milestone: The Lexical Code Generation is completely rewritten Read More »
"It’s too big! It doesn’t fit!" The above does not refer to any particular pornographic feature film, but rather, to a longstanding problem in JavaCC: if you write a very big, complex lexical grammar, the generated XXXTokenManager would fail to compile, with the compiler reporting the error: "Code too large". Well, this has now been …
The Dreaded “Code too large” Problem is a Thing of the Past Read More »
A few weeks ago, in a previous post I wrote: The problem, at the time of this writing, is that neither JavaCC21 nor legacy JavaCC have any concept of characters outside of plane 0, which are the ones that require 4 bytes of internal storage. Or another way of putting this is that the assumption …
JDK 16 is out! Though JDK 16 was released on 16 March, a week ago, I only became aware of that a few days later on March 20. Once I saw that news item, I figured I would just immediately add support for the new JDK 16 language features to JavaCC. It didn’t take me …
Now that the ability to generate fault-tolerant parsers is coming along so well, I have been thinking about what to do with the Java grammar included in CongoCC. I decided that the best thing to do was to do the incremental work to make it maximally correct. Ideally, it will serve as a reference implementation …
Moving Towards a Maximally Correct Reference Java Grammar Read More »
For some time, it has been a goal of JavaCC 21 to provide the ability to generate fault-tolerant parsers. I started working on the problem about a year ago. However, I had not put in a comprehensive solution until now for several reasons. Basically these: The codebase, though already significantly refactored and cleaned up, was …
A Glimpse of the Promised Land: Fault-tolerant parsing Read More »