AntlrVSIX is a tool to support editing, analysis, refactoring, and conversion of context-free grammars, including Antlr, Bison, ISO 14977, LBNF, and W3C EBNF. It contains extensions for Visual Studio 2019, Visual Studio Code, Gnu Emacs, a Language Server Protocol (LSP) server, and a stand-alone command-line tool known as Trash. Trash is analogous to Bash but for parsing, in the lingua-franca of parsing: parse trees and XPath. All clients use the LSP server implemented in this repository. The clients each are implemented elsewhere, but a thin shim is provided in this repository to get the client to work with the server. For Visual Studio 2019, the client is implemented using Microsoft's client. The server is implemented using Antlr, Antlr4BuildTasks, XPath2, S-expressions for Antlr parse trees, and a number of other tools. Most of the code is implemented in C#. The extension for VS Code is written in Typescript. Supported are colorized tagging, hover, go to def, find all refs, replace, and reformat. Beyond the basic LSP commands, the tool supports analysis, refactoring, and conversion. NB: Antlrvsix for Visual Studio 2019 v8.3 will not offer semantic highlighting by default because the MS LSP client in VS2019 does not support semantic highlighting. In fact, it is 3 years behind the protocol. In order to support it for VSCode, I had to rewrite the entire language server protocol library (replacement here)--in a week--in order to bring it up to date. MS is too slow. If you want semantic highlighting, you can turn on the feature in the Antlrvsix options. Or, better, you could edit your grammar file using VSCode with Antlrvsix-vscode or Mike's vscode-antlr extension for VSCode. This problem will be addressed in release v8.4 or v9.0.
Supported grammars and features
Visual Studio 2019
For Visual Studio 2019, I recommend that you use the latest, upgraded version since it has the latest bug fixes.
To install, visit the Visual Studio Marketplace for VS2019 for Antlrvsix. Download and install the .vsix file. You can also install it through the
If you find this extension not to your liking, you can try to use the extension for Visual Studio Code. I found that it is sometimes easier to install Open in Visual Studio Code and edit the grammar in VSCode due to VS2019 not directly supporting LSP semantic highlighting.
Visual Studio Code
For Visual Studio Code, visit the Visual Studio Marketplace for VSCode for Antlrvsix. Download and install the the .vsix file. NB: the vsix file between VS2019 and VSCode are not the same files.
Note, to get semantic highlighting of grammar symbols, you need to make sure to pick a color theme that has them, such as "Light+", "Dark+", etc., and make sure to adjust the settings "File > Settings > Text Editor > Semantic Highlighting: Enabled, true". If you just start out with VSCode, you get nothing.
For Emacs, you will need to follow the instructions here.
Each night at 0h 0m UTC, Appveyor builds the latest source. The output of the build can be downloaded from Appveyor.
VsIde v8.3, VsCode 1.2 (30 Dec 2020)
After two months of work, release 8.3 for Antlrvsix has been completed. Most of the changes pertain to Trash, the command-line interpreter shell contained within Antlrvsix, but there are a few other important changes to the extension itself.
Release 8.3 features an all-new input/output pipe implementation between commands in Trash. It uses a JSON serialization of parse trees between commands. The JSON serialization also contains the parser, lexer, token stream, file contents, and file location. Basically, it includes everything that one would need for a command to manipulate a parse tree. The purpose of the JSON file is so that each command can now be implemented “out-of-process”, meaning that Trash can now be replaced by Bash and each command implemented as an independent program in any language of choice. Nobody wants yet another monolithic program to implement programming language tools. This release works towards an open system. With this release, all the commands and Trash still are implemented in one program, but it will be switched over in a month or two.
I’ve also included a few new goodies:
The “diff” program I implemented with this release is interesting. I used the Zhang-Shasha tree-edit distance algorithm, extending it to record the actual tree edits that correspond to the minimum tree-edit distance. This algorithm is, unfortunately, for an ordered tree, so it works best for small differences. And, for large grammars, it is too slow. I will be trying to implement other algorithms in the next month or two. There is certainly a lot that could be done here. One important type of difference is to include no only simple single-node inserts and deletes, but more complex operations like fold and unfold.
In addition, with this release, I’m disabling semantic highlighting for VS2019 (but not for VSCode). This is because it’s buggy and slow, and despite my warning people, they complain about it being buggy and slow! Use the extension for VSCode. It works fine. In the next release, I will try to fix Antlrvsix for VS2019, but you never know.
Guide to basic features
For information on how basic use, see the User Guide.
Running the Trash command-line engine
The Trash tool is a command-line interpreter for executing operations on grammar files. With it, you can read Antlr2, Antlr3, Antrl4, and Bison grammars, convert to Antlr4, find sub-trees in the parse tree for the grammar using XPath expressions, analyze, and refactor the grammar.
To run the tool, you will need to find the Trash.exe executable after installing the extension in Visual Studio 2019, or unpacking it directly from the downloaded .vsix--a ZIP file. It should be under "C:/Users/userid/AppData/Local/Microsoft/VisualStudio/.../AntlrVSIX/8.0/Trash/netcoreapp3.1/Trash.exe". Once you find the executable, please refer to the documentation.
Antlrvsix provides a number of transformations that can help to make grammars cleaner (reformatting), more readable (reducing the length of the RHS of a rule), and more efficient (reducing the number of non-terminals) for Antlr.
Some of these refactorings are very specific for Antlr due to the way the parser works, e.g., converting a prioritized chain of productions recognizing an arithmetic expression to a recursive alternate form. The refactorings implemented are:
Raw tree editing
Splitting and combining
The source code for the extension is open source, free of charge, and free of ads. For the latest developments on the extension, check out my blog.
The extension is at ./Client/bin/Debug/AntlrVSIX.vsix after building successfully.
See this guide.
Planned for v9 (end Dec 2020)
Mike Lischke's vscode-antlr4 is a good tool for VSCode for Antlr grammars, which you may prefer. However, Antlrvsix contains a command-line tool that can be used to automate transformations on an Antlr grammar, or want to quickly run a parser over a grammar file and use XPath to query items in the parse tree. There is no tool like it, and it is quite powerful.
If you are just building and running an Antlr application and don't care about editing features (i.e., you don't care for highlighting the grammar, go to def, etc.), then you just use Antlr4BuildTasks. Antlr4BuildTasks is a replacement of Antlr4cs and the build tool contained in that library. The Antlr4cs is several versions behind the latest Antlr release.
Any questions, email me at ken.domino gmail.com