Mike Lewis posted a marvelous experience report dubbed ‘Optimizing Boost Spirit – Blazing fast AST generation using boost::spirit’. He describes how he took an old compiler for the Epoch programming language (which was based on Spirit.Classic) and tuned it for performance using Spirit.Qi and Spirit.Lex. His results are exceptional, he got roughly a thousand fold speedup compared to the old version. The complete code for his compiler can be downloaded from here.
He writes:
This code illustrates several advanced techniques for parsing large inputs with complex Spirit grammars:
- Deferred construction and minimal copying of attribute values
- Lexical analysis for faster backtracking
- A special directive for using qi::symbols alongside a lexer
- Linear allocators for faster AST node allocation
- Intrusive reference counting for even faster AST node allocation/copying
- Grammar transformations for general optimality
- Abuse of the &-predicate for skipping expensive productions
- Dividing grammars into multiple implementation files for minimal recompilation times
Thanks Mike for sharing your work! I’m sure many Spirit developers will find it very enlightening and encouraging to read about your work. Keep up the excellent work!
