Skip to content

Move to compiled parser generator - #2984

Open
rodinaarssen wants to merge 36 commits into
mainfrom
feature/compiled-parser-generator
Open

rodinaarssen wants to merge 36 commits into
mainfrom
feature/compiled-parser-generator

Conversation

@rodinaarssen

Copy link
Copy Markdown
Member

In this PR, the compiled parser generator is introduced. The generated code is linked in as a git submodule (generated) and replaces the evaluator that used to run the parser generator.

A benchmark in which the grammar rules and parser of Rascal itself were calculated 100 times in succession showed:

  • compiled parser generator: 157 seconds, ~37% spent in javac
  • interpreted parser generator: 478 seconds, ~14% spent in javac

Inspired by an earlier experiment by @jurgenvinju

@rodinaarssen
rodinaarssen force-pushed the feature/compiled-parser-generator branch from d72e5bb to 4d53bca Compare November 11, 2026 09:53
@jurgenvinju

Copy link
Copy Markdown
Member

excellent! It would also be interesting to see the difference in loading time of the parser generator between the two modes.

Additionally, I think we (temporarily) need license headers on the generated Java code. Otherwise the code in the generated files is not open-source and (commercial) users can be confronted with unnecessary liabilities. It might be practical to include this as an optional feature in the Rascal compiler?

@rodinaarssen
rodinaarssen force-pushed the feature/compiled-parser-generator branch from 4d53bca to 9fcca3a Compare November 11, 2026 09:59
for (URL found : Collections.list(getClass().getClassLoader().getResources(fileName))) {
try {
result.add($RVF.sourceLocation(found.toURI()));
result.add($RVF.sourceLocation(URIUtil.fromURL(found)));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch!

@jurgenvinju jurgenvinju left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that a README.md close to the generated folder with the relevant git commands to keep the submodule up-to-date would be good. Similarly for generating new code a small README entry would be good.

@PaulKlint PaulKlint left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@PaulKlint

Copy link
Copy Markdown
Member

@jurgenvinju Give me a license text and I will add it (together with a warning that this generated code and a timestamp) to the generated code.

@codecov

codecov Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 58.82353% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 47%. Comparing base (23dde9c) to head (11ee5c4).

Files with missing lines Patch % Lines
src/org/rascalmpl/parser/ParserGenerator.java 70% 9 Missing ⚠️
src/org/rascalmpl/shell/RascalGenerateSources.java 0% 7 Missing ⚠️
.../org/rascalmpl/runtime/RascalExecutionContext.java 70% 3 Missing ⚠️
...rc/org/rascalmpl/library/lang/java/JavaRunner.java 0% 1 Missing ⚠️
src/org/rascalmpl/runtime/$RascalModule.java 50% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##              main   #2984    +/-   ##
========================================
+ Coverage       46%     47%    +1%     
- Complexity    6899    7026   +127     
========================================
  Files          851     852     +1     
  Lines        69233   69195    -38     
  Branches     10084   10080     -4     
========================================
+ Hits         32064   32825   +761     
+ Misses       34737   33776   -961     
- Partials      2432    2594   +162     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@DavyLandman DavyLandman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but we should (to extend on @jurgenvinju's points):

  • Put a developers.md file on rascal repo on how to clone & pull
  • put a developers.md file on the generated repo that explains how to update the generated source code and what steps to follow in git
  • explain how to use released version instead of current main?

Comment thread pom.xml Resolved
@PaulKlint

Copy link
Copy Markdown
Member

PR #2985 adds the capability to add a license header to generated code.

@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants