:bug: add a `DoubleColon` to the `Token` struct in the lexer.rs by LesterEvSe · Pull Request #243 · BlockstreamResearch/SimplicityHL

LesterEvSe · 2026-03-17T12:39:14Z

This prevents the lexer from allowing spaces between colons (e.g., a: :b),

KyrylR · 2026-03-17T12:44:16Z

src/lexer.rs

+    /// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`),
+    /// ensuring we strictly parse valid paths.


Suggested change

/// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`),

/// ensuring we strictly parse valid paths.

/// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`)

KyrylR · 2026-03-17T12:44:55Z

Cargo.toml

 doc_link_with_quotes = "warn"
 doc_markdown = "warn"
-empty_enum = "warn"
+empty_enums = "warn"


Explain in description why this is changed

After updating to Rust 1.94.0, cargo clippy started complaining about these lints. I just applied the automatic suggestions from Clippy to fix the warnings.

KyrylR · 2026-03-17T12:45:00Z

Cargo.toml

 transmute_ptr_to_ptr = "warn"
 trivially_copy_pass_by_ref = "warn"
-unchecked_duration_subtraction = "warn"
+unchecked_time_subtraction = "warn"


The solution is the same as in the previous answer.

KyrylR · 2026-03-17T12:46:08Z

src/lexer.rs

        just("->").to(Token::Arrow),
        just("=>").to(Token::FatArrow),
        just("=").to(Token::Eq),
+        just("::").to(Token::DoubleColon),


This is confusing, is param:::: now valid?

No. It's working as usual; the error is being shown by jet::::. So I think it's behaving similarly to param::::.

stringhandler

4fe0378
Seems fine but would be useful to have some tests that show the problem. Have a look here for a simple way to create one https://github.com/BlockstreamResearch/SimplicityHL/pull/213/changes#diff-5d5fd55cd63fa75e4c8902fa6b6d63da3f8da03742454636a8d9c092f6f6eeb4R346

LesterEvSe · 2026-03-17T14:12:41Z

4fe0378 Seems fine but would be useful to have some tests that show the problem. Have a look here for a simple way to create one https://github.com/BlockstreamResearch/SimplicityHL/pull/213/changes#diff-5d5fd55cd63fa75e4c8902fa6b6d63da3f8da03742454636a8d9c092f6f6eeb4R346

Done

stringhandler

24f4e44

I think let's add some tests for witness:: and witness::::

stringhandler · 2026-03-17T14:18:10Z

src/lexer.rs

I think we can remove the double colons here now. The correct lexer should create [Token::Witness, Token::DoubleColon]

Is this OK?

let jet = just("jet") .padded() .ignore_then(just("::")) .padded() .ignore_then(text::ident()) .map(Token::Jet); let witness = just("witness") .padded() .ignore_then(just("::")) .padded() .ignore_then(text::ident()) .map(Token::Witness); let param = just("param") .padded() .ignore_then(just("::")) .padded() .ignore_then(text::ident()) .map(Token::Param);

Or do I need to change this part of the code?

let keyword = text::ident().map(|s| match s { "fn" => Token::Fn, "let" => Token::Let, "type" => Token::Type, "mod" => Token::Mod, "const" => Token::Const, "match" => Token::Match, "true" => Token::Bool(true), "false" => Token::Bool(false), _ => Token::Ident(s), });

If I add these kinds of words here, then I need to use Token::Jet("") or delete the (&'src str) payload here:

// Jets, witnesses, and params Jet(&'src str), Witness(&'src str), Param(&'src str),

So, which approach should I choose?

I chose the option with the fewest changes (.padded(), ignore_then("::"))

So those are valid now

jet :: add_32, witness :: PK, and witness:: PK

?

Yes, it's an analogy to the Rust syntax.

KyrylR · 2026-03-19T13:32:58Z

src/lexer.rs

+    let jet = just("jet")
+        .padded()
+        .ignore_then(just("::"))
+        .padded()


This is a breaking change in respect to the previous grammar

Let's not add .padded(), so it becomes like this

let jet = just("jet") .then_ignore(just("::")) .ignore_then(text::ident()) .map(Token::Jet);

Or do you have a reasoning why .padded() is needed?

Yes, it is a breaking change. We don't have it in version 0.4.1, so I removed it in the last commit.

delta1 · 2026-03-24T10:12:26Z

@KyrylR @stringhandler can you please re-review?

KyrylR · 2026-03-24T10:30:40Z

LGTM, needs rebase

KyrylR

ACK e11d27a ran tests locally

KyrylR · 2026-03-24T10:58:49Z

In regard to #249,

Naturally, we are going to have quite a few deviations from 0.4.1, though even though technically this is a breaking change, I would not bump it to 0.6.0-rc.0 (unless there is a bug that makes the current state of parsing wrong)

What we will need to figure out is how to support follow-up versions of the compiler, as well as how to make feature rollouts easier (without the fear that something will break)

stringhandler · 2026-03-24T11:26:33Z

I have run this branch on all the scripts in every-simplicity and there are no changes. I agree that this is not breaking.

For feature rollouts, I can suggest -Z "<unstable-feature-name>" for allowing users to try it, as seen in #232

stringhandler

ACK e11d27a

LesterEvSe requested a review from KyrylR March 17, 2026 12:39

LesterEvSe self-assigned this Mar 17, 2026

LesterEvSe requested a review from delta1 as a code owner March 17, 2026 12:39

LesterEvSe added the bug Something isn't working label Mar 17, 2026

KyrylR requested changes Mar 17, 2026

View reviewed changes

stringhandler reviewed Mar 17, 2026

View reviewed changes

LesterEvSe force-pushed the fix/double-colon branch 2 times, most recently from 917a87a to 24f4e44 Compare March 17, 2026 14:12

LesterEvSe requested review from KyrylR and stringhandler March 17, 2026 14:12

stringhandler reviewed Mar 17, 2026

View reviewed changes

LesterEvSe force-pushed the fix/double-colon branch from 24f4e44 to 9ae7f36 Compare March 17, 2026 15:09

LesterEvSe requested a review from stringhandler March 17, 2026 15:09

KyrylR reviewed Mar 19, 2026

View reviewed changes

LesterEvSe force-pushed the fix/double-colon branch from 9ae7f36 to 9ee12f7 Compare March 20, 2026 13:53

LesterEvSe requested a review from KyrylR March 20, 2026 13:58

fix: add a DoubleColon to the Token struct in the lexer.rs

e11d27a

LesterEvSe force-pushed the fix/double-colon branch from 9ee12f7 to e11d27a Compare March 24, 2026 10:36

KyrylR approved these changes Mar 24, 2026

View reviewed changes

stringhandler approved these changes Mar 24, 2026

View reviewed changes

KyrylR merged commit 682d829 into BlockstreamResearch:master Mar 24, 2026
11 checks passed

		/// This prevents the lexer from allowing spaces between colons (e.g., `use a: :b`),
		/// ensuring we strictly parse valid paths.

Conversation

LesterEvSe commented Mar 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stringhandler left a comment

Choose a reason for hiding this comment

Uh oh!

LesterEvSe commented Mar 17, 2026

Uh oh!

stringhandler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LesterEvSe Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

delta1 commented Mar 24, 2026

Uh oh!

KyrylR commented Mar 24, 2026

Uh oh!

KyrylR left a comment

Choose a reason for hiding this comment

Uh oh!

KyrylR commented Mar 24, 2026

Uh oh!

stringhandler commented Mar 24, 2026

Uh oh!

stringhandler left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

LesterEvSe Mar 17, 2026 •

edited

Loading