Skip to content

TokenStream::from_str sees (/*ERROR*/) as a literal rather than an empty group #510

@maxdexh

Description

@maxdexh

The error marker (/*ERROR*/) is parsed as a literal, rather than a parenthesized comment.

To demonstrate, the following code:

println!("{:#?}", proc_macro2::TokenStream::from_str("(/*ERROR*/)").unwrap());

outputs TokenStream [Literal { lit: (/*ERROR*/) }]
while the same code with proc_macro would output something like TokenStream [Group { delimiter: Parenthesis, stream: TokenStream [], span: ... }].

As a consequence of the inconsistency, the false literal TokenTree will cause a panic when it is passed to wrapper::into_compiler_token. For example, running this in a proc macro:

proc_macro2::fallback::force();
let bad = proc_macro2::TokenStream::from_str("(/*ERROR*/)").unwrap();
proc_macro2::fallback::unforce();
let _ = proc_macro2::TokenStream::from_iter(bad); // panic: compiler/fallback mismatch L934

Version: 1.0.95 on stable 1.88.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions