JuliaSyntax tokenize.jl: don‘t hardcode unicode ops range (#60198)

pthariensflame · web-flow · commit 22cfa2bdddb1 · 2025-11-21T23:18:46.000-05:00
There are very nice `(BEGIN|END)_UNICODE_OPS` kinds; they should be used! --- Ported from JuliaLang/JuliaSyntax.jl#404; thanks are due to @c42f for the notification and instructions on how to move this.
diff --git a/JuliaSyntax/src/julia/tokenize.jl b/JuliaSyntax/src/julia/tokenize.jl
@@ -173,7 +173,7 @@ function optakessuffix(k)
         k == K"!"   ||
         k == K".'"  ||
         k == K"->"  ||
-        K"¬" <= k <= K"∜"
+        K"BEGIN_UNICODE_OPS" <= k <= K"END_UNICODE_OPS"
     )
 end
 

Original file line number	Diff line number	Diff line change
`@@ -173,7 +173,7 @@ function optakessuffix(k)`
`173`	`173`	`k == K"!" \|\|`
`174`	`174`	`k == K".'" \|\|`
`175`	`175`	`k == K"->" \|\|`
`176`		`- K"¬" <= k <= K"∜"`
	`176`	`+ K"BEGIN_UNICODE_OPS" <= k <= K"END_UNICODE_OPS"`
`177`	`177`	`)`
`178`	`178`	`end`
`179`	`179`