Champion: "Permit surrogate pairs and wide Unicode-escaped code points in identifiers"

[gafter]

• Proposal added: (none needed; the C# language specification already supports this)
• Discussed in LDM
• Decision in LDM
• Finalized (done, rejected, inactive)
• Spec'ed

The C# language specification already requires that we accept Unicode letters (etc) in identifiers. But the Roslyn compiler doesn't support Unicode letters higher than U+10000 (dotnet/roslyn#9731), or those that are represented as a Unicode escape sequence for such a code point (dotnet/roslyn#13560). We could treat it as a bug and fix it, but that fix probably needs to be tied to a language version change to prevent a backward breaking change. This championed issue is to track the fix (as a language change) in some language version.

[jcouv update:] this restriction is documented as a known/expected/explicit decision from roslyn.

[svick]

How would allowing this break backwards compatibility?

[alrz]

@svick since only a later version of the compiler would be able to parse those identifiers, this should be tied to a lang version to not silently break code in older versions (i.e. this is an observable "bug fix")

[gafter]

@svick A source file that successfully compiles using language version 6 on a recent compiler should also compile using an older version of the compiler in the same language version. Similarly, APIs described in an assembly that was compiled with a recent compiler using language version 6 should be usable by an older compiler using language version 6. Both of these would be violated by permitting a newer compiler to accept identifiers that were rejected by previous compilers.

[ufcpp]

@svick
https://github.com/dotnet/roslyn/blob/master/docs/compilers/CSharp/Unicode%20Version.md
Unicode sometimes has breaking changes.

[Joe4evr]

As a prime example: Ask Jon Skeet about the Mongolian Vowel Separator. 🙃

[ghord]

Since this is touching Unicode support, wouldn't it be nice if C# could support wider range of characters in identifiers? I personally don't care about emoji, but it would be nice if the following code would compile:

int Sum(int x₁, int x₂)
{
    return x₁ + x₂;
}

[ufcpp]

@ghord
Do you want any characters to be supported? or do you have any rule?
Many languages - C#, Java, Go, Python, Elixir, etc. - support Unicode identifiers as defined in the Unicode Annex #31.

[ghord]

@ufcpp
There are some other languages with more relaxed definitions like JavaScript and Swift (Fragment of Swift spec with ranges defined). That would be a good place to start.

[ufcpp]

@ghord
Yes but... Relaxation is not necessarily good. For instance, the following code is valid in Swift. I personally don't like this identifier rule.

import Foundation

var ⁡ = 2 // function apply
var ⁢ = 3 // invisible times
var ⁣ = 5 // invisible separator
var ⁤ = 7 // invisible plus
print(⁡ * ⁢ * ⁣ * ⁤)

var 𝟎 = 2 // bold
var 𝟘 = 3 // double-struck
var 𝟢 = 5 // sans-serif
var 𝟬 = 7 // sans-serif bold
var 𝟶 = 11 // mono-space
print(𝟎 * 𝟘 * 𝟢 * 𝟬 * 𝟶)

prefix operator ∑ // 'N-ARY SUMMATION' (U+2211)

prefix func ∑ (array : [Int]) -> Int {
    var sum : Int = 0
    for item in array {
        sum += item
    }
    return sum
}

let a = [ 1, 2, 3, 4, 5]
let Σa = 0 // 'GREEK CAPITAL LETTER SIGMA' (U+03A3) + a

print(∑a) // 15
print(Σa) // 0

[ghord]

@ufcpp I admit it looks confusing when you use characters like these, but it's alreay possible in current language

            var а = 2; // U+0430
            var a = 3; // ASCII a

            var b = а + a; // 5

The thing is, nobody writes code like this. I would not consider it a strong argument against the changes.

[gafter]

@ghord If you'd like the definition of identifier to change, please lobby for that change in the Unicode spec rather than the C# spec. We are unlikely to depart from the Unicode spec further than we already do (we treat _ as a letter).

[gafter]

This championed issue is not a wish list of changes to the spec. It is about implementing what is already in the spec. If you want to propose changes to the specification, please open new issues.