Update GDShader grammar to support at least preprocessor keyword syntax and strings, like `#include "..."`

[AlfishSoftware]

Godot version

4.4.stable

VS Code version

VSCodium 1.100.33714

Godot Tools VS Code extension version

2.5.1

System information

Kubuntu 25.04 x64

Problem statement

GitHub now seems to support GDShader (see github-linguist/linguist#7071 and github-linguist/linguist#7067) 🎉
It seems to use the grammar from this extension, which is great!

But the official grammar is not quite complete for Godot 4 without syntax-coloring preprocessor directives and strings.
Look at how Github renders an example using the gdshader lang id in markdown triple-fences:

shader_type spatial;
#include "./file.gdshaderinc"
#define PI 3.14
#if false
#endif
uniform vec4 albedo : source_color;
void fragment() {
}

Compare the preprocessor directives to how they look on c language id:

#include "./file.gdshaderinc"
#define PI 3.14
#if false
#endif

The same can be seen in VSCode (when my Godot Files plugin is disabled or not installed).

Proposed solution

@DaelonSuzuka I'd suggest merging stuff from the latest version of my grammar again, or at least enough so it colors correctly:

• preprocessor directives like #include, #define, #if, etc.
• more importantly, strings, used in #include "..." and : hint_enum("...", "...")

Note that if you want to follow just a specific folder or file in Github for updates, you can use a RSS feed (e.g. in Thunderbird or a similar program), so you catch updates soon without having to be @ pinged there, e.g.:

• https://github.com/AlfishSoftware/godot-files-vscode/commits/master/lang.gdshader/gdshader.tmLanguage.json.atom

That's what I'm doing to follow these:

• https://github.com/godotengine/godot/commits/master/servers/rendering/shader_language.cpp.atom
• https://github.com/godotengine/godot/commits/master/servers/rendering/shader_preprocessor.cpp.atom

Test files

Here's a pair of test files mostly copied from the docs for the preprocessor on Godot 4.0, I think it has all preprocessor directives.
I also added a hint_enum usage.

fancy_color.gdshaderinc

// While technically allowed, there is usually no `shader_type` declaration in include files.

vec3 get_fancy_color() {
    return vec3(0.3, 0.6, 0.9);
}

uniform int character_speed: hint_enum("Slow:30", "Average:60", "Very Fast:200") = 60;

docs_preprocessor.gdshader

shader_type spatial;

// Notice the lack of semicolon at the end of the line, as the replacement text
// shouldn't insert a semicolon on its own.
// If the directive ends with a semicolon, the semicolon is inserted in every usage
// of the directive, even when this causes a syntax error.
#define USE_MY_COLOR
#define MY_COLOR vec3(1, 0 ## .0, 0)

// Replacement with arguments.
// All arguments are required (no default values can be provided).
#define BRIGHTEN_COLOR(r, g, b) vec3(r + 0.5, g + 0.5, b + 0.5)

// Multiline replacement using backslashes for continuation:
#define SAMPLE(param1, param2, param3, param4) long_function_call( \
        p_##param1, \
        p_ ##param2, \
        p_## param3, \
        p_ ## param4 \
)
group_uniforms my_group;
/// Description
uniform sampler2D yy : hint_default_transparent;
group_uniforms MyGroup.MySubgroup;
/** Other Description */
uniform sampler2D xx : hint_default_white;
group_uniforms;
uniform sampler2D zz : hint_default_transparent;
void fragment() {
#ifdef USE_MY_COLOR
    ALBEDO = MY_COLOR;
#endif
}

// ===

#undef MY_COLOR
#define MY_COLOR vec3(1, 0, 0)

vec3 get_red_color() {
    return MY_COLOR;
}

#undef MY_COLOR
#define MY_COLOR vec3(0, 1, 0)

vec3 get_green_color() {
    return MY_COLOR;
}

// Like in most preprocessors, undefining a define that was not previously defined is allowed
// (and won't print any warning or error).
#undef THIS_DOES_NOT_EXIST

// ===

#define VAR 3
#define USE_LIGHT 0 // Evaluates to `false`.
#define USE_COLOR 1 // Evaluates to `true`.

#if VAR == 3 && (USE_LIGHT || USE_COLOR)
// Condition is `true`. Include this portion in the final shader.
#endif

// ===

// Correct syntax:
#if defined(USE_LIGHT) || defined(USE_COLOR) || !defined(USE_REFRACTION)
// Condition is `true`. Include this portion in the final shader.
#endif

// ===

#if VAR == 0
// Not included.
#elif VAR == 1
// Not included.
#elif VAR == 2
// Condition is `true`. Include this portion in the final shader.
#else
// Not included.
#endif

// ===

#define SHADOW_QUALITY_MEDIUM

#ifdef SHADOW_QUALITY_HIGH
// High shadow quality.
#else
#ifdef SHADOW_QUALITY_MEDIUM
// Medium shadow quality.
#else
// Low shadow quality.
#endif // This ends `SHADOW_QUALITY_MEDIUM`'s branch.
#endif // This ends `SHADOW_QUALITY_HIGH`'s branch.

#ifndef USE_LIGHT
// Evaluates to `false`. This portion won't be included in the final shader.
#endif

#ifndef USE_COLOR
// Evaluates to `true`. This portion will be included in the final shader.
#endif

// ===

// material.gdshader

//shader_type spatial;

#include "./fancy_color.gdshaderinc"

void fragment() {
    // No error, as we've included a definition for `get_fancy_color()` via the shader include.
    COLOR = get_fancy_color();
}

// ===

#pragma disable_preprocessor

/*
//#if USE_LIGHT
// This would cause a shader compilation error, as the `#if USE_LIGHT` and `#endif`
// are included as-is in the final shader code.
//#endif
*/

Note

I'm picking "enhancement", I dunno if it should be "bug".