Replace JsonWebToken ReadPayloadValue with a delegate

[pmaytak]

Fixes #2982

Updated perf results: #2981 (comment)

This pull request introduces several updates and improvements to the Microsoft.IdentityModel.JsonWebTokens library, focusing on enhancing the handling of token payloads and custom claims. The most important changes include adding new constructors to the JsonWebToken class, updating the JsonClaimSet initialization, and introducing a new delegate for reading custom token payload values.

Enhancements to JsonWebToken:

• Added new constructors to the JsonWebToken class to support initializing tokens with custom delegates for reading token payload values. (src/Microsoft.IdentityModel.JsonWebTokens/JsonWebToken.cs) [1] [2] [3]
• Introduced the ReadTokenPayloadValueDelegates property to the JsonWebToken class, allowing custom handling of specific claim names during token reading. (src/Microsoft.IdentityModel.JsonWebTokens/JsonWebToken.cs)

Updates to TokenValidationParameters:

• Added the ReadTokenPayloadValueDelegates property to the TokenValidationParameters class to support custom claim handling during token validation. (src/Microsoft.IdentityModel.Tokens/TokenValidationParameters.cs) [1] [2]
• Updated the copy constructor of TokenValidationParameters to include the new ReadTokenPayloadValueDelegates property. (src/Microsoft.IdentityModel.Tokens/TokenValidationParameters.cs)

Refactoring and cleanup:

• Refactored the ReadPayloadValue method in JsonWebToken.PayloadClaimSet to simplify the handling of standard claims and integrate custom delegate handling. (src/Microsoft.IdentityModel.JsonWebTokens/Json/JsonWebToken.PayloadClaimSet.cs) [1] [2]
• Removed the CustomJsonWebToken class from the tests, as its functionality is now covered by the new delegate-based approach. (test/Microsoft.IdentityModel.JsonWebTokens.Tests/CustomJsonWebToken.cs)

Delegate introduction:

• Introduced the ReadTokenPayloadValueDelegate delegate to handle custom claim reading during token payload processing. (src/Microsoft.IdentityModel.Tokens/Delegates.cs)

[pmaytak]

Edit: These results are outdated.

Looks like when using delegates there're extra allocations because of:

string claimName = reader.GetString();
claims[claimName] = ReadTokenPayloadValueDelegate(ref reader, claimName);

Delegates	Method	Mean	Ratio	Error	StdDev	P90	P95	P100	Gen0	Allocated	Alloc Ratio
Without	JsonWebTokenHandler_ValidateTokenAsyncWithTVP	30.72 μs	1.000	0.026 μs	0.055 μs	30.79 μs	30.81 μs	30.89 μs	0.2441	7.23 KB	1.00
With	JsonWebTokenHandler_ValidateTokenAsyncWithTVP	30.66 μs	0.998	0.124 μs	0.273 μs	30.96 μs	31.02 μs	31.10 μs	0.3052	7.55 KB	1.044

Delegates	Method	Mean	Ratio	Error	StdDev	P90	P95	P100	Gen0	Allocated	Alloc Ratio
Without	ReadJWS_FromMemory	8.373 μs	1.00	0.0331 μs	0.0727 μs	8.465 μs	8.499 μs	8.530 μs	0.2289	5.84 KB	1.00
With	ReadJWS_FromMemory	8.201 μs	0.979	0.0287 μs	0.0636 μs	8.295 μs	8.313 μs	8.357 μs	0.2441	6.07 KB	1.039

[Copilot]

Copilot reviewed 5 out of 9 changed files in this pull request and generated no comments.

Files not reviewed (4)

• src/Microsoft.IdentityModel.JsonWebTokens/InternalAPI.Unshipped.txt: Language not supported
• src/Microsoft.IdentityModel.Tokens/InternalAPI.Unshipped.txt: Language not supported
• test/Microsoft.IdentityModel.JsonWebTokens.Tests/CustomJsonWebToken.cs: Evaluated as low risk
• src/Microsoft.IdentityModel.JsonWebTokens/Json/JsonWebToken.PayloadClaimSet.cs: Evaluated as low risk

Comments suppressed due to low confidence (1)

src/Microsoft.IdentityModel.JsonWebTokens/Json/JsonClaimSet.cs:31

• The initialization of _jsonClaims should be 'new Dictionary<string, object>()' instead of '[]'.

_jsonClaims = [];

[pmaytak]

Edit: These results are outdated.

Updated results comparing delegates which have a dictionary as a parameter. Ran JsonWebTokenHandler_ValidateTokenAsyncWithTVP and ReadJWS_FromMemory with extended claims. These test using the default delegate implemenation.

Delegates	Method	Mean	Ratio	Error	StdDev	P90	P95	P100	Gen0	Allocated	Alloc Ratio
Without	JsonWebTokenHandler_ValidateTokenAsyncWithTVP	30.13 μs	1.00	0.169 μs	0.374 μs	30.61 μs	30.74 μs	31.07 μs	0.2441	7.23 KB	1.00
With	JsonWebTokenHandler_ValidateTokenAsyncWithTVP	29.34 μs	0.974	0.085 μs	0.188 μs	29.59 μs	29.61 μs	29.87 μs	0.2441	7.33 KB	1.014

Delegates	Method	Mean	Ratio	Error	StdDev	P90	P95	P100	Gen0	Allocated	Alloc Ratio
Without	ReadJWS_FromMemory	7.523 μs	1.00	0.0333 μs	0.0724 μs	7.599 μs	7.612 μs	7.672 μs	0.2365	5.84 KB	1.00
With	ReadJWS_FromMemory	7.059 μs	0.938	0.0416 μs	0.0905 μs	7.160 μs	7.170 μs	7.205 μs	0.2365	5.84 KB	1.00

BenchmarkDotNet v0.13.12, Windows 11 (10.0.26100.2605) (Hyper-V)
Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK 9.0.101
[Host] : .NET 8.0.11 (8.0.1124.51707), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MediumRun : .NET 8.0.11 (8.0.1124.51707), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI

[github-actions]

Summary

Summary

Generated on:	2/11/2025 - 8:47:25 AM
Coverage date:	2/11/2025 - 8:37:55 AM - 2/11/2025 - 8:47:00 AM
Parser:	MultiReport (60x Cobertura)
Assemblies:	1
Classes:	7
Files:	2
Line coverage:	80.3% (620 of 772)
Covered lines:	620
Uncovered lines:	152
Coverable lines:	772
Total lines:	483
Branch coverage:	67.8% (228 of 336)
Covered branches:	228
Total branches:	336
Method coverage:	Feature is only available for sponsors

Coverage

Microsoft.IdentityModel.JsonWebTokens - 80.3%

Name	Line	Branch
Microsoft.IdentityModel.JsonWebTokens	80.3%	67.8%
Microsoft.IdentityModel.JsonWebTokens.JwtTokenUtilities	100%
System.Text.RegularExpressions.Generated	80.3%	67.8%
System.Text.RegularExpressions.Generated	80.3%	67.8%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F12A1AEDDDFE32BA DF4DBFF323AF1BCB48B9F9721B7CD3E05F5E034CF225E3DF8__CreateJweRegex_1	79.2%	68%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F12A1AEDDDFE32BA DF4DBFF323AF1BCB48B9F9721B7CD3E05F5E034CF225E3DF8__CreateJwsRegex_0	81.4%	67.6%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F334844C618E00D3 CEC5D3FE0D00CF0141BBEE98635313BB2CB8D3921464CE05A__CreateJweRegex_1	79.2%	68%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F334844C618E00D3 CEC5D3FE0D00CF0141BBEE98635313BB2CB8D3921464CE05A__CreateJwsRegex_0	81.4%	67.6%

[github-actions]

Summary

Summary

Generated on:	2/11/2025 - 9:00:21 AM
Coverage date:	2/11/2025 - 8:50:12 AM - 2/11/2025 - 8:59:54 AM
Parser:	MultiReport (60x Cobertura)
Assemblies:	1
Classes:	7
Files:	2
Line coverage:	80.3% (620 of 772)
Covered lines:	620
Uncovered lines:	152
Coverable lines:	772
Total lines:	483
Branch coverage:	67.8% (228 of 336)
Covered branches:	228
Total branches:	336
Method coverage:	Feature is only available for sponsors

Coverage

Microsoft.IdentityModel.JsonWebTokens - 80.3%

Name	Line	Branch
Microsoft.IdentityModel.JsonWebTokens	80.3%	67.8%
Microsoft.IdentityModel.JsonWebTokens.JwtTokenUtilities	100%
System.Text.RegularExpressions.Generated	80.3%	67.8%
System.Text.RegularExpressions.Generated	80.3%	67.8%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F12A1AEDDDFE32BA DF4DBFF323AF1BCB48B9F9721B7CD3E05F5E034CF225E3DF8__CreateJweRegex_1	79.2%	68%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F12A1AEDDDFE32BA DF4DBFF323AF1BCB48B9F9721B7CD3E05F5E034CF225E3DF8__CreateJwsRegex_0	81.4%	67.6%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F334844C618E00D3 CEC5D3FE0D00CF0141BBEE98635313BB2CB8D3921464CE05A__CreateJweRegex_1	79.2%	68%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F334844C618E00D3 CEC5D3FE0D00CF0141BBEE98635313BB2CB8D3921464CE05A__CreateJwsRegex_0	81.4%	67.6%

[brentschmaltz]

It would be great to have the same design when reading the header.

[brentschmaltz]

We need a way to allow the user to parse all claims in a performant manner.
This design skips claims that are parsed.

Allowing the user to parse all claims allows for an implementation where validation could fail fast.

Failing fast needs to have a design that can propagate a detailed error result to upper layers for error reporting.

[github-actions]

Summary

Summary

Generated on:	3/24/2025 - 5:04:56 AM
Coverage date:	3/24/2025 - 4:55:10 AM - 3/24/2025 - 5:04:29 AM
Parser:	MultiReport (60x Cobertura)
Assemblies:	1
Classes:	7
Files:	2
Line coverage:	80.3% (620 of 772)
Covered lines:	620
Uncovered lines:	152
Coverable lines:	772
Total lines:	483
Branch coverage:	67.8% (228 of 336)
Covered branches:	228
Total branches:	336
Method coverage:	Feature is only available for sponsors

Coverage

Microsoft.IdentityModel.JsonWebTokens - 80.3%

Name	Line	Branch
Microsoft.IdentityModel.JsonWebTokens	80.3%	67.8%
Microsoft.IdentityModel.JsonWebTokens.JwtTokenUtilities	100%
System.Text.RegularExpressions.Generated	80.3%	67.8%
System.Text.RegularExpressions.Generated	80.3%	67.8%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F12A1AEDDDFE32BA DF4DBFF323AF1BCB48B9F9721B7CD3E05F5E034CF225E3DF8__CreateJweRegex_1	79.2%	68%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F12A1AEDDDFE32BA DF4DBFF323AF1BCB48B9F9721B7CD3E05F5E034CF225E3DF8__CreateJwsRegex_0	81.4%	67.6%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F334844C618E00D3 CEC5D3FE0D00CF0141BBEE98635313BB2CB8D3921464CE05A__CreateJweRegex_1	79.2%	68%
System.Text.RegularExpressions.Generated.<RegexGenerator_g>F334844C618E00D3 CEC5D3FE0D00CF0141BBEE98635313BB2CB8D3921464CE05A__CreateJwsRegex_0	81.4%	67.6%

[pmaytak]

Edit: These results are outdated. See update in the most recent post.

The three scenarios compared below:

• Dev branch, JWT has custom claims, these are saved as string JSON.
• This feature branch, JWT has custom claims, no delegates are set, claims are saved as strings.
• This feature branch, JWT has custom claims and delegates which parse them into an object.
• Same cases as above, also creates claims.

Branch	Method	Mean	Ratio	Error	StdDev	P90	P95	P100	Gen0	Allocated	Alloc Ratio
Base	JsonWebTokenHandler_ValidateTokenAsync	30.59 μs	1.000	0.037 μs	0.079 μs	30.68 μs	30.72 μs	30.85 μs	0.4272	7.51 KB	1.000
Feature	JsonWebTokenHandler_ValidateTokenAsync	30.61 μs	1.000	0.068 μs	0.145 μs	30.84 μs	30.88 μs	31.00 μs	0.4272	7.94 KB	1.057
Feature	JsonWebTokenHandler_ValidateTokenAsyncWithDelegates	31.36 μs	1.025	0.065 μs	0.142 μs	31.51 μs	31.59 μs	31.94 μs	0.4883	8 KB	1.065
Base	JsonWebTokenHandler_ValidateTokenAsync_CreateClaims	35.38 μs	1.000	0.130 μs	0.286 μs	35.80 μs	35.88 μs	36.10 μs	1.0376	17.3 KB	1.000
Feature	JsonWebTokenHandler_ValidateTokenAsync_CreateClaims	34.88 μs	0.986	0.123 μs	0.260 μs	35.24 μs	35.37 μs	35.71 μs	1.0376	17.73 KB	1.025
Feature	JsonWebTokenHandler_ValidateTokenAsyncWithDelegates_CreateClaims	35.42 μs	1.001	0.072 μs	0.158 μs	35.62 μs	35.72 μs	35.80 μs	0.9766	17.79 KB	1.028

[pmaytak]

Edit: These results are outdated. See update in the most recent post.

The difference in code from the previous perf results is that this time the delegates are null by default (and not set to new dictionary) which reduced the allocations from the previous run. Latest commit: 3c2fbbc

The three scenarios compared below:

• Dev branch, JWT has custom claims, these are saved as string JSON.
• This feature branch, JWT has custom claims, no delegates are set, claims are saved as strings.
• This feature branch, JWT has custom claims and delegates which parse them into an object.
• Same cases as above, also creates claims.

Branch	Method	Mean	Ratio	Error	StdDev	P90	P95	P100	Gen0	Allocated	Alloc Ratio
Base	JsonWebTokenHandler_ValidateTokenAsync	30.59 μs	1.000	0.037 μs	0.079 μs	30.68 μs	30.72 μs	30.85 μs	0.4272	7.51 KB	1.000
Feature	JsonWebTokenHandler_ValidateTokenAsync	30.29 μs	0.990	0.061 μs	0.131 μs	30.47 μs	30.51 μs	30.65 μs	0.4272	7.63 KB	1.016
Feature	JsonWebTokenHandler_ValidateTokenAsyncWithDelegates	31.16 μs	1.019	0.099 μs	0.217 μs	31.51 μs	31.54 μs	31.59 μs	0.4272	7.77 KB	1.035
Base	JsonWebTokenHandler_ValidateTokenAsync_CreateClaims	35.38 μs	1.000	0.130 μs	0.286 μs	35.80 μs	35.88 μs	36.10 μs	1.0376	17.3 KB	1.000
Feature	JsonWebTokenHandler_ValidateTokenAsync_CreateClaims	34.65 μs	0.979	0.074 μs	0.162 μs	34.84 μs	34.93 μs	35.03 μs	0.9766	17.41 KB	1.006
Feature	JsonWebTokenHandler_ValidateTokenAsyncWithDelegates_CreateClaims	35.70 μs	1.009	0.083 μs	0.180 μs	35.86 μs	35.96 μs	36.47 μs	1.0376	17.55 KB	1.008