[Bug]: Hit LNK1237 when using IntrinsicLib.inf

[JonsonTsai1]

Is there an existing issue for this?

• I have searched existing issues

Bug Type

• Firmware
• Tool
• Unit Test

What packages are impacted?

MdePkg, CryptoPkg

Which targets are impacted by this bug?

DEBUG, RELEASE

Current Behavior

My DXE driver will hit LNK1237 error when using IntrinsicLib.

BaseIntrinsicLib.lib(MemoryIntrinsics.obj) : fatal error LNK1237: during code generation, compiler introduced reference to symbol 'CompareMem' defined in module 'BaseMemoryLibRepStr.lib(CompareMemWrapper.obj)' compiled with /GL

Finished generating code
NMAKE : fatal error U1077: 'C:\BEA\BuildTools\VS2019_5d18eea\VC\bin\Hostx86\x64\link.exe' : return code '0x4d5'
Stop.

The problem is raised because IntrinsicLib.inf overwrites CC_FLAGS to remove /GL while it references to BaseMemoryLib which is built with /GL flag.

https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-error-lnk1237?view=msvc-170

I've checking why /GL f lag is removed in the log. However it gets me further confused: f754e61

It said the modification is to:

1. enable /GL optimization building on OpensslLib.
2. add DDK3790 compiler option for IA32.

However the code is inconsistent with the change in IntrinsicLib.inf

Expected Behavior

Build process should be completed without stop.

Steps To Reproduce

In your DXE driver, C code, add the following lines:

static UINT8 array[20] = { 0 }

This will introduce memset() call by C compiler.

In the INF file, add IntrinsicLib into [LibraryClasses]

build your module....then LNK1237 error hit.

Build Environment

- OS(s): Windows 11
- Tool Chain(s): VS2019

Version Information

Commit: EDK2 later than f754e61

Urgency

Medium

Are you going to fix this?

Someone else needs to fix it

Do you need maintainer feedback?

Maintainer feedback requested

Anything else?

No response

[JonsonTsai1]

A temporary solution is to add the following line in your C code:

• #pragma comment(linker, "/include:memset")

[lgao4]

@JonsonTsai1 , could you give the full log and code base version?

@liyi77 , could you help check this build failure about Openssl?

[JonsonTsai1]

EDK2_Build_CryptoPkg.txt

This is the build log in Windows environment. https://github.com/JonsonTsai1/edk2/tree/IntrinsicLib/LNK1237
Steps:

1. Open a command prompt at EDK2 folder
2. Modify Config\Target.txt
3. Modify ACTIVE_PLATFORM = CryptoPkg/CryptoPkg.dsc
4. Run Edk2Setup.bat Rebuild
5. Run Edk2Setup.bat
6. Run VS2022.bat or step 7,8,9
7. Run set PYTHON_HOME=C:\Program Files\Python310
8. Run set NASM_PREFIX=C:\BEA\AgS_2024\DellPkgs\BuildTools\NASM\
9. Run set IASL_PREFIX=C:\BEA\AgS_2024\DellPkgs\BuildTools\ASL\
10. Build
11. You could find LNK1237 error
12. You could repeatedly duplicate this error by "build -m MdeModulePkg/Application/HelloWorld/HelloWorld.inf"

[liyi77]

Are you trying to use IntrinsicLib outside of Crypto libraries? This lib was designed to serve OpenSSL and is not recommended for other uses.
If you really need it, please make sure the entire driver or lib turns off whole program optimization (/GL).

[JonsonTsai1]

No. We actually use Crypto libraries which indirectly includes IntrinsicLib.
The branch I provided was just a demonstration to the issue.