how to debug build hip-clr and rocr-runtime

[robert-g-liu]

I was running below command to build a minimal system:

make -B build-dir -GNinja .
-DTHEROCK_ENABLE_ALL=OFF
-DTHEROCK_ENABLE_HIP_RUNTIME=ON
-DCMAKE_BUILD_TYPE=RelWithDebInfo
-DTHEROCK_AMDGPU_FAMILIES=gfx1036
-DCMAKE_INSTALL_PREFIX=/home/USER/TheRock/install-dir-dbg

However, I would like to build hipclr and ROCR-runtime sub projects with full debug capability. After trying several ways or hacks, I still couldn't make it. Any idea?

[ScottTodd]

The docs at https://github.com/ROCm/TheRock/blob/main/docs/development/development_guide.md#per-project-cmake-flags may be helpful. You can override the build type for each subproject. For those in particular you would set -DROCR-Runtime_BUILD_TYPE=Debug -Dhip-clr_BUILD_TYPE=Debug. The configure logs should show this message if you set it correctly:

TheRock/cmake/therock_subproject.cmake

Lines 747 to 754 in 5c520ac

	# Derive the CMAKE_BUILD_TYPE from either {project}_BUILD_TYPE or the global
	# CMAKE_BUILD_TYPE.
	set(_cmake_build_type "${${target_name}_BUILD_TYPE}")
	if(NOT _cmake_build_type)
	set(_cmake_build_type "${CMAKE_BUILD_TYPE}")
	else()
	message(STATUS " PROJECT SPECIFIC CMAKE_BUILD_TYPE=${_cmake_build_type}")
	endif()

then you should also see:

• The build flag specified in build/core/hip-clr/_init.cmake
• The build type reported in per-project logs at paths like build/logs/hip-clr_configure.log

The docs at https://github.com/ROCm/TheRock/blob/main/docs/development/development_guide.md#tips-for-using-vscode also show an example of how to attach a debugger to a binary built in this way

[robert-g-liu]

Thanks, the build worked but the hip_saxpy compiled with this build failed to run but reported the missing library hsa-amd-aqlprofile

LoadLib(libhsa-amd-aqlprofile64.so) failed: libhsa-amd-aqlprofile64.so: cannot open shared object file: No such file or directory
hip_saxpy: /home/USER/TheRock/rocm-systems/projects/rocr-runtime/runtime/hsa-runtime/image/addrlib/src/gfx10/gfx10addrlib.cpp:1087: virtual rocr::Addr::ChipFamily rocr::Addr::V2::Gfx10Lib::HwlConvertChipFamily(unsigned int, unsigned int): Assertion `false' failed.
Aborted (core dumped)

Actually this happened also when specifying CMAKE_BUILD_TYPE=Debug for the entire super project.

My latest cmake config was

cmake -B build-dir -GNinja .
-DTHEROCK_ENABLE_ALL=OFF
-DTHEROCK_ENABLE_HIP_RUNTIME=ON
-DCMAKE_BUILD_TYPE=MinSizeRel
-DROCR-Runtime_BUILD_TYPE=Debug
-Dhip-clr_BUILD_TYPE=Debug
-DTHEROCK_AMDGPU_FAMILIES=gfx1036
-DCMAKE_INSTALL_PREFIX=/home/USER/TheRock/install-dir-dbg

The debug build for two sub projects added more dependency which seems not needed with release or RelWithDebInfo build.

[robert-g-liu]

ltrace -e dlopen ./hip_saxpy
libamd_comgr_loader.so.1->dlopen("libamd_comgr.so.3", 2) = 0x607cb8ef1bf0
libamd_comgr_loader.so.1->dlopen("libamd_comgr.so.3", 4098) = 0x607cb8ef1bf0
libhsa-runtime64.so.1->dlopen("libhsa-amd-aqlprofile64.so", 1) = 0

[robert-g-liu]

Actually the failure of dlopen with libhsa-amd-aqlprofile64.so had nothing to do with the assertion.

The assertion was from image/addrlib only with DEBUG build.

#if DEBUG
#if defined(GNUC)
#define ADDR_DBG_BREAK() { assert(false); }
#elif defined(APPLE)
#define ADDR_DBG_BREAK() { IOPanic("");}
#else
#define ADDR_DBG_BREAK() { __debugbreak(); }
#endif
#else
#define ADDR_DBG_BREAK()
#endif
////////////////////////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////////////////////////
// Debug assertions used in AddrLib
////////////////////////////////////////////////////////////////////////////////////////////////////
#if defined(_WIN32) && (_MSC_VER >= 1400)
#define ADDR_ANALYSIS_ASSUME(expr) __analysis_assume(expr)
#else
#define ADDR_ANALYSIS_ASSUME(expr) do { (void)(expr); } while (0)
#endif

#if DEBUG
#if defined( _WIN32 )
#define ADDR_ASSERT(__e)
{
ADDR_ANALYSIS_ASSUME(__e);
if ( !((__e) ? TRUE : FALSE)) { ADDR_DBG_BREAK(); }
}
#else
#define ADDR_ASSERT(__e) if ( !((__e) ? TRUE : FALSE)) { ADDR_DBG_BREAK(); }
#endif

#if ADDR_SILENCE_ASSERT_ALWAYS
    #define ADDR_ASSERT_ALWAYS()
#else
    #define ADDR_ASSERT_ALWAYS() ADDR_DBG_BREAK()
#endif

#define ADDR_UNHANDLED_CASE() ADDR_ASSERT(!"Unhandled case")
#define ADDR_NOT_IMPLEMENTED() ADDR_ASSERT(!"Not implemented");

#else //DEBUG
#if defined( _WIN32 )
#define ADDR_ASSERT(__e) { ADDR_ANALYSIS_ASSUME(__e); }
#else
#define ADDR_ASSERT(__e)
#endif
#define ADDR_ASSERT_ALWAYS()
#define ADDR_UNHANDLED_CASE()
#define ADDR_NOT_IMPLEMENTED()
#endif //DEBUG

In gfx10addrlib.cpp, somehow the chipRevision of my GPU is 0. That triggered the assertion.

    case FAMILY_MDN:
        if (ASICREV_IS_MENDOCINO(chipRevision))
        {
            m_settings.supportRbPlus   = 1;
            m_settings.dccUnsup3DSwDis = 0;
        }
        else
        {
            ADDR_ASSERT(!"Unknown chip revision");
        }
        break;

[robert-g-liu]

With some more research, it's literally caused by chipRevision = 0 assignment in below method:

hsa_status_t ImageManagerKv::Initialize(hsa_agent_t agent_handle) {
...
  addr_create_input.chipFamily = family_type_;
  addr_create_input.chipRevision = 0;  // TODO(bwicakso): find how to get this.

[robert-g-liu]

opened an issue in rocm-systems repo. ROCm/rocm-systems#1155