fix(profiling): link libatomic statically [backport 3.19] #15850
Conversation
(cherry picked from commit 884c38a)
|
|
taegyunkim
changed the title
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 249 ± 4 ms. The average import time from base is: 251 ± 4 ms. The import time difference between this PR and base is: -1.6 ± 0.2 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate taegyunkim/3.19-libatomic-static (0c15844) with baseline 3.19 (53f54f6) 🟡 Near SLO Breach (1 suite)🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 4.596ms (SLO: <4.750ms -3.2%) vs baseline: ~same Memory: ✅ 64.112MB (SLO: <66.500MB -3.6%) vs baseline: +5.0% ✅ appsec-postTime: ✅ 6.637ms (SLO: <6.750ms 🟡 -1.7%) vs baseline: ~same Memory: ✅ 63.999MB (SLO: <66.500MB -3.8%) vs baseline: +4.9% ✅ appsec-telemetryTime: ✅ 4.605ms (SLO: <4.750ms -3.0%) vs baseline: +0.5% Memory: ✅ 64.036MB (SLO: <66.500MB -3.7%) vs baseline: +4.9% ✅ debuggerTime: ✅ 1.859ms (SLO: <2.000ms -7.0%) vs baseline: +0.2% Memory: ✅ 47.765MB (SLO: <49.500MB -3.5%) vs baseline: +4.6% ✅ iast-getTime: ✅ 1.862ms (SLO: <2.000ms -6.9%) vs baseline: ~same Memory: ✅ 44.589MB (SLO: <49.000MB -9.0%) vs baseline: +5.1% ✅ profilerTime: ✅ 1.917ms (SLO: <2.100ms -8.7%) vs baseline: -0.2% Memory: ✅ 48.492MB (SLO: <50.000MB -3.0%) vs baseline: +4.6% ✅ resource-renamingTime: ✅ 3.364ms (SLO: <3.650ms -7.8%) vs baseline: ~same Memory: ✅ 54.237MB (SLO: <56.000MB -3.1%) vs baseline: +4.8% ✅ tracerTime: ✅ 3.355ms (SLO: <3.650ms -8.1%) vs baseline: -0.3% Memory: ✅ 54.413MB (SLO: <56.500MB -3.7%) vs baseline: +5.1% ✅ tracer-nativeTime: ✅ 3.348ms (SLO: <3.650ms -8.3%) vs baseline: -0.5% Memory: ✅ 54.399MB (SLO: <60.000MB -9.3%) vs baseline: +5.2%
|
## Description No need to depend on libatomic, it probably was needed for linux 32 bit which we no longer build for. https://app.datadoghq.com/notebook/13639339/postmortem-ir-47431-python-memory-profiler-not-loading-due-to-a-missing-dep Similar to 3.19 and 4.0 fix PRs - #15744 - #15850 On `python:3.14-slim` image using `ddtrace==4.1.2`, we get ``` root@8a55e06cad23:~# python -c "try: from ddtrace.profiling.collector import _memalloc except ImportError as e: print(e)" libatomic.so.1: cannot open shared object file: No such file or directory ``` Also on the latest head commit d18e59e, we get the same error. When using pre-built wheel from this branch, https://github.com/DataDog/dd-trace-py/actions/runs/20732195747?pr=15853, we no longer get that error from `pythono:3.14-slim` image ``` root@c34443e03a1b:/# python -c "from ddtrace.profiling.collector import _memalloc; print('import success')" import success ``` Also tested that the wheel successfully generates memory/stack samples using dd-trace-doe, DataDog/dd-trace-doe#227 using `python:3.14-slim-bookworm` image You can find example profiles https://app.datadoghq.com/profiling/explorer?query=service%3Adoe%2Fpython%20doe.hash%3A2ca08969421b8009eaeac31de1b2a699&agg_m=%40prof_python_cpu_cores&agg_m_source=base&agg_t=sum&extra_search_fields=%7B%22filters_query%22%3A%22%22%2C%22sample_type%22%3A%22cpu-time%22%7D&fromUser=true&my_code=disabled&profile_type=alloc-size&refresh_mode=paused&viz=stream&from_ts=1767589200000&to_ts=1767667659396&live=false ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers -->
## Description No need to depend on libatomic, it probably was needed for linux 32 bit which we no longer build for. https://app.datadoghq.com/notebook/13639339/postmortem-ir-47431-python-memory-profiler-not-loading-due-to-a-missing-dep Similar to 3.19 and 4.0 fix PRs - #15744 - #15850 On `python:3.14-slim` image using `ddtrace==4.1.2`, we get ``` root@8a55e06cad23:~# python -c "try: from ddtrace.profiling.collector import _memalloc except ImportError as e: print(e)" libatomic.so.1: cannot open shared object file: No such file or directory ``` Also on the latest head commit d18e59e, we get the same error. When using pre-built wheel from this branch, https://github.com/DataDog/dd-trace-py/actions/runs/20732195747?pr=15853, we no longer get that error from `pythono:3.14-slim` image ``` root@c34443e03a1b:/# python -c "from ddtrace.profiling.collector import _memalloc; print('import success')" import success ``` Also tested that the wheel successfully generates memory/stack samples using dd-trace-doe, DataDog/dd-trace-doe#227 using `python:3.14-slim-bookworm` image You can find example profiles https://app.datadoghq.com/profiling/explorer?query=service%3Adoe%2Fpython%20doe.hash%3A2ca08969421b8009eaeac31de1b2a699&agg_m=%40prof_python_cpu_cores&agg_m_source=base&agg_t=sum&extra_search_fields=%7B%22filters_query%22%3A%22%22%2C%22sample_type%22%3A%22cpu-time%22%7D&fromUser=true&my_code=disabled&profile_type=alloc-size&refresh_mode=paused&viz=stream&from_ts=1767589200000&to_ts=1767667659396&live=false ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers --> (cherry picked from commit 3c05a2e)
No need to depend on libatomic, it probably was needed for linux 32 bit which we no longer build for. https://app.datadoghq.com/notebook/13639339/postmortem-ir-47431-python-memory-profiler-not-loading-due-to-a-missing-dep Similar to 3.19 and 4.0 fix PRs - #15744 - #15850 On `python:3.14-slim` image using `ddtrace==4.1.2`, we get ``` root@8a55e06cad23:~# python -c "try: from ddtrace.profiling.collector import _memalloc except ImportError as e: print(e)" libatomic.so.1: cannot open shared object file: No such file or directory ``` Also on the latest head commit d18e59e, we get the same error. When using pre-built wheel from this branch, https://github.com/DataDog/dd-trace-py/actions/runs/20732195747?pr=15853, we no longer get that error from `pythono:3.14-slim` image ``` root@c34443e03a1b:/# python -c "from ddtrace.profiling.collector import _memalloc; print('import success')" import success ``` Also tested that the wheel successfully generates memory/stack samples using dd-trace-doe, DataDog/dd-trace-doe#227 using `python:3.14-slim-bookworm` image You can find example profiles https://app.datadoghq.com/profiling/explorer?query=service%3Adoe%2Fpython%20doe.hash%3A2ca08969421b8009eaeac31de1b2a699&agg_m=%40prof_python_cpu_cores&agg_m_source=base&agg_t=sum&extra_search_fields=%7B%22filters_query%22%3A%22%22%2C%22sample_type%22%3A%22cpu-time%22%7D&fromUser=true&my_code=disabled&profile_type=alloc-size&refresh_mode=paused&viz=stream&from_ts=1767589200000&to_ts=1767667659396&live=false <!-- Describe your testing strategy or note what tests are included --> <!-- Note any risks associated with this change, or "None" if no risks --> <!-- Any other information that would be helpful for reviewers --> (cherry picked from commit 3c05a2e)
|
Closing in favor of #15872 |
Backport 3c05a2e from #15853 to 4.1. ## Description No need to depend on libatomic, it probably was needed for linux 32 bit which we no longer build for. https://app.datadoghq.com/notebook/13639339/postmortem-ir-47431-python-memory-profiler-not-loading-due-to-a-missing-dep Similar to 3.19 and 4.0 fix PRs - #15744 - #15850 On `python:3.14-slim` image using `ddtrace==4.1.2`, we get ``` root@8a55e06cad23:~# python -c "try: from ddtrace.profiling.collector import _memalloc except ImportError as e: print(e)" libatomic.so.1: cannot open shared object file: No such file or directory ``` Also on the latest head commit d18e59e, we get the same error. When using pre-built wheel from this branch, https://github.com/DataDog/dd-trace-py/actions/runs/20732195747?pr=15853, we no longer get that error from `pythono:3.14-slim` image ``` root@c34443e03a1b:/# python -c "from ddtrace.profiling.collector import _memalloc; print('import success')" import success ``` Also tested that the wheel successfully generates memory/stack samples using dd-trace-doe, DataDog/dd-trace-doe#227 using `python:3.14-slim-bookworm` image You can find example profiles https://app.datadoghq.com/profiling/explorer?query=service%3Adoe%2Fpython%20doe.hash%3A2ca08969421b8009eaeac31de1b2a699&agg_m=%40prof_python_cpu_cores&agg_m_source=base&agg_t=sum&extra_search_fields=%7B%22filters_query%22%3A%22%22%2C%22sample_type%22%3A%22cpu-time%22%7D&fromUser=true&my_code=disabled&profile_type=alloc-size&refresh_mode=paused&viz=stream&from_ts=1767589200000&to_ts=1767667659396&live=false ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers --> Co-authored-by: Taegyun Kim <[email protected]>
## Description No need to depend on libatomic, it probably was needed for linux 32 bit which we no longer build for. https://app.datadoghq.com/notebook/13639339/postmortem-ir-47431-python-memory-profiler-not-loading-due-to-a-missing-dep Similar to 3.19 and 4.0 fix PRs - DataDog#15744 - DataDog#15850 On `python:3.14-slim` image using `ddtrace==4.1.2`, we get ``` root@8a55e06cad23:~# python -c "try: from ddtrace.profiling.collector import _memalloc except ImportError as e: print(e)" libatomic.so.1: cannot open shared object file: No such file or directory ``` Also on the latest head commit d18e59e, we get the same error. When using pre-built wheel from this branch, https://github.com/DataDog/dd-trace-py/actions/runs/20732195747?pr=15853, we no longer get that error from `pythono:3.14-slim` image ``` root@c34443e03a1b:/# python -c "from ddtrace.profiling.collector import _memalloc; print('import success')" import success ``` Also tested that the wheel successfully generates memory/stack samples using dd-trace-doe, https://github.com/DataDog/dd-trace-doe/pull/227 using `python:3.14-slim-bookworm` image You can find example profiles https://app.datadoghq.com/profiling/explorer?query=service%3Adoe%2Fpython%20doe.hash%3A2ca08969421b8009eaeac31de1b2a699&agg_m=%40prof_python_cpu_cores&agg_m_source=base&agg_t=sum&extra_search_fields=%7B%22filters_query%22%3A%22%22%2C%22sample_type%22%3A%22cpu-time%22%7D&fromUser=true&my_code=disabled&profile_type=alloc-size&refresh_mode=paused&viz=stream&from_ts=1767589200000&to_ts=1767667659396&live=false ## Testing <!-- Describe your testing strategy or note what tests are included --> ## Risks <!-- Note any risks associated with this change, or "None" if no risks --> ## Additional Notes <!-- Any other information that would be helpful for reviewers -->
Description
https://app.datadoghq.com/notebook/13639339/postmortem-ir-47431-python-memory-profiler-not-loading-due-to-a-missing-dep
Cherry-picked commit 884c38a from #15744
Some of the images don't come with
libatomicinstalled, one can check that by runningdpkg -l | grep atomic.On
python:3.13, we getOn
python:3.13-slimwe get empty outputOn
python:3.13-slimimage usingddtrace==3.19.3,On same image using prebuilt wheel from this branch, wheels-cp313-manylinux_x86_64
Testing
Risks
Additional Notes