Skip to content

Move moving weights to memory to the end of Graph::Resolve() #25626

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Aug 2, 2025

Conversation

yuslepukhin
Copy link
Member

Description

Move moving weights to memory to the end of Graph::Resolve().
Modify Inject so it copies data into TensorProto according to the C API docs.

Motivation and Context

TypeAndShape inference runs as a part of Resolve() and it unable to inspect and load the initializers that point to OrtValues at that time. We choose to move TensorProto to OrtValue conversion at the end of Resolve().

References: #25579

@yuslepukhin yuslepukhin requested a review from skottmckay August 1, 2025 03:02
@yuslepukhin yuslepukhin marked this pull request as ready for review August 1, 2025 17:17
@jywu-msft
Copy link
Member

is the test failure real or a flakey test?

[----------] 3 tests from WriteScores
1: [ RUN ] WriteScores.multiple_scores_transform_none
1: unknown file: error: SEH exception with code 0xc0000006 thrown in the test fixture's constructor.
1: Stack trace:
1: 00007FFCEAF3E5CF: _C_specific_handler
1: 00007FFCF9F0623F: _chkstk
1: 00007FFCF9E145D7: RtlWow64GetCurrentCpuArea
1: 00007FFCF9F05B7E: KiUserExceptionDispatcher
1: 00007FFCF9E170C3: RtlVirtualUnwind2
1: 00007FFCF9E14548: RtlWow64GetCurrentCpuArea
1: 00007FFCF9F05B7E: KiUserExceptionDispatcher
1: 00007FFCEAA478B0: std::_Random_device
1: 00007FF777BA3A12: WriteScores::WriteScores
1: 00007FF777BA1F15: testing::internal::TestFactoryImpl<WriteScores_multiple_scores_transform_none_Test>::CreateTest
1: 00007FF778C80192: testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase,testing::Test *>
1: 00007FF778C800A6: testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase,testing::Test *>
1: 00007FF778CA617B: testing::TestInfo::Run
1: 00007FF778CA6341: testing::TestSuite::Run
1: 00007FF778CA6E25: testing::internal::UnitTestImpl::RunAllTests
1: ... Google Test internal frames ...

@yuslepukhin
Copy link
Member Author

is the test failure real or a flakey test?

[----------] 3 tests from WriteScores 1: [ RUN ] WriteScores.multiple_scores_transform_none 1: unknown file: error: SEH exception with code 0xc0000006 thrown in the test fixture's constructor. 1: Stack trace: 1: 00007FFCEAF3E5CF: _C_specific_handler 1: 00007FFCF9F0623F: _chkstk 1: 00007FFCF9E145D7: RtlWow64GetCurrentCpuArea 1: 00007FFCF9F05B7E: KiUserExceptionDispatcher 1: 00007FFCF9E170C3: RtlVirtualUnwind2 1: 00007FFCF9E14548: RtlWow64GetCurrentCpuArea 1: 00007FFCF9F05B7E: KiUserExceptionDispatcher 1: 00007FFCEAA478B0: std::_Random_device 1: 00007FF777BA3A12: WriteScores::WriteScores 1: 00007FF777BA1F15: testing::internal::TestFactoryImpl<WriteScores_multiple_scores_transform_none_Test>::CreateTest 1: 00007FF778C80192: testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase,testing::Test *> 1: 00007FF778C800A6: testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase,testing::Test *> 1: 00007FF778CA617B: testing::TestInfo::Run 1: 00007FF778CA6341: testing::TestSuite::Run 1: 00007FF778CA6E25: testing::internal::UnitTestImpl::RunAllTests 1: ... Google Test internal frames ...

This appears to be real. I am getting help with NNAPI.
I am expecting to get QC machine this afternoon.

NNAPI makes use of Initializer class that handles all kinds of external initializers.
@yuslepukhin yuslepukhin merged commit ca8ee36 into main Aug 2, 2025
92 checks passed
@yuslepukhin yuslepukhin deleted the yuslepukhin/move_ortinitializers_resolve branch August 2, 2025 01:38
snnn pushed a commit that referenced this pull request Aug 3, 2025
### Description
<!-- Describe your changes. -->
Move moving weights to memory to the end of Graph::Resolve().
Modify Inject so it copies data into TensorProto according to the C API
docs.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
TypeAndShape inference runs as a part of `Resolve()` and it unable to
inspect and load the initializers that point to OrtValues at that time.
We choose to move TensorProto to OrtValue conversion at the end of
`Resolve()`.

References: #25579
adrianlizarraga pushed a commit that referenced this pull request Aug 4, 2025
### Description
<!-- Describe your changes. -->
Move moving weights to memory to the end of Graph::Resolve().
Modify Inject so it copies data into TensorProto according to the C API
docs.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->
TypeAndShape inference runs as a part of `Resolve()` and it unable to
inspect and load the initializers that point to OrtValues at that time.
We choose to move TensorProto to OrtValue conversion at the end of
`Resolve()`.

References: #25579

Co-authored-by: Dmitri Smirnov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants