OpTest enhancement: unify forward backward logics; bug fix; support in-place operator test #6064
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolve #5842
As described in issue Support "In place" computation in operator unit test #5842, I was trying to add a small feature to OpTest class to support convenient unit test for in-place computation operators, which is supported by the
in_place_map
attribute added in base classOpTest
.After reading the original implementation, I find several places that can be improved and start to rewrite some of the methods, which mainly to improve the following aspects:
Unify forward/backward code
In the old implementation, forward test and backward test are implemented in completely different logics, probably due to historical reasons. They use different ways to create operators, to create variable descriptions, feed variables, and execute the program. Check the "#FIXME" comment in the old code.
In this PR, both forward and backward use similar process to do computation:
Scope and Block
The old code uses both
block
andscope
to manage variables. I find it quite confusing, so I completely removescope
related code and replace it withblock
methods.I'm actually a little fuzzy about the
scope
andblock
concept in PaddlePaddle, even after reading all the design docs in current repo, so Please help to make sure this is ok. I ran all the unit tests, it seems alright.New batch_norm_op unit test
Due to lack of in-place computation support, the original batch_norm_op unit test rewrites most of the logic in the
OpTest
base class. Now a simpler version is possible after the enhancement.