Commits


cao lei authored and GitHub committed d69823f7644
Do not create Barrier and triggerDownstream steps if the corresponding nodes are split by yield Op in training scenario (#14570) ### Description Do not create Barrier and triggerDownstream steps during execution plan creation if the corresponding nodes are split by yield Op in training scenario. ### Motivation and Context In training scenario, forward and backward processes are running two different partial nodes of a graph. If there are two nodes each in one of the partial graph and separate in two streams, there are still triggerDownstream/barrier steps between them which work quite different from inference process as one of the steps will not be executed due to it is not in the correct range. To make it work, there is a hacky way to trigger the barrier step explicitly for training. This PR is to do some check, and do not create Barrier and triggerDownstream steps if the corresponding nodes are split by yield Op in training scenario. So the hacky way is not needed.