Commits


Scott McKay authored and GitHub committed 00d48d9c300
Add enhanced partitioning utils for use by compiling EPs (#7991) * Add enhanced partitioning utils and convert internal testing EP to use it. Will convert NNAPI EP once checked in. Background: Currently most EPs do their partitioning by iterating the model in the topologically sorted order. Whilst this works, it doesn’t ensure that all nodes which could possibly be added to the current group are, as the group is closed as when the first unsupported node is seen. Changes: - Ask EP for all nodes it supports first - Do partitioning aware topological sort - Groups nodes and flips between processing supported and unsupported nodes to maximise inputs that will be available for each partition - Create groups of nodes for the partition using the new order of nodes - Create ComputeCapability for each group There’s also an additional ability to specify operators to stop at. The processing will find all downstream nodes from ‘stop at’ operators and exclude them. If NonMaxSuppression is specified we can prevent the post-processing from SSD Mobilenet and MobileDet attempting to use NNAPI (so easy way to have parity with the TF Lite behavior). I don’t think there’s an automated way to determine what if any ‘stop at’ operators are required for a model, so this will need to be a configuration parameter for the EP and we’ll need to document recommended values for popular models.