Commits
S. Manohar Karlapalem authored and jywu-msft committed 8d15ffd8f5d
Initial commit for OpenVINO Execution Provider (#935) * Initial commit for OpenVINO Execution Provider OpenVINO Execution Provider provides the interface for ONNX Runtime applications to access Intel's hardware accelerators using Intel's OpenVINO Toolkit. * Fixed bug in GetCapability to disable custom ops Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Added OPENVINO ci pipeline Added new pipeline for openvino provider, made changes to support the docker build and onnxruntime build with openvino. Signed-off-by: Luis Daniel Castellanos <luis.daniel.castellanos@intel.com> * Enabled all unit tests for OpenVINO EP Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Fixed syntax issue in run_docker_build.sh file * Added missing default OPENVINO_VERSION Default value for OPENVINO_VERSION env was missing causing the build to fail * Added install Model Optimizer deps step * Fixed python unit tests and some tests from onnx_backend_test_series Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Fixed indentation bug Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled some of the python backend tests for OpenVINO Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled some model tests Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Remove Duplicate checks for openvino in build.py Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Modified GetCapability for FP16 Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled GPU FP32 tests that are not supported Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Convert modelProto to string and use it in compile Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Pass byte-array input args to MO * Serialized ModelProto passed in-memory to MO ModelOptimizer python module receives the serialized ModelProto in-memory. Uses appropriate ONNX function to load the serialized bytes. * Make Py_Finalize compatible with older python versions Also, remove pFunc unassigned variable possibility. * Fallback if input dims of Matmul is greater than 2 Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * fixup: Device #define syntax * Updated the documentation Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Enable dynamic dim value * removed commented out code * Added Dockerfile for openvino EP Updated instructions on dockerfiles/README.md file Signed-off-by: Luis Daniel Castellanos <luis.daniel.castellanos@intel.com> * Disabled fp16_inception_v1 test Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Code formatting with clang-format Uses style from the .clang-format file in root directory. * fixup: docker tag and build error fixes * Heuristics to automatically detect batching Distributes slices from batch into parallel infer-request objects. * Handle disabled tests in GetCapability Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled average pool and max pool if ceil_mode is 1 Also dilations are not supported if they are greater than 1 Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled Unsqueeze int32 test Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * changes to fix output results bug * Disabled a few C++ unit tests for MYRIAD FP16 Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Manually revert '9fe162bb Enable dynamic dim value' Reverts compile time setting of dynamic shape Reverting manually due to significantly huge auto-revert conflicts. * Fixed unused variable warning Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled Mul test for GPU_FP16 due to accuracy issue Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * VPU documentation update * Disabled inception_v1 for MYRIAD and HDDL *Also disabled few C++ accuracy tests for HDDL Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * updates from upstream * use the new CustomOpApis for I/O interfacing * Pass initializers as subgraph meta-def inputs in GetCapability() Requirement due to API changes introduced with PR# 1019. * Remove obsolete functions * Save indexes of graph inputs from fused_node info Both inputs and initializers are passed as data inputs to the infer function. To identify only inputs among them, save thier index info from fused_node in Compile function. * Documentation changes to enable VPU * Fix VPU related changes in documentation * Fix minor changes in documentation * Fix VPU related changes in documentation * Use Node.In/OutputDefs() to track graph inputs and outputs. Don't use graph_viewer's GetInputs() or GetInputsIncludingInitializers(). * Permit "SAME_UPPER" auto_pad attribute from MaxPool * Disabled fp16_tiny_yolov2 in onnx model tests * Updated documentation to include configuration guides for myriad and hddl Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Use 8 Infer requests only for VAD-R * disable debug prints * Clang-format source files * Updated BUILD.md with OpenVINO R5 links Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled same upper python tests Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Update test exclusion syntax * Change path of install_onnx.sh Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disable tiny_yolov2 in broken tests Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Revert "Change path of install_onnx.sh" This reverts commit ba9db165f3be430f2aff1ef413299ed04637196a. This change is only required for Intel internal CI pipeline until the settings are matched with the upstream's CI pipeline. * Added debug statements for debugging CI error Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Add --build_wheel to linux openvino pipeline Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Added -v option to onnx_test_runner for debugging Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Removed path change patch Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Added -c 1 to onnx_test_runner Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Refactor MO python invocation in separate function Cleans up Model Optimizer python invocation check and conversion logic. Invokes MO only once in GetCapability() and passes the IR strings (xml and bin) to the Compiler as meta-def attributes. * Add comments * code cleanup and comments * Code cleanup for GetCapability Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Removed unnecessary files Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Revert "Added -v option to onnx_test_runner for debugging" This reverts commit d1dd70938a94d648df1a1dbbc2e48d0b97e49ec8. * Revert "Added debug statements for debugging CI error" This reverts commit b86d41afed2aa29c3508155d6f9c8d3a7263cc60. * incorporate Status Code changes * ComputeFunc returns Status::OK() on success * Use test names to disable tests for MYRIAD and VAD-R Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Rename local identifiers from CNNNetwork to OpenVINO network CNNNetwork is an OpenVINO's API class that represents more than just convolutional neural networks (CNNs). Renaming helps to avoid confusion that the API's only support CNN type models. * Added error message if building on windows * Removed duplicate option in Cmake * Removed unnecessary parameters in activation_opt_test Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Refactor Map search and access logic for efficiently and cleanliness. * use C++ style casts * Use os.path.join for python directory path operations * use C++ style casts * EP classes should use onnxruntime namespace * Clean up fixes from PR comments * Don't explicitly shutdown Py interpreter * Remove debug print statements Prints will be re-enabled later with a logging mechanism with debug/verbose printing options. * Decrement ref counts for used pyObjects * Restore build instructions for other compilers Content under the "Using other compilers" section has been accidentally deleted by a previous commit. Restoring back that content from the latest upstream repo. * CMake code cleanup Code clean up, commenting and formatting of CMake code. * Don't pass the unused device_info parameter to OpenVINOGraph ctor. * Add support for multiple I/O data types Adds support for the following tensor data types for graph inputs and outputs: 1) float 2) float16 3) int32 4) int16 5) int8 6) uint16 7) uint8 * cleanup setup.py module list definition * Deduce index of input using tracked input index map Ignores initializers in case they are ordered before inputs. * Removed debug statement in MO code Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * PR feedback * Removed per_sample_tolerance for openvino * Removed unnecessary disabled tests Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Removed debug function Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled tiny_yolo_v2 due to accuracy issues Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Changed the disabled reason for broken tests Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Disabled Reshape with no input Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Python formatting with Autopep8 * Minor fix for MYRIAD devices * Added zero dimension check *Removed setting batch size for the network Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Set the threshold to larger value for MNIST Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Removed setting higher threshold in provider_test_utils Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com> * Check for --use_openvino in python wheel setup.py Add openvino modules to the setup script for building the wheel package only for --use_openvino a build option. * Removed nullptr checks for GetNode() Signed-off-by: suryasidd <surya.siddharth.pemmaraju@intel.com>