Commits


kunal-vaishnavi authored and GitHub committed ca22a5a9d0f
Add fusions for OpenAI CLIP (#20721) ### Description This PR adds fusions for [OpenAI's CLIP model](https://huggingface.co/openai/clip-vit-large-patch14-336). Here is an example of how to run the ORT transformer optimizer for the linked CLIP model. ``` $ git clone https://github.com/microsoft/onnxruntime $ cd onnxruntime/onnxruntime/python/tools/transformers $ python3 optimizer.py --input /path/to/model.onnx --output /path/to/model_opt.onnx --model_type clip --num_heads 16 --hidden_size 1024 --use_external_data_format --opt_level 0 ``` ### Motivation and Context This PR helps optimize multi-modal models that use CLIP for the vision encoder.