Commits


pengwa authored and GitHub committed 37f743680a0
Fix build when flash attention and memory efficient attention are disabled (#18761) ### Fix build when flash attention and memory efficient attention are disabled On a customer env with lower version of CUDA < 11.6. Both flash attention and memory efficient attention is turned OFF according to https://github.com/microsoft/onnxruntime/blob/e8f33b54bab5129b0dea177669bbd1c1d0894dd8/cmake/CMakeLists.txt#L701. So https://github.com/microsoft/onnxruntime/blob/e8f33b54bab5129b0dea177669bbd1c1d0894dd8/cmake/external/cutlass.cmake#L1 condition check return false. No cutlass lib is built. ``` Turn off flash attention since CUDA compiler version < 11.6 ``` While, the kernels in https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/contrib_ops/cuda/moe/ft_moe are depending on cutass for its build, so we get error like this: ``` [ 77%] Building CUDA object CMakeFiles/onnxruntime_providers_cuda.dir/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu.o In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17: /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory 23 | #include "cutlass/array.h" | ^~~~~~~~~~~~~~~~~ compilation terminated. In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17: /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory 23 | #include "cutlass/array.h" | ^~~~~~~~~~~~~~~~~ compilation terminated. In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17: /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory 23 | #include "cutlass/array.h" | ^~~~~~~~~~~~~~~~~ compilation terminated. In file included from /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu:17: /tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_template.h:23:10: fatal error: cutlass/array.h: No such file or directory 23 | #include "cutlass/array.h" | ^~~~~~~~~~~~~~~~~ compilation terminated. fatal : Could not open input file /tmp/tmpxft_00044da3_00000000-11_moe_gemm_kernels_fp16_fp16.compute_60.cpp1.ii make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/build.make:6290: CMakeFiles/onnxruntime_providers_cuda.dir/tmp/onnxruntime/onnxruntime/contrib_ops/cuda/moe/ft_moe/moe_gemm_kernels_fp16_fp16.cu.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [CMakeFiles/Makefile2:2210: CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2 make: *** [Makefile:166: all] Error 2 Traceback (most recent call last): File "/tmp/onnxruntime/tools/ci_build/build.py", line 2746, in <module> sys.exit(main()) File "/tmp/onnxruntime/tools/ci_build/build.py", line 2639, in main build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target) File "/tmp/onnxruntime/tools/ci_build/build.py", line 1527, in build_targets run_subprocess(cmd_args, env=env) File "/tmp/onnxruntime/tools/ci_build/build.py", line 824, in run_subprocess return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env) File "/tmp/onnxruntime/tools/python/util/run.py", line 49, in run completed_process = subprocess.run( File "/opt/conda/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, ``` ### Motivation and Context To summarize, there are two cases we will have build failure for Linux CUDA build: 1. User use cuda version < 11.6 2. User disabled Flash attention and memory efficient attention explictly with onnxruntime_USE_FLASH_ATTENTION and onnxruntime_USE_MEMORY_EFFICIENT_ATTENTION