iree-test-suites

Test suites for IREE and related projects.

Test suites

litert_models/: LiteRT models

• Tests that import, compile, and run LiteRT (formerly TensorFlow Lite) models through IREE then compare the outputs against a reference (ai-edge-litert).
• Runnable via pytest.

onnx_models/ : Open Neural Network Exchange models

• Tests that import, compile, and run ONNX models through IREE then compare the outputs against a reference (ONNX Runtime).
• Runnable via pytest.

onnx_ops/ : Open Neural Network Exchange operations

• 1000+ tests for ONNX framework operators.
• Runnable via pytest using a configurable set of flags to iree-compile and iree-run-module.

sharktank_models/ : Models exported using sharktank

• Tests for small scale versions of Large Language Models (LLMs) and other Generative AI (GenAI) programs exported using the sharktank package built as part of the shark-ai project.
• Runnable via pytest for both CPU and GPU targets.

torch_models/ : PyTorch models as Torch Dialect

• Tests that compile and run PyTorch models, converted using torch-mlir.
• Runnable via pytest for any iree registered target.

Git repository details

This repository uses Git Large File Storage (LFS) to store some model files (.mlir, .mlirbc) and parameters (*.irpa).

• If you would like to access these files, ensure you have git-lfs installed.
• If you have git-lfs installed and do not want to download these files, you can set the GIT_LFS_SKIP_SMUDGE=1 environment variable before cloning.