Add OpenVINO backend to pip wheel for Linux x86_64 (#18309)#18309
Open
shoumikhin wants to merge 1 commit intopytorch:mainfrom
Open
Add OpenVINO backend to pip wheel for Linux x86_64 (#18309)#18309shoumikhin wants to merge 1 commit intopytorch:mainfrom
shoumikhin wants to merge 1 commit intopytorch:mainfrom
Conversation
shoumikhin
requested review from
kirklandsign,
larryliu0820 and
mergennachin
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18309
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
Contributor
|
@shoumikhin has exported this pull request. If you are a Meta employee, you can view the originating Diff in D97202714. |
This PR needs a
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the OpenVINO backend to be included in the Linux x86_64 pip wheel by switching the runtime to dlopen/dlsym against the OpenVINO C API, removing any build-time dependency on the OpenVINO SDK.
Changes:
- Reworked the OpenVINO runtime backend to dynamically load
libopenvino_c.soand call into the C API via function pointers. - Enabled building and wheel packaging for the OpenVINO backend on Linux x86_64, plus CI wheel checks and an
executorch[openvino]optional dependency. - Added/updated documentation describing pip-wheel usage and runtime installation.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/cmake/preset/pybind.cmake | Adds an overridable preset option for EXECUTORCH_BUILD_OPENVINO. |
| setup.py | Turns on OpenVINO backend build for Linux x86_64 and builds the openvino_backend target when enabled. |
| pyproject.toml | Adds an openvino optional dependency extra with Linux x86_64 marker. |
| docs/source/build-run-openvino.md | Adds pip-wheel quick start instructions for Linux x86_64. |
| backends/openvino/runtime/OpenvinoBackend.h | Switches backend handle/types to C API and adds lazy-loader state. |
| backends/openvino/runtime/OpenvinoBackend.cpp | Implements runtime dlopen/dlsym loading and C API execution path. |
| backends/openvino/runtime/OpenvinoApi.h | Introduces OpenVINO C ABI forward decls + function pointer table. |
| backends/openvino/README.md | Adds pip-wheel quick start + env var guidance. |
| backends/openvino/CMakeLists.txt | Removes find_package(OpenVINO) and links only executorch_core + dl libs. |
| README-wheel.md | Documents that Linux x86_64 wheels include QNN + OpenVINO (runtime required). |
| .ci/scripts/wheel/test_linux.py | Asserts OpenvinoBackend is registered on Linux x86_64 wheels. |
| .ci/scripts/wheel/pre_build_script.sh | Installs OpenVINO runtime in Linux x86_64 wheel test environment. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
meta-codesync
bot
changed the title
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enable build on Linux x86_64 unconditionally in setup.py since there is no build-time dependency. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
d6de3b1 to
e6675bc
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enable build on Linux x86_64 unconditionally in setup.py since there is no build-time dependency. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
e6675bc to
48541fc
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enable build on Linux x86_64 unconditionally in setup.py since there is no build-time dependency. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
48541fc to
4c53cb6
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Refactors the OpenVINO runtime backend to load the OpenVINO C API dynamically at runtime (dlopen/dlsym), enabling inclusion of the OpenVINO backend in the Linux x86_64 pip wheel without a build-time OpenVINO SDK dependency.
Changes:
- Reworked OpenVINO runtime to use OpenVINO C API function pointers with thread-safe lazy loading.
- Updated build/packaging to enable building the OpenVINO backend into the Linux x86_64 wheel and added an
executorch[openvino]optional dependency. - Updated docs and CI wheel checks to reflect/verify OpenVINO backend presence and runtime installation.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/cmake/preset/pybind.cmake | Adds an overridable preset option for building OpenVINO in pybind builds. |
| setup.py | Enables OpenVINO build for Linux x86_64 wheels and adds the build target when enabled. |
| pyproject.toml | Adds executorch[openvino] optional dependency with Linux x86_64 marker. |
| docs/source/build-run-openvino.md | Adds pip-wheel Quick Start instructions for Linux x86_64. |
| backends/openvino/runtime/OpenvinoBackend.h | Updates backend interface to use C API types and lazy-loading state. |
| backends/openvino/runtime/OpenvinoBackend.cpp | Implements dlopen/dlsym loading and rewrites init/execute to use the C API. |
| backends/openvino/runtime/OpenvinoApi.h | Introduces forward declarations + function pointer table for the OpenVINO C API. |
| backends/openvino/README.md | Documents pip-wheel usage and runtime library path configuration. |
| backends/openvino/CMakeLists.txt | Removes build-time OpenVINO SDK dependency; links against libdl instead. |
| README-wheel.md | Documents inclusion of QNN/OpenVINO backends in Linux x86_64 wheels. |
| .ci/scripts/wheel/test_linux.py | Verifies OpenvinoBackend is registered in Linux x86_64 wheel tests. |
| .ci/scripts/wheel/pre_build_script.sh | Installs OpenVINO runtime package on Linux x86_64 for wheel testing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
pyproject.toml
Outdated
|
|
||
| [project.optional-dependencies] | ||
| openvino = [ | ||
| "openvino>=2025.1.0,<2026.0.0; platform_system == 'Linux' and platform_machine == 'x86_64'", |
Comment on lines
+54
to
+57
| ### Building from Source | ||
|
|
||
| If you need a custom build (different platform, custom configuration, etc.), follow the instructions below. | ||
|
|
shoumikhin
force-pushed
the
export-D97202714
branch
from
4c53cb6 to
727ca1d
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enabled on all Linux architectures (x86_64 and aarch64) since the backend has no build-time dependency and the openvino pip package supports both. ExecuTorch builds wheels for both platforms. Robustness: - LOAD_SYM macro zeros ov_ struct on partial failure to prevent dangling function pointers. - Bounds check on execute() args (ET_CHECK_OR_RETURN_ERROR). - Allocator null check with proper cleanup on failure. - Device string safely constructed from CompileSpec buffer with trailing null byte stripping. - Tensor rank validated against 1024 upper bound. - create_ov_tensor uses RAII (unique_ptr) for heap dims allocation. - Detailed comments on OpenVINO shared_ptr tensor ownership and destroy() lifecycle constraints. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
727ca1d to
33555f8
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Mar 19, 2026
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enabled on all Linux architectures (x86_64 and aarch64) since the backend has no build-time dependency and the openvino pip package supports both. ExecuTorch builds wheels for both platforms. Robustness: - LOAD_SYM macro zeros ov_ struct on partial failure to prevent dangling function pointers. - Bounds check on execute() args (ET_CHECK_OR_RETURN_ERROR). - Allocator null check with proper cleanup on failure. - Device string safely constructed from CompileSpec buffer with trailing null byte stripping. - Tensor rank validated against 1024 upper bound. - create_ov_tensor uses RAII (unique_ptr) for heap dims allocation. - Detailed comments on OpenVINO shared_ptr tensor ownership and destroy() lifecycle constraints. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
33555f8 to
c1be61d
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reworks the OpenVINO backend runtime to remove any build-time dependency on the OpenVINO SDK by dynamically loading the OpenVINO C API (libopenvino_c.so) via dlopen/dlsym, and wires the backend into Linux wheel builds plus user-facing docs and CI smoke tests.
Changes:
- Replaced the OpenVINO runtime implementation to use the OpenVINO C API via runtime symbol loading (no OpenVINO headers/libs needed at build time).
- Enabled building and packaging the
openvino_backendtarget in the pip wheel build flow and added anexecutorch[openvino]extra. - Updated docs/README and wheel smoke tests to reflect backend availability/registration.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/cmake/preset/pybind.cmake | Adds an overridable preset option for EXECUTORCH_BUILD_OPENVINO. |
| setup.py | Auto-enables OpenVINO backend on Linux builds and adds openvino_backend build target when enabled. |
| pyproject.toml | Adds executorch[openvino] optional dependency with a pinned OpenVINO version range on Linux. |
| docs/source/build-run-openvino.md | Adds quick-start instructions for using the pip wheel with OpenVINO runtime. |
| backends/openvino/runtime/OpenvinoBackend.h | Switches handle/types to OpenVINO C API pointers and adds lazy-loader state. |
| backends/openvino/runtime/OpenvinoBackend.cpp | Implements dlopen/dlsym loading, C API execution path, and updated type conversions/tensor wrapping. |
| backends/openvino/runtime/OpenvinoApi.h | Introduces forward declarations and function-pointer table for the OpenVINO C API. |
| backends/openvino/README.md | Documents pip-wheel quick start, env vars, and updated runtime file layout. |
| backends/openvino/CMakeLists.txt | Removes find_package(OpenVINO) and links only executorch_core + dl for runtime loading. |
| README-wheel.md | Updates wheel documentation to mention OpenVINO backend presence and extra install. |
| .ci/scripts/wheel/test_linux_aarch64.py | Adds smoke assertion that OpenvinoBackend is registered. |
| .ci/scripts/wheel/test_linux.py | Adds smoke assertion that OpenvinoBackend is registered (alongside QNN check). |
| .ci/scripts/wheel/pre_build_script.sh | Installs OpenVINO Python package on Linux in wheel CI pre-step. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
setup.py
Outdated
| import importlib.util | ||
| import logging | ||
| import os | ||
| import platform |
There was a problem hiding this comment.
platform is imported but never used, which will trip linters and suggests an incomplete platform/arch gating. Either remove the import or use it (e.g., to gate OpenVINO enablement to specific Linux architectures if intended).
Suggested change
| import platform |
Comment on lines
+721
to
+731
| # Enable OpenVINO backend on Linux. The backend uses dlopen at | ||
| # runtime so it has no build-time SDK dependency. | ||
| if ( | ||
| sys.platform == "linux" | ||
| and install_utils.is_cmake_option_on( | ||
| cmake_configuration_args, | ||
| "EXECUTORCH_BUILD_OPENVINO", | ||
| default=True, | ||
| ) | ||
| ): | ||
| cmake_configuration_args += ["-DEXECUTORCH_BUILD_OPENVINO=ON"] |
There was a problem hiding this comment.
This enables EXECUTORCH_BUILD_OPENVINO for all Linux platforms, but the PR title/docs suggest the wheel support is Linux x86_64-specific. If OpenVINO should only be built into x86_64 wheels, add an architecture check (e.g., platform.machine()), otherwise update the user-facing docs/title to reflect broader Linux arch support.
Comment on lines
+66
to
+68
| # Install OpenVINO Python package on Linux for wheel testing. | ||
| # The backend itself has no build-time dependency (uses dlopen at runtime). | ||
| if [[ "$(uname -s)" == "Linux" ]]; then |
There was a problem hiding this comment.
This installs openvino on all Linux wheel builds, including the linux-aarch64 workflow that also uses this pre-script. Since the aarch64 smoke test only checks backend registration (no runtime load), consider gating this install to x86_64 (or making it best-effort/optional) to avoid unnecessary CI time and potential platform-specific install failures.
Suggested change
| # Install OpenVINO Python package on Linux for wheel testing. | |
| # The backend itself has no build-time dependency (uses dlopen at runtime). | |
| if [[ "$(uname -s)" == "Linux" ]]; then | |
| # Install OpenVINO Python package on Linux x86_64 for wheel testing. | |
| # The backend itself has no build-time dependency (uses dlopen at runtime). | |
| if [[ "$(uname -s)" == "Linux" && "$(uname -m)" == "x86_64" ]]; then |
Comment on lines
+49
to
+50
| "OpenVINO runtime not found (%s). " | ||
| "Install with: pip install \"openvino>=2025.1.0,<2026.0.0\"", |
There was a problem hiding this comment.
The load failure message only suggests installing the Python package, but the default dlopen("libopenvino_c.so") also depends on the library being in the dynamic loader search path. Consider mentioning OPENVINO_LIB_PATH/LD_LIBRARY_PATH in the error to help users who already installed openvino but still can't be found at runtime.
Suggested change
| "OpenVINO runtime not found (%s). " | |
| "Install with: pip install \"openvino>=2025.1.0,<2026.0.0\"", | |
| "OpenVINO runtime not found (dlopen failed: %s). " | |
| "Ensure that 'libopenvino_c.so' is on your dynamic loader search path " | |
| "(e.g., set OPENVINO_LIB_PATH or LD_LIBRARY_PATH), or install with: " | |
| "pip install \"openvino>=2025.1.0,<2026.0.0\"", |
Comment on lines
+40
to
+97
| bool OpenvinoBackend::ensure_loaded() const { | ||
| std::call_once(load_flag_, [this]() { | ||
| const char* lib_path = std::getenv("OPENVINO_LIB_PATH"); | ||
| const char* effective_path = lib_path ? lib_path : kDefaultLibName; | ||
|
|
||
| void* handle = dlopen(effective_path, RTLD_NOW | RTLD_LOCAL); | ||
| if (!handle) { | ||
| ET_LOG( | ||
| Error, | ||
| "OpenVINO runtime not found (%s). " | ||
| "Install with: pip install \"openvino>=2025.1.0,<2026.0.0\"", | ||
| dlerror()); | ||
| return; | ||
| } | ||
| } catch (const std::exception& e) { | ||
| // Log the exception if OpenVINO runtime is not available | ||
| ET_LOG(Error, "OpenVINO is not available: %s", e.what()); | ||
| } catch (...) { | ||
| // Handle any unexpected errors | ||
| lib_handle_.reset(handle); | ||
|
|
||
| #define LOAD_SYM(field, symbol_name) \ | ||
| ov_.field = load_symbol<decltype(ov_.field)>(handle, #symbol_name); \ | ||
| if (!ov_.field) { \ | ||
| ov_ = OpenvinoFunctions{}; \ | ||
| lib_handle_.reset(); \ | ||
| return; \ | ||
| } | ||
|
|
||
| LOAD_SYM(core_create, ov_core_create) | ||
| LOAD_SYM(core_free, ov_core_free) | ||
| LOAD_SYM(core_get_available_devices, ov_core_get_available_devices) | ||
| LOAD_SYM(available_devices_free, ov_available_devices_free) | ||
| LOAD_SYM(core_import_model, ov_core_import_model) | ||
| LOAD_SYM( | ||
| compiled_model_create_infer_request, | ||
| ov_compiled_model_create_infer_request) | ||
| LOAD_SYM(compiled_model_inputs_size, ov_compiled_model_inputs_size) | ||
| LOAD_SYM(compiled_model_outputs_size, ov_compiled_model_outputs_size) | ||
| LOAD_SYM(compiled_model_free, ov_compiled_model_free) | ||
| LOAD_SYM( | ||
| infer_request_set_input_tensor_by_index, | ||
| ov_infer_request_set_input_tensor_by_index) | ||
| LOAD_SYM( | ||
| infer_request_set_output_tensor_by_index, | ||
| ov_infer_request_set_output_tensor_by_index) | ||
| LOAD_SYM(infer_request_infer, ov_infer_request_infer) | ||
| LOAD_SYM(infer_request_free, ov_infer_request_free) | ||
| LOAD_SYM(tensor_create_from_host_ptr, ov_tensor_create_from_host_ptr) | ||
| LOAD_SYM(tensor_free, ov_tensor_free) | ||
| LOAD_SYM(shape_create, ov_shape_create) | ||
| LOAD_SYM(shape_free, ov_shape_free) | ||
|
|
||
| #undef LOAD_SYM | ||
|
|
||
| loaded_ = true; | ||
| ET_LOG( | ||
| Error, "OpenVINO availability check failed due to an unknown error."); | ||
| Info, | ||
| "OpenVINO: runtime loaded successfully from '%s'", | ||
| effective_path); | ||
| }); | ||
| return loaded_; | ||
| } |
There was a problem hiding this comment.
std::call_once makes the first dlopen attempt final: if loading fails due to a transient misconfiguration (e.g., user sets OPENVINO_LIB_PATH/LD_LIBRARY_PATH after importing), subsequent is_available()/init() calls will never retry in the same process. Consider allowing retries on failure (e.g., guard with a mutex and only keep the once_flag for the success path) or documenting this behavior clearly.
Comment on lines
+17
to
+20
| * \[Linux x86_64] [QNN](docs/source/backends-qualcomm.md) backend is linked into the prebuilt module. | ||
| * \[Linux] [OpenVINO](docs/source/build-run-openvino.md) backend is also linked into the | ||
| prebuilt module. OpenVINO requires the runtime to be installed separately: | ||
| `pip install executorch[openvino]` |
There was a problem hiding this comment.
This section says OpenVINO is linked into the prebuilt module for "[Linux]", but earlier in the same doc the package compatibility is stated as "Linux x86_64". Please clarify the intended architecture support here (e.g., change to "[Linux x86_64]" or update the compatibility statement) to avoid confusing wheel users on other Linux architectures.
Summary: Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the OpenVINO C API, eliminating the build-time dependency on the OpenVINO SDK. This follows the QNN backend's proven pattern: the backend links statically into _portable_lib.so with zero external library dependencies and loads libopenvino_c.so at runtime via dlopen when the user has the openvino pip package installed. Key design decisions: - Use OpenVINO C API (openvino/c/openvino.h) instead of C++ API to enable dlopen with dlsym function pointers. The C++ class-based API (ov::Core, ov::InferRequest) cannot be resolved via dlsym. - Forward-declare OpenVINO C types in OpenvinoApi.h instead of including headers, so there is no build-time SDK dependency at all. - Thread-safe lazy loading via std::call_once in ensure_loaded(). - OPENVINO_LIB_PATH env var for explicit library path override. - Backend always registers via static initializer (zero cost); actual OpenVINO loading deferred to first is_available()/init() call. - No changes to portable_lib.py needed — no RTLD_GLOBAL, no separate .so, no auditwheel workarounds. The backend is statically linked into _portable_lib.so like XNNPACK and has no NEEDED entries for any OpenVINO library. - Enabled on all Linux architectures (x86_64 and aarch64) since the backend has no build-time dependency and the openvino pip package supports both. ExecuTorch builds wheels for both platforms. Robustness: - LOAD_SYM macro zeros ov_ struct on partial failure to prevent dangling function pointers. - Bounds check on execute() args (ET_CHECK_OR_RETURN_ERROR). - Allocator null check with proper cleanup on failure. - Device string safely constructed from CompileSpec buffer with trailing null byte stripping. - Tensor rank validated against 1024 upper bound. - create_ov_tensor uses RAII (unique_ptr) for heap dims allocation. - Detailed comments on OpenVINO shared_ptr tensor ownership and destroy() lifecycle constraints. User experience: pip install executorch # XNNPACK+QNN+OpenVINO registered pip install executorch[openvino] # + openvino runtime, backend activates Differential Revision: D97202714
shoumikhin
force-pushed
the
export-D97202714
branch
from
c1be61d to
1eee10b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Rewrite the OpenVINO backend runtime to use dlopen/dlsym with the
OpenVINO C API, eliminating the build-time dependency on the OpenVINO
SDK. This follows the QNN backend's proven pattern: the backend links
statically into _portable_lib.so with zero external library dependencies
and loads libopenvino_c.so at runtime via dlopen when the user has the
openvino pip package installed.
Key design decisions:
enable dlopen with dlsym function pointers. The C++ class-based API
(ov::Core, ov::InferRequest) cannot be resolved via dlsym.
including headers, so there is no build-time SDK dependency at all.
OpenVINO loading deferred to first is_available()/init() call.
.so, no auditwheel workarounds. The backend is statically linked
into _portable_lib.so like XNNPACK and has no NEEDED entries for
any OpenVINO library.
there is no build-time dependency.
User experience:
pip install executorch # XNNPACK+QNN+OpenVINO registered
pip install executorch[openvino] # + openvino runtime, backend activates
Differential Revision: D97202714