Cythonize _program.py by Andy-Jost · Pull Request #1565 · NVIDIA/cuda-python

Andy-Jost · 2026-02-03T00:54:56Z

Summary

Converts _program.py to Cython (_program.pyx) with full cythonization of NVRTC and NVVM paths, RAII-based resource management, and nogil error handling.

Changes

Cythonization

_program.py → _program.pyx with cdef class Program
New _program.pxd with typed attribute declarations
Program_init cythonized with direct cynvrtc/cynvvm calls in nogil blocks
Program_compile_nvrtc and Program_compile_nvvm use nogil for C API calls
Long methods factored into cdef inline helpers

Resource Handles

Add NvrtcProgramHandle and NvvmProgramHandle (std::shared_ptr-based RAII)
Add create_nvrtc_program_handle() and create_nvvm_program_handle()
Remove _MembersNeededForFinalize / weakref.finalize pattern
as_py(NvvmProgramHandle) returns Python int (matches cuda_bindings design)

Error Handling

Add HANDLE_RETURN_NVRTC for nogil NVRTC error handling with program log
Add HANDLE_RETURN_NVVM for nogil NVVM error handling with program log
Add NVVMError exception class
Simplify HANDLE_RETURN to directly take cydriver.CUresult

Validation & API

Add SUPPORTED_TARGETS map for unified backend/target validation
Add public Program.driver_can_load_nvrtc_ptx_output() API
Consistent error message format for unsupported targets

Code Quality

Module docstring, __all__, type alias section added
Docstrings updated to use :class: refs and public paths
Type annotations added to public methods
Union import removed in favor of | syntax
Compiler options use std::vector<const char*> instead of malloc/free

Tests

Extend test_object_protocols.py with Program and ObjectCode variations
Update test_program.py: test close() idempotency instead of handle state
Update error message regex for unified format

Test plan

All existing test_program.py tests pass
All existing test_module.py tests pass
test_object_protocols.py passes with new variations
pre-commit passes

Closes #1082

copy-pr-bot · 2026-02-03T00:54:59Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Andy-Jost · 2026-02-03T00:55:09Z

/ok to test 85dbbb5

github-actions · 2026-02-03T01:06:01Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-1565/
https://nvidia.github.io/cuda-python/pr-preview/pr-1565/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-1565/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-1565/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

Andy-Jost · 2026-02-03T15:20:04Z

cuda_core/tests/test_module.py

    mod_obj = ObjectCode.from_ptx(ptx, symbol_mapping=sym_map)
    assert mod.code == ptx
-    if not Program._can_load_generated_ptx():
+    if not Program.driver_can_load_nvrtc_ptx_output():


This is worthy of discussion. I changed this private API into a public one after refactoring because it seems somewhat useful to an end user. An alternative would be to continue using the private API if the value here is too low.

I'd definitely not make the API change in this PR, in part so that such a subtle change doesn't get drowned out by the many other changes here, but also to avoid entangled unintended side-effects from the disjoint changes (those tend to be hard to sort out, especially later, after this PR is merged already).

But this simply renames a private function, making it public. The implementation remains the same. Do you think there could be unintended side-effects from that?

The questions I'm interested in are whether we expect users to end up with systems that can generate unloadable PTX, and if so, can they benefit from a function to check it?

That's a good question, but in my experience it's usually far better to discuss separately. E.g. yesterday I already got derailed here looking through the very large diffs. I saw there were a bunch of test failures, then I searched the code and saw new code still referencing _can_load_generated_ptx, then decided to suggest it's best to not convolute things.

What makes it beneficial to tag on the change & discussion here?

What makes it beneficial to tag on the change & discussion here?

CI and review latency, TBH. If changes were much cheaper I would break things down much more.

I'm not the best to judge if this should stay private or become public. If public, I'd consider Program.nvrtc_generated_ptx_is_loadable() as the API.

- Rename _program.py to _program.pyx - Convert Program to cdef class with _program.pxd declarations - Extract _MembersNeededForFinalize to module-level _ProgramMNFF (nested classes not allowed in cdef class) - Add __repr__ method to Program - Keep ProgramOptions as @DataClass (unchanged) - Keep weakref.finalize pattern for handle cleanup

- Move _translate_program_options to Program_translate_options (cdef) - Move _can_load_generated_ptx to Program_can_load_generated_ptx (cdef) - Remove unused TYPE_CHECKING import block - Follow _memory/_buffer.pyx helper function patterns

- Reorganize file structure per developer guide (principal class first) - Add module docstring, __all__, type alias section - Factor long methods into cdef inline helpers - Add proper exception specs to cdef functions - Fix docstrings (use :class: refs, public paths) - Add type annotations to public methods - Inline _nvvm_exception_manager (single use) - Remove Union import, use | syntax - Add public Program.driver_can_load_nvrtc_ptx_output() API - Update tests to use new public API Closes NVIDIA#1082

Add fixtures for different Program backends (NVRTC, PTX, NVVM) and ObjectCode code types (cubin, PTX, LTOIR). Split API_TYPES into more precise HASH_TYPES, EQ_TYPES, and WEAKREF_TYPES lists. Derive DICT_KEY_TYPES and WEAK_KEY_TYPES for collection tests.

- Add NvrtcProgramHandle and NvvmProgramHandle to resource handles module - Add function pointer initialization for nvrtcDestroyProgram and nvvmDestroyProgram - Forward-declare nvvmProgram to avoid nvvm.h dependency - Refactor detail::make_py to accept module name parameter - Remove _ProgramMNFF class from _program.pyx - Program now uses typed handles directly with RAII cleanup - Update handle property to return None when handle is null

- Add NVVMError exception class - Add HANDLE_RETURN_NVRTC for nogil NVRTC error handling with program log - Add HANDLE_RETURN_NVVM for nogil NVVM error handling with program log - Remove vestigial supported_error_type fused type - Simplify HANDLE_RETURN to directly take cydriver.CUresult

Andy-Jost · 2026-02-04T00:25:43Z

/ok to test 44fd98e

Andy-Jost · 2026-02-04T00:32:38Z

cuda_core/cuda/core/_cpp/resource_handles.hpp

+// Forward declaration for NVVM - avoids nvvm.h dependency
+// Use void* to match cuda.bindings.cynvvm's typedef for compatibility
+#ifndef CYTHON_EXTERN_C
+typedef void *nvvmProgram;
+#endif


Since nvvm is an optional dependency, I opted to define nvvmProgram this way rather than include the header.

Andy-Jost · 2026-02-04T00:36:10Z

/ok to test 0f06313

Andy-Jost · 2026-02-05T00:48:58Z

/ok to test 5220d47

Andy-Jost · 2026-02-05T01:06:01Z

/ok to test dbf2602

- Change cdef function return types from ObjectCode to object (Cython limitation) - Remove unused imports: intptr_t, NvrtcProgramHandle, NvvmProgramHandle, as_intptr - Update as_py(NvvmProgramHandle) to return Python int via PyLong_FromSsize_t - Update test assertions: remove handle checks after close(), test idempotency instead - Update NVVM error message regex to match new unified format

Andy-Jost · 2026-02-05T01:09:55Z

/ok to test 754e295

Andy-Jost · 2026-02-05T01:18:07Z

cuda_core/cuda/core/_module.pyx

        except KeyError:
-            name = name.encode()
+            if isinstance(name, str):
+                name = name.encode()


Slipped in this small fix. ObjectCode.get_kernel failed when given bytes. New test in test_module.py 107-8

Andy-Jost added this to the cuda.core beta 12 milestone Feb 3, 2026

Andy-Jost added P0 High priority - Must do! cuda.core Everything related to the cuda.core module labels Feb 3, 2026

Andy-Jost self-assigned this Feb 3, 2026

leofang self-requested a review February 3, 2026 06:02

Andy-Jost commented Feb 3, 2026

View reviewed changes

Andy-Jost requested review from cpcloud and rwgk February 3, 2026 15:27

Andy-Jost added 6 commits February 3, 2026 16:24

Andy-Jost force-pushed the cythonize-program branch from 85dbbb5 to 44fd98e Compare February 4, 2026 00:25

Andy-Jost commented Feb 4, 2026

View reviewed changes

Andy-Jost force-pushed the cythonize-program branch from 44fd98e to 0f06313 Compare February 4, 2026 00:36

Andy-Jost force-pushed the cythonize-program branch from 0f06313 to 5220d47 Compare February 5, 2026 00:48

Andy-Jost force-pushed the cythonize-program branch from 5220d47 to dbf2602 Compare February 5, 2026 01:05

rwgk mentioned this pull request Feb 5, 2026

Fix to use resident device stream in IPC peer access tests #1568

Closed

Andy-Jost added 2 commits February 4, 2026 17:09

Merge remote-tracking branch 'origin/main' into cythonize-program

754e295

Andy-Jost force-pushed the cythonize-program branch from dbf2602 to 754e295 Compare February 5, 2026 01:09

Andy-Jost commented Feb 5, 2026

View reviewed changes

Conversation

Andy-Jost commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Cythonization

Resource Handles

Error Handling

Validation & API

Code Quality

Tests

Test plan

Uh oh!

copy-pr-bot bot commented Feb 3, 2026

Uh oh!

Andy-Jost commented Feb 3, 2026

Uh oh!

github-actions bot commented Feb 3, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

Andy-Jost Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

rwgk Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Andy-Jost Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

rwgk Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Andy-Jost Feb 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rwgk Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Andy-Jost commented Feb 4, 2026

Uh oh!

Andy-Jost Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Andy-Jost commented Feb 4, 2026

Uh oh!

Andy-Jost commented Feb 5, 2026

Uh oh!

Andy-Jost commented Feb 5, 2026

Uh oh!

Andy-Jost commented Feb 5, 2026

Uh oh!

Andy-Jost Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Andy-Jost commented Feb 3, 2026 •

edited

Loading

Andy-Jost Feb 5, 2026 •

edited

Loading