PythonVirtualenvOperator: render_template_as_native_obj=True causes PicklingError when using {{ ti }} / {{ task_instance }} (cloudpickle/structlog)

[akhundovte]

Hi Airflow team,

The PythonVirtualenvOperator docs (“Passing in arguments”) mention that Airflow does not support serializing ti / task_instance. I understand this limitation and that a workaround is to pass only identifiers and, if needed, fetch task instance data via the Airflow API.

However, I found a reproducible case where the behavior changes depending on render_template_as_native_obj, and enabling native rendering makes the task fail during argument serialization.

Minimal repro

from pendulum import datetime

from airflow.sdk import dag
from airflow.providers.standard.operators.python import PythonVirtualenvOperator


def venv_callable(
    ti,
    task_instance
):
    print("ti =", ti)
    print("task_instance =", task_instance)


@dag(
    start_date=datetime(2026, 1, 1),
    schedule=None,
    catchup=False,
    render_template_as_native_obj=False,
)
def test_simple():
    PythonVirtualenvOperator(
        task_id="repro",
        python_callable=venv_callable,
        python_version="3.10",
        serializer="cloudpickle",
        op_kwargs={
            "x": 1,
            "ti": "{{ ti }}",
            "task_instance": "{{ task_instance }}",
        },
        requirements=[
            "apache-airflow==3.1.6"
        ],
        system_site_packages=False,
    )


test_simple()

Observation

• With render_template_as_native_obj=False, the task succeeds (values are printed).
• If I change only render_template_as_native_obj=True, the task fails while serializing arguments for the subprocess.

Error with render_template_as_native_obj=True

[2026-01-29 23:06:37] INFO - Use 'cloudpickle' as serializer.
[2026-01-29 23:06:37] ERROR - Task failed with exception
PicklingError: Only BytesLoggers to sys.stdout and sys.stderr can be pickled.
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1004 in run
...
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 529 in _write_args
File "/home/airflow/.local/lib/python3.12/site-packages/cloudpickle/cloudpickle.py", line 1537 in dumps
...
File "/home/airflow/.local/lib/python3.12/site-packages/structlog/_output.py", line 278 in __getstate__

Questions

1. Can you confirm whether this behavior difference is an expected consequence of render_template_as_native_obj=True (Jinja NativeEnvironment), where {{ ti }} / {{ task_instance }} render to native objects that then get pickled for PythonVirtualenvOperator?

2. Is there any plan to provide a supported mechanism for isolated environments instead of passing the “live” TaskInstance object, for example:

   • a lightweight serializable proxy/handle (e.g., dag_id, run_id, task_id, try_number, map_index) and
   • a limited set of operations (or a small client) to retrieve basic TaskInstance info from within PythonVirtualenvOperator without serializing the full object?

I understand one can call the Airflow API directly, but an official/supported approach (or at least clearer validation / a more explicit error message for this case) would be very helpful.

Environment

• Airflow: 3.1.6
• Provider: standard (PythonVirtualenvOperator)
• Python: 3.12 (runner), virtualenv: 3.10
• serializer: cloudpickle

Thanks!

[Felix-neko]

I recently encountered the same problem. It worked on airflow 2, but not on version 3.

[pyroman30]

Same here

[potiuk]

I suggest you open an issue about it.