SparkKubernetesSensor rejects deferrable=True (Invalid arguments)

[nikhilkarve]

Hi team,

I’m trying to make a SparkKubernetesSensor non-blocking by using deferrable=True, but my DAG fails at parse/init time with:

TypeError: Invalid arguments were passed to SparkKubernetesSensor (task_id: ack_spark_job_sensor).
Invalid arguments were: **kwargs: { 'deferrable': True}

Snippet:

from airflow.providers.cncf.kubernetes.sensors.spark_kubernetes import SparkKubernetesSensor

spark_job_sensor = SparkKubernetesSensor(
    task_id="ack_spark_job_sensor",
    poke_interval=60,
    timeout=3600,
    attach_log=True,
    namespace="...",
    kubernetes_conn_id="...",
    application_name="...",
    deferrable=True,
)

If we are using it wrong, what would be the correct way to use it?

Environment:

Airflow version: 3.0.6
apache-airflow-providers-cncf-kubernetes version: 10.9.0