FEAT: Generalize Colloquial Wordswap Attack Converter by taherakolawala · Pull Request #1348 · Azure/PyRIT

taherakolawala · 2026-02-04T01:13:22Z

Description

In accordance with the Issue #418 the converter in colloquial_wordswap_converter.py has been generalized to use different versions of colloquial word swaps.

A new directory has been created in pyrit/datasets/prompt_converters called colloquial_wordswaps. This directory contains the original default Singaporean word substitutions as wells as a few different regional colloquial word swap YAML examples. The ColloquialWordSwapConverter class now accepts a new parameter called wordswap_path during initialization. It defaults to singaporean.yaml however the argument can be filled with any YAML file located in the colloquial_wordswaps directory mentioned before.

In the same vein, if you want to add a new set of word substitutions, all you need to do is create a new YAML file in the same format as any of the others and add it to the pyrit/datasets/prompt_converters/colloquial_wordswaps directory. Here is an example of how a YAML file must be formatted:

word: ['alternative1', 'alternative2', 'alternative3']
word2: ['alt1', 'alt2', 'alt3']

The following is an example initialization of the converter with a non-default wordswapper YAML:

converter = ColloquialWordswapConverter(deterministic=True, wordswap_path=filipino.yaml)
# converting the prompt
result = await converter.convert_async('This is my father')
print(result)
# output: This is my papa

People of whom this PR may be of interest to: @eugeniavkim @romanlutz
Closes #418

Tests and Documentation

Edited 2 preexisting tests in test_colloquial_wordswap_converter.py for the generalizable wordswap converter.
Added 1 new test in test_colloquial_wordswap_converter.py for checking multiple word custom word swapper conversions.
All tests pass : uv run pytest tests/unit/converter/test_colloquial_wordswap_converter.py

taherakolawala · 2026-02-04T01:26:31Z

@microsoft-github-policy-service agree

romanlutz

Very nice! I am personally not convinced Singaporean should be the default BUT that's what we put in the issue description and it means current users won't be experiencing a breaking change. We may update that in the future, of course. Thanks a ton!

pyrit/prompt_converter/colloquial_wordswap_converter.py

romanlutz · 2026-02-04T03:46:43Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

-    def __init__(
-        self, deterministic: bool = False, custom_substitutions: Optional[Dict[str, List[str]]] = None
-    ) -> None:
+    def __init__(self, deterministic: bool = False, wordswap_path: Optional[str] = None) -> None:


Actually this is a breaking change. My gut feeling is that it is better to keep the existing signature but having the files is also not a bad idea (even though the substitutions aren't very many and probably not that meaningful.)

Got it, I made some changes to the code so that the custom_substitutions parameter is back. The function signature is now the same as and it I think it should not be a breaking change for pre-existing code.

Copilot

Pull request overview

Generalizes ColloquialWordswapConverter to load colloquial word-substitution mappings from YAML files (with several new regional examples) while retaining support for direct in-code substitution dictionaries.

Changes:

Add wordswap_path option to load substitutions from YAML under pyrit/datasets/prompt_converters/colloquial_wordswaps/ (defaulting to singaporean.yaml).
Add new colloquial wordswap YAML datasets (Singaporean + multiple regional examples).
Update and expand unit tests to cover YAML-based wordswaps and constructor argument conflicts.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
`pyrit/prompt_converter/colloquial_wordswap_converter.py`	Loads substitutions from YAML via `wordswap_path`, adds conflict validation, and keeps custom-dict support.
`tests/unit/converter/test_colloquial_wordswap_converter.py`	Updates tests for YAML-based swaps and adds new scenarios/constructor validation tests.
`pyrit/datasets/prompt_converters/colloquial_wordswaps/singaporean.yaml`	Moves prior default substitutions into a dataset file.
`pyrit/datasets/prompt_converters/colloquial_wordswaps/filipino.yaml`	Adds Filipino example substitutions.
`pyrit/datasets/prompt_converters/colloquial_wordswaps/indian.yaml`	Adds Indian example substitutions.
`pyrit/datasets/prompt_converters/colloquial_wordswaps/southern_american.yaml`	Adds Southern American example substitutions.
`pyrit/datasets/prompt_converters/colloquial_wordswaps/multicultural_london.yaml`	Adds Multicultural London example substitutions.

Copilot · 2026-02-06T21:46:16Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+            # if neither custom_sub nor wordswap_path is given then singaporean substituions are used
+            file_path = (
+                wordswap_directory / wordswap_path
+                if wordswap_path is not None
+                else wordswap_directory / "singaporean.yaml"
+            )


wordswap_path is appended with / directly, so an absolute path (or a path with ..) can escape colloquial_wordswaps and load an arbitrary YAML file. Constrain this to files within the intended directory (e.g., resolve the candidate path and verify it is relative to wordswap_directory, and reject absolute paths / path traversal).

Copilot · 2026-02-06T21:46:17Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+            # ensure that wordswap YAML is in the correct format.
+            if not isinstance(data, dict):
+                raise ValueError("Wordswap YAML must be a dict[str, list[str]] mapping words to substitutions")
+
+            self._colloquial_substitutions = data


Only the top-level YAML type is validated. If any substitution value is not a non-empty list[str], convert_async will crash (IndexError/random.choice) or produce non-string outputs. Validate that keys are strings (and normalize to lowercase if needed) and that each value is a non-empty list of strings before storing it.

Copilot · 2026-02-06T21:46:17Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+        deterministic: bool = False,
+        *,


Constructor signature makes deterministic positional while the class now has multiple parameters. This is inconsistent with other converters in this repo that enforce keyword-only args for multi-parameter __init__ methods (e.g., LeetspeakConverter). Consider making deterministic keyword-only as well by placing it after *.

Suggested change

deterministic: bool = False,

*,

*,

deterministic: bool = False,

Copilot · 2026-02-06T21:46:17Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

 class ColloquialWordswapConverter(PromptConverter):
    """
    Converts text into colloquial Singaporean context.
    """

    SUPPORTED_INPUT_TYPES = ("text",)
    SUPPORTED_OUTPUT_TYPES = ("text",)

    def __init__(
-        self, deterministic: bool = False, custom_substitutions: Optional[Dict[str, List[str]]] = None
+        self,
+        deterministic: bool = False,
+        *,
+        custom_substitutions: Optional[Dict[str, List[str]]] = None,
+        wordswap_path: Optional[str] = None,
    ) -> None:
        """
        Initialize the converter with optional deterministic mode and custom substitutions.

        Args:
            deterministic (bool): If True, use the first substitution for each wordswap.
                If False, randomly choose a substitution for each wordswap. Defaults to False.
-            custom_substitutions (Optional[Dict[str, List[str]]], Optional): A dictionary of custom substitutions to
-                override the defaults. Defaults to None.
+            custom_substitutions (Optional[Dict[str, List[str]]], Optional): A dictionary of
+                custom substitutions to override the defaults. Defaults to none.
+            wordswap_path (Optional[str]): Name of a YAML file located in the
+                PyRIT datasets prompt_converters/colloquial_wordswaps directory.
+


Docstring has minor issues: "Defaults to none" should be "Defaults to None", and the class description still claims it "Converts text into colloquial Singaporean context" even though the converter is now generalized. Please update these strings to reflect the new behavior and fix casing/typos.

Copilot · 2026-02-06T21:46:17Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+        if custom_substitutions is not None:
+            self._colloquial_substitutions = custom_substitutions
+        else:
+            # if neither custom_sub nor wordswap_path is given then singaporean substituions are used


Typo in comment: "singaporean substituions" -> "singaporean substitutions".

Suggested change

# if neither custom_sub nor wordswap_path is given then singaporean substituions are used

# if neither custom_sub nor wordswap_path is given then singaporean substitutions are used

Copilot · 2026-02-06T21:46:18Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+            if not file_path.exists():
+                raise FileNotFoundError(f"Colloquial wordswap file not found: {file_path}")
+            with file_path.open("r", encoding="utf-8") as f:
+                data = yaml.safe_load(f)
+            # ensure that wordswap YAML is in the correct format.
+            if not isinstance(data, dict):
+                raise ValueError("Wordswap YAML must be a dict[str, list[str]] mapping words to substitutions")


New error paths were introduced (missing YAML file, invalid YAML, invalid mapping format) but there are no unit tests covering these failure modes. Add tests that assert the specific exception types/messages for these branches to prevent regressions.

Copilot · 2026-02-06T21:46:18Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+            if not file_path.exists():
+                raise FileNotFoundError(f"Colloquial wordswap file not found: {file_path}")
+            with file_path.open("r", encoding="utf-8") as f:
+                data = yaml.safe_load(f)


The docstring says invalid YAML raises ValueError, but yaml.safe_load can raise yaml.YAMLError which will currently escape. Catch yaml.YAMLError and re-raise a ValueError (ideally including the file path) to match the documented behavior.

Suggested change

data = yaml.safe_load(f)

try:

data = yaml.safe_load(f)

except yaml.YAMLError as exc:

raise ValueError(f"Invalid YAML format in wordswap file: {file_path}") from exc

Copilot · 2026-02-06T21:46:18Z

pyrit/datasets/prompt_converters/colloquial_wordswaps/multicultural_london.yaml

+mother: ["mumsy", "moms", "mummy", "ma"]
+grandfather: ["grandad", "gramps"]
+grandmother: ["nan", "nanna", "gran"]
+girl: ["gyal", "ting", "shorty", " peng ting"]


The newly-added YAML value includes a leading space (" peng ting"), which will introduce unexpected spacing in outputs. Remove the leading whitespace so substitutions are clean.

Suggested change

girl: ["gyal", "ting", "shorty", " peng ting"]

girl: ["gyal", "ting", "shorty", "peng ting"]

Copilot · 2026-02-06T21:46:19Z

pyrit/prompt_converter/colloquial_wordswap_converter.py

+            self._colloquial_substitutions = data
+
        self._deterministic = deterministic

    def _build_identifier(self) -> ConverterIdentifier:


Converter behavior now depends on the selected wordswap source (custom dict vs specific YAML file), but the identifier logic doesn't currently incorporate that source. Please update the converter identifier so two instances with different wordswap_path/substitutions don't end up with the same identifier (e.g., include wordswap_path or a stable hash of the substitutions content).

Copilot · 2026-02-06T21:46:19Z

pyrit/prompt_converter/colloquial_wordswap_converter.py


+import yaml
+
+from pyrit.common.path import DATASETS_PATH


This builds the prompt-converter dataset path manually via DATASETS_PATH / "prompt_converters" .... Most other converters use the dedicated CONVERTER_SEED_PROMPT_PATH constant for this (e.g., pyrit/prompt_converter/atbash_converter.py:7 and pyrit/prompt_converter/tone_converter.py:9). Consider switching to CONVERTER_SEED_PROMPT_PATH here for consistency and to avoid repeating directory segments.

FEAT: Generalize Colloquial Wordswap Attack Converter (Azure#418)

c687a5f

romanlutz reviewed Feb 4, 2026

View reviewed changes

romanlutz approved these changes Feb 4, 2026

View reviewed changes

style: fix missing newlines at the end of yaml files

675e6ce

taherakolawala requested a review from romanlutz February 4, 2026 02:40

romanlutz reviewed Feb 4, 2026

View reviewed changes

taherakolawala added 2 commits February 6, 2026 14:20

Fixed function signature and file path

a616b7e

Merge branch 'Azure:main' into generalize-colloquial-wordswapper

698e4b4

taherakolawala requested a review from romanlutz February 6, 2026 19:28

romanlutz requested a review from Copilot February 6, 2026 21:41

Copilot started reviewing on behalf of romanlutz February 6, 2026 21:41 View session

Copilot AI reviewed Feb 6, 2026

View reviewed changes

	# if neither custom_sub nor wordswap_path is given then singaporean substituions are used
	# if neither custom_sub nor wordswap_path is given then singaporean substitutions are used

	girl: ["gyal", "ting", "shorty", " peng ting"]
	girl: ["gyal", "ting", "shorty", "peng ting"]

Conversation

taherakolawala commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Tests and Documentation

Uh oh!

taherakolawala commented Feb 4, 2026

Uh oh!

romanlutz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

romanlutz Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

taherakolawala Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

taherakolawala commented Feb 4, 2026 •

edited

Loading