Skip to content

Conversation

@jaeopt
Copy link
Owner

@jaeopt jaeopt commented Feb 2, 2026

Summary

This PR excludes CMAB (Contextual Multi-Armed Bandit) experiments from User Profile Service (UPS) for sticky bucketing.

Jira Ticket: FSSDK-12262

Changes

  • Updated optimizely/decision_service.py to exclude CMAB experiments from UPS
  • Added condition and not experiment.cmab in two critical locations:
    1. Line 461: When looking up stored variation from UPS
    2. Line 533: When saving variation decision to UPS

Rationale

CMAB should not use UPS for sticky bucketing because:

  • UPS maintains decisions across the experiment lifetime without considering TTL or user attributes
  • This contradicts CMAB's dynamic nature which requires fresh decisions based on current context

Test Coverage

New Tests Added (3):

  • test_get_variation_cmab_experiment_excludes_user_profile_service_lookup
  • test_get_variation_cmab_experiment_excludes_user_profile_service_save
  • test_get_variation_non_cmab_experiment_uses_user_profile_service

Test Results: ✅ All 51 tests pass (no regressions)

Quality Assurance

  • QA Iterations: 1/5 (SUCCESS on first try)
  • Tests: 51/51 passed
  • Code Review: APPROVED
  • Status: Production-ready

Files Modified

  • optimizely/decision_service.py (6 lines changed)
  • tests/test_decision_service.py (180 lines added)

🤖 Generated with Claude Code

- Updated decision_service.py to exclude CMAB experiments from UPS
- CMAB should not use UPS for sticky bucketing due to dynamic nature
- Added check: `and not experiment.cmab` in two locations
  1. When looking up stored variation from UPS (line 461)
  2. When saving variation decision to UPS (line 533)

Test Coverage:
- Added 3 comprehensive unit tests (all passing)
- test_get_variation_cmab_experiment_excludes_user_profile_service_lookup
- test_get_variation_cmab_experiment_excludes_user_profile_service_save
- test_get_variation_non_cmab_experiment_uses_user_profile_service
- All 51 tests in test_decision_service.py pass

QA Summary:
- Iterations: 1/5 (SUCCESS on first try)
- Tests: 51/51 passed
- Code Review: APPROVED (clean, minimal changes)
- Files Modified: optimizely/decision_service.py, tests/test_decision_service.py

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@jaeopt jaeopt closed this Feb 2, 2026
@jaeopt jaeopt deleted the ai/jaeopt/fssdk-12262-cmab-ups branch February 2, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants