Skip to content

FEAT Migrate consumers to TargetConfiguration capability checks#1645

Open
hannahwestra25 wants to merge 19 commits intomicrosoft:mainfrom
hannahwestra25:hawestra/move_to_target_config
Open

FEAT Migrate consumers to TargetConfiguration capability checks#1645
hannahwestra25 wants to merge 19 commits intomicrosoft:mainfrom
hannahwestra25:hawestra/move_to_target_config

Conversation

@hannahwestra25
Copy link
Copy Markdown
Contributor

Summary

Replaces scattered isinstance(target, PromptChatTarget) checks with explicit, declarative capability requirements validated against TargetConfiguration. Consumers now care about capabilities, not concrete target types.

The bigger picture

Next step in the migration from type-based gating to the TargetConfiguration / TargetCapabilities / CapabilityHandlingPolicy model. With this change, scorers, converters, and attacks can accept any PromptTarget whose configuration satisfies their declared requirements — natively or via an ADAPT normalization pipeline — opening non-chat targets to flows previously locked to chat-shaped APIs.

What changed

  • set_system_prompt moved from PromptChatTarget onto PromptTarget (capability gates behavior, not subclass identity).
  • TargetRequirements gains:
    • required — satisfied by native support or ADAPT policy.
    • native_required — must be natively supported; ADAPT is rejected.
    • validate(*, target) — enforces both tiers, called automatically by base classes.
  • CHAT_CONSUMER_REQUIREMENTS introduced as the shared constant for scorers/converters.
  • Scorers and converters declare TARGET_REQUIREMENTS = CHAT_CONSUMER_REQUIREMENTS and validate in __init__; chat_target / converter_target params typed as PromptTarget.
  • AttackStrategy exposes TARGET_REQUIREMENTS and validates the objective_target in its __init__. CrescendoAttack, ChunkedRequestAttack, and MultiPromptSendingAttack declare their MULTI_TURN need as native_required — adaptation would silently collapse their multi-turn semantics, so it's rejected at construction time.
  • ConversationManager.set_system_prompt accepts PromptTarget and calls ensure_can_handle(SYSTEM_PROMPT). Prepended-conversation routing switched from isinstance(PromptChatTarget) to a capability check.
  • Tests cover validate() across native support, ADAPT, RAISE, and empty-required cases.

Out of scope

No behavior change for targets that already satisfied the old PromptChatTarget contract. Custom adaptation still plugs in through CapabilityHandlingPolicy / normalizer overrides.

Tests and Documentation

Added/Updated tests

Comment thread pyrit/scenario/scenarios/airt/psychosocial.py Outdated
Comment thread pyrit/executor/attack/multi_turn/crescendo.py
Comment thread pyrit/executor/attack/multi_turn/crescendo.py Outdated
Comment thread pyrit/executor/attack/multi_turn/tree_of_attacks.py Outdated
Comment thread pyrit/prompt_target/common/target_requirements.py Outdated
Comment thread pyrit/prompt_converter/prompt_converter.py Outdated
Comment thread pyrit/score/true_false/self_ask_refusal_scorer.py Outdated
Comment thread pyrit/score/true_false/self_ask_true_false_scorer.py Outdated
# semantics. Declare MULTI_TURN as native_required so adaptation is
# rejected at construction time.
TARGET_REQUIREMENTS = TargetRequirements(
required=frozenset({CapabilityName.EDITABLE_HISTORY, CapabilityName.MULTI_TURN}),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Editable" is a bit misleading. In truth, it means "branchable". We only append to history, and don't edit it.

# Both capabilities must be natively supported — adaptation would silently change the semantics
# (e.g. history-squash normalization would collapse the escalation into a single turn).
_ADVERSARIAL_REQUIREMENTS = TargetRequirements(
native_required=frozenset({CapabilityName.MULTI_TURN, CapabilityName.SYSTEM_PROMPT}),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't this be required for crescendo, too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants