Intelligent Ops Proposal Deduplication
Prevent duplicate work proposals using freshness gates, fuzzy title matching, and pattern-based dedup across queue and active proposals.
Overview
As research systems generate work proposals automatically, duplicate or stale proposals can flood the approval queue. This system implements three layers of deduplication: freshness gates (only recent data generates proposals), queue fuzzy matching (checks pending/approved proposals by title similarity), and within-run pattern tracking (max one proposal per competitor-pattern combo per execution). Oversaturated patterns require high-impact signals to trigger proposals.
How It Works
The proposal generation agent first filters input data to entries less than 3 days old (with exceptions for high-impact items never processed). It fetches all open proposals from the ops API and fuzzy-matches new proposal titles against existing ones using a 0.72 similarity threshold. It also checks exact competitor:pattern key matches. Within a single run, it tracks created proposals by pattern and blocks duplicates. Low-signal patterns (technology adoption, budget funding) require high-impact classification to generate proposals. When no research qualifies, the system fills remaining slots with fresh industry trend proposals.
Tools Used
Outcome
Eliminated 14 duplicate intel proposals in first cleanup run. Reduced proposal volume from 8/day to 5/day while increasing approval rate from 62% to 71%. Intel pipeline now focuses on quality signals and fresh data, preventing proposal fatigue for leadership.