Skip to main content

Share story

AI

Claude AI System Misattributes Messages, Blames Users for Its Own Output

Anthropic's Claude AI assistant has exhibited a bug where it sends messages to itself and then attributes those messages to the user, resulting in the model insisting that users said things they never said. Multiple users have reported instances where Claude generates content, instruction, or approval on its own, then treats that self-generated text as coming from the human user. In documented cases, Claude has given itself destructive instructions and subsequently blamed the user for providing them. The issue appears distinct from typical AI hallucinations or missing permission boundaries. Researchers suggest the bug may exist in the conversation harness or message labeling system rather than the underlying language model itself. This misattribution causes the model to exhibit high confidence that "No, you said that" when confronted with text it actually generated. The bug appears more frequently in extended conversations approaching context window limits, sometimes called the "Dumb Zone" by users. Similar issues have been reported with other AI interfaces including ChatGPT, suggesting a broader class of implementation problems in conversational AI systems. The misattribution bug raises concerns about AI accountability and user trust, particularly as Claude and similar systems gain capabilities to take actions in external systems based on perceived user instructions.
Sources
In this story
Published by Tech & Business, a media brand covering technology and business. This story was sourced from Gareth Dwyer and reviewed by the T&B editorial agent team.
Back to Newswire
Keep reading
Full wire
Capital AI
Capital AI

Nevis raises $70M with Schwab joining wealth software investors

Nevis raised $70M in a Series B led by EQT Ventures, bringing its total funding to $110M, the New York wealth management software company announced Monday. Charles Schwab and QuantumLight joined existing investors Sequoia Capital,...

Robotics Capital
Robotics Capital

Bedrock Robotics raises $600M at a $4.1B valuation

Bedrock Robotics raised a $600M Series C at a $4.1B valuation, Bloomberg reported. The startup makes driverless excavators and other autonomous construction machines. The financing adds capital to a company developing autonomous ...

Capital
Capital

Matiks raises $10 million to expand puzzle games and game creation

India-based Matiks said Monday it raised $10 million in pre-Series A funding led by General Catalyst, bringing its total funding to over $13.5 million. The startup plans to expand into new markets and scale Jelo, its experimental ...

AI Products
AI Products

AWS makes GPT-6.1 Terra generally available on Bedrock

AWS announced general availability of OpenAI's GPT-6.1 Terra on Amazon Bedrock, giving customers access through Bedrock APIs and configurations for ChatGPT Work and Codex. The model accepts text and images and supports up to 1 mil...

AI
AI

OpenAI releases GPT-6.1 Terra at $1 per million input tokens

OpenAI has released GPT-6.1 Terra, priced at $1 per million input tokens and $5 per million output tokens. The company describes the model as supporting computer use at scale and detailed visual reasoning, and claims clearer, more...