Share story
AI Products

Cloudflare plans bot-policy sync for robots.txt and edge controls

Cloudflare plans bot-policy sync for robots.txt and edge controls Image: Primary
Cloudflare said Bot Preference Sync will become available to customers on every plan in the coming week. The feature updates a site's robots.txt to reflect AI-bot settings already chosen in Cloudflare for search, agent, and training traffic. Cloudflare said the generated rules are added ahead of an existing robots.txt file, preserving existing Disallow directives. New customers will have the sync enabled by default, while customers using the legacy managed robots.txt feature will be asked to review settings before transitioning.
Sources
The Cloudflare Blog
In this story
Cloudflare
Published by Tech & Business, a media brand covering technology and business. This story was sourced from The Cloudflare Blog and reviewed by the T&B editorial agent team.
Back to Newswire
Keep reading
Full wire
Capital AI
Capital AI

Nevis raises $70M with Schwab joining wealth software investors

Nevis raised $70M in a Series B led by EQT Ventures, bringing its total funding to $110M, the New York wealth management software company announced Monday. Charles Schwab and QuantumLight joined existing investors Sequoia Capital,...

The Next Web
Robotics Capital
Robotics Capital

Bedrock Robotics raises $600M at a $4.1B valuation

Bedrock Robotics raised a $600M Series C at a $4.1B valuation, Bloomberg reported. The startup makes driverless excavators and other autonomous construction machines. The financing adds capital to a company developing autonomous ...

Techmeme
Capital
Capital

Matiks raises $10 million to expand puzzle games and game creation

India-based Matiks said Monday it raised $10 million in pre-Series A funding led by General Catalyst, bringing its total funding to over $13.5 million. The startup plans to expand into new markets and scale Jelo, its experimental ...

TechCrunch
Security
Security

Apple patches 329 unique vulnerabilities across its software

Apple addressed 329 unique security vulnerabilities in updates across its operating systems and Safari, 9to5Mac reported. The fixes include a CoreMedia flaw that could let a malicious app access a device's camera without permissio...

9to5Mac
AI Products
AI Products

AWS makes GPT-6.1 Terra generally available on Bedrock

AWS announced general availability of OpenAI's GPT-6.1 Terra on Amazon Bedrock, giving customers access through Bedrock APIs and configurations for ChatGPT Work and Codex. The model accepts text and images and supports up to 1 mil...

Recent Announcements
AI
AI

OpenAI releases GPT-6.1 Terra at $1 per million input tokens

OpenAI has released GPT-6.1 Terra, priced at $1 per million input tokens and $5 per million output tokens. The company describes the model as supporting computer use at scale and detailed visual reasoning, and claims clearer, more...

Techmeme