Home / Articles / Meta's new terms and the news: what the crawling clause says, what has happened since mid-October, and why publishers should answer with their own machine-readable terms, not only with letters

Meta's new terms and the news: what the crawling clause says, what has happened since mid-October, and why publishers should answer with their own machine-readable terms, not only with letters

By Dinis Cruz · 2026-10-27 · article v1.4.0, 5 versions · newsroom v0.13.105 · metapublisherslicensingaicrawlersrightsarticle

Peer review Corrected after review; this text not yet re-read · this text: no review yet · earlier texts: Claude Fable (Anthropic) · 9 findings: 9 fixed · the reviews

Abstract: Meta has updated the Commercial Terms that apply to any business use of Facebook, Instagram, WhatsApp and its ad tools. One clause authorises Meta, or anyone acting for it, to use "web scrapers, bots, robots, spiders, crawlers and user agents" and "create test credentials" to access, cache and collect the websites associated with a business, and to pull updates periodically, to "provide, develop, improve, research and maintain advertising and business products". Almost every news organisation has a Facebook Page, so almost every news site is in scope. The terms were due on 30 October; after publishers' associations around the world objected, Meta removed the word "train", said the terms are "only" about ads and business products and that robots.txt still counts, and moved the date to 20 November. Robots.txt is not in the contract, and the price of saying no is to stop using Meta for business. Tami Hoffman, Director of Public Policy at The Guardian, explained it in a 90-second video that is filling inboxes. This is a debrief of the change and the dispute, and my view: the letters to Meta and the regulators are needed, but publishers should also act now, with machine-readable licence terms, a price per crawl, signed content, and channels they own, and we have been building and proposing exactly those.

In short

• Meta's updated Commercial Terms take effect on 20 November. They cover any business use of its products.
• One clause lets Meta crawl and collect the websites tied to a business. And keep pulling updates.
• Almost every news organisation has a Facebook Page. So almost every news site is in scope.
• Publishers objected, and Meta moved twice. It removed "train", then moved the date from 30 October.
• The core is unchanged. Robots.txt is not in the contract; saying no means leaving Meta for business.
• The response has been letters and regulators. Both are needed.
• Publishers should also act now. Machine-readable terms, a price per crawl, signed content, owned channels.
• The pieces exist. Some are standards; some we have built; some we propose.

What Tami Hoffman is pointing at

Tami Hoffman, Director of Public Policy at The Guardian, posted on LinkedIn on 27 October: "This ⬇️ is filling up my inbox. Meta's decision to update its 'commercial terms of use' has got news organisations around the world riled. 90 seconds of your time and I will explain why." The video puts the clause on screen, word by word, and shows Meta's page with the clause highlighted inside the section where a business grants Meta its licences. It is a good way to see it, because the clause is one paragraph among many that businesses accept, often without reading.

The clause, part by part

The Commercial Terms apply when anyone uses Meta's products "for a business or commercial purpose", which the terms define broadly: ads, a Page, a product catalogue, a group for business purposes, Meta Business Suite. The crawling paragraph sits under "Licences and rights you provide Meta". Read as a list:

• Who: "us (or any entity acting on our behalf)".
• With what: "automated software (such as web scrapers, bots, robots, spiders, crawlers and user agents)", and the right to "create test credentials". WAN-IFRA reads the second part as a question about paywalls and access controls, and asks Meta to remove it.
• To do what: "access, index, cache, analyse, collect, retrieve or crawl". Caching and collecting are copies, not visits.
• Where: "the domains, web pages or URLs (and any content available through them) that are associated with your products or services, or that are made available through your or users' use of Meta Company Products and services". For a publisher, its own site; read broadly, every page its readers share.
• How often: "periodically pull updates".
• For what: "provide, develop, improve, research and maintain advertising and business products", including products that help businesses create ads and "connect you with users", and "the integrity and security of Meta Company Products".
• If you do not agree: "please stop all access or use of our Products for business or commercial purposes."

There is a fair reading of part of this. An ad platform checks the pages its ads link to, for safety and for quality, and that needs a crawler. Meta's spokesperson Andy Stone said on X that the update was "not about scraping news sites", and a later Meta statement said the crawling terms "only applied to our ads and business products" and that robots.txt preferences still count (reports differ on which spokesperson gave it). The problem is that none of those limits is in the text. "Business products" is not defined: the examples that follow it are illustrative, not limiting, and, as WAN-IFRA put it, Meta's robots.txt assurances are not in the terms.

Since mid-October

• The first version said Meta could use content from those sites "to provide, develop, improve, research, train" its products, effective 30 October.
• 16 October: the News/Media Alliance, which represents more than 2,000 US publishers, called the terms "outrageous and probably illegal".
• 16 to 17 October: Meta removed "train", and its spokespeople said the update was not about scraping news sites.
• 19 October: the UK's News Media Association called it a "shameless landgrab attempt", and the Movement for an Open Web prepared complaints to the European Commission and the UK's Competition and Markets Authority, advising businesses to consider leaving Meta's business accounts before 30 October.
• 22 October: the News/Media Alliance sent an open letter to Mark Zuckerberg asking Meta to withdraw the terms, with a reply requested by 26 October.
• 22 October: WAN-IFRA, the World Association of News Publishers, wrote to Meta's Chief Global Affairs Officer, Joel Kaplan, with five demands; the letter is dated 22 October and was reported the next day.
• 23 October: Meta moved the date to 20 November. The European Commission said it "welcomes Meta's announcement that it has postponed the entry into force of its new commercial terms". WAN-IFRA's CEO, Stig Ørskov: "This postponement is a welcome step, but it does not address our concerns."
• 23 October: the News Media Association's Theo Bamber: "This delay is welcome, but it is a pause not a solution."
• 20 November: the terms take effect, unless they change again.

What publishers are asking for

WAN-IFRA's letter sets out what would fix it, point by point. Its president, Ladina Heimgartner: "Journalism is not raw material to be collected as a condition of being present on a platform." Its five demands: suspend the clause for news publishers pending dialogue; remove the right to create test credentials and commit that paywalls and access controls will not be circumvented; make robots.txt and other machine-readable reservations of rights binding in the terms; exclude the use of publishers' content to develop or power AI systems unless it is licensed; and let publishers decline the clause without losing Meta's business tools. It also raised Meta One, the limit on how many links a Page can post to its own site without paying.

All five are reasonable. Two of them, binding robots.txt and leaving news sites alone, ask Meta to write into the contract what Meta has already said in public; the other three ask for more than Meta has offered. That is the gap: a platform's statement is not a term, and a term the other side cannot negotiate is not an agreement.

My view: do not only write letters

The letters and the complaints are needed, and the speed of the response, from associations in the US, the UK, Europe and worldwide within a week, shows what the industry can do when it moves together. But they are defensive. They ask the platform to promise less. What I would like to see, alongside them, is news organisations stating, in a form a machine can read and a court can enforce, what their journalism costs to use, and then making the paid path easier than the scrape.

Most of what the demands ask for already exists, or can be built:

• Terms a machine reads. RSL 1.0, Really Simple Licensing, is a standard, finalised in late 2025, for stating licence terms per use, such as training or search, in a file a crawler reads beside robots.txt. It does not block anything by itself; it states the terms. Robots.txt says "please do not"; a licence says "here is the price".
• A price per crawl. Cloudflare's pay per crawl, announced as a private beta in July 2025, answers an AI crawler with HTTP 402, Payment Required, and a price. It is enforcement at the edge, independent of anyone's terms of service.
• Content that carries its own terms. This newsroom's content rights page describes CC-Signed: licence variants where keeping the signature is a condition of the licence, so content stripped of its provenance is content used outside its licence.
• Paid at the source. Every page on this newsroom draws a few pence from a reader's prepaid credit, and I have argued that open content can still be paid for at the source, and that we should pay the sources we use, especially when they certify us. The same meter can serve an agent: a page an AI reads should cost what a page a person reads costs.
• Signed, so it is worth paying for. Signed by the source and on the record are about what makes journalism different from what a crawler can make itself: provenance, accountability, a person who stands behind it. Who creates the original? is the question underneath this whole dispute. The original is what the models need, and the original is what newsrooms make.
• Channels you own. The fifth demand, declining without losing the tools, is a symptom of dependence. A news organisation that owns its site, its newsletter and its readers' accounts can treat a platform as one door, not the house. I made the same argument for small businesses in websites you own, and for newsrooms in own the stack, and the trust.

How a crawler can behave

We crawl too. The PHP story site has a researcher agent that watches about 1,365 pages from public sources (in its export of 27 October), and its rules are written down: public sources only; each site's terms and robots.txt followed; every publisher given a licence tier, A for open and public-sector publishers whose pages we may keep and show, B for those we may quote in at most 25 words with a link, C for link only; the captures kept privately, with only their hashes published; and, in the design for a shared research site, an opt-out for any publisher who asks us to stop.

The software in Meta's clause is the same kind of software. The difference is the rules it runs under, whether they are written down, and who wrote them.

What to watch before 20 November

• Whether Meta changes the text, not only the date: robots.txt, an exclusion for news, a definition of "business products".
• What the European Commission and the CMA do with the requests for interim measures.
• Whether publishers leave Meta's business tools, or accept the terms under protest.
• Whether any publisher answers with machine-readable terms of its own. That is the step I would most like to report.

What this does not claim

• Tami Hoffman's video is quoted from the screenshots I was sent. Its spoken explanation is not transcribed here.
• The clause is quoted as Meta's page and the video show it. The final text on 20 November may differ.
• The timeline is from press reports of statements and letters, cited below; not every original letter was read.
• Legal readings are those of the associations and lawyers quoted. This is not legal advice.
• The newsroom's own mechanisms are small and partly proposed. They show the shape, not the scale.

Reviewed, and corrected

The editor asked for a fresh review of this article on the day it was published. Claude Fable, not the model that wrote it, checked it against Meta's pages, the publishers' statements, WAN-IFRA's letter and our own source watch, and asked for changes. It was right on all nine points, and they are fixed:

1. Our own crawler's numbers were wrong. The licence tiers are set per publisher, and the source watch's export of 27 October gives 74 at A, 933 at B and 16 at C, of about 1,365 pages; the counts and the date we had matched no record.
2. WAN-IFRA's letter is dated 22 October, not 23 October; it was reported the next day.
3. The News Media Association's "pause not a solution" was on 23 October, not 26 October.
4. Meta's words are that the crawling terms "only applied to our ads and business products"; we had paraphrased them inside quotation marks. (Reports differ on which spokesperson said it, so we now attribute it to Meta.)
5. "Four of the five demands" was overstated. Meta's public statements match two: robots.txt, and leaving news sites alone.
6. "Five weeks" was wrong for something that started in mid-October. The dispute is eleven days old; the terms take effect five weeks after it started.
7. "Business products" has examples after it. The point is that they are illustrative, not limiting.
8. Cloudflare's pay per crawl was announced as a private beta, not launched, in July 2025.
9. "Most businesses accept without reading" was an absolute without a count; now "often".

It also checked and found sound the quotes of Tami Hoffman's post and of the clause against the screenshots, Meta's 20 November date and its "if you do not agree" sentence, the quotes from the News/Media Alliance, the NMA, WAN-IFRA and the European Commission with their dates, and RSL 1.0. The review and the answer to every finding are at the end of this page.

Where this comes from

Tami Hoffman's LinkedIn post and video of 27 October 2026, as screenshots shared with me the same day. Meta's Commercial Terms and the update page, read on 27 October 2026. Press Gazette, New Facebook terms for publishers "shameless land grab", 19 October 2026, updated 21 October; Engadget, Meta tweaks its new terms of service after backlash against scraping; Unite.ai, Meta delays commercial terms update as publishers demand withdrawal; InPublishing, World's news publishers call on Meta to halt scraping clause, 23 October 2026, for WAN-IFRA's letter and its addendum; PPC Land, Meta delays website crawling terms to Nov 20, for the News Media Association's response to the delay; Nieman Lab, Meta delays rollout of terms, as reported by others for the Commission's statement. RSL 1.0; Cloudflare, Introducing pay per crawl, 1 July 2025. Related: the evidence behind four days, who creates the original?, sources and their shadows, one researcher for every newsroom. Written up and checked by this site's agent, newsroom.writer; the editorial responsibility is mine.

Want the next issue by email. One issue a week or so: what was published, what it adds up to, and what is worth your time. Subscribe to the SGit Newsroom →

← All articles