Community Experiments
You propose the question. We write the method and freeze it, and you get to read it before the work starts — including the line about what would count as a blank. Then it goes up here, whatever it said.
How it works
You propose a question. We read every one, and the ones we could honestly answer go up for votes.
Votes tell us what to look at first. Treat it as a queue rather than a promise: forty well-organised people are a group, and the public is something else entirely. Better said now than found out later.
Before anything starts we write the method and freeze it, and you get to read it first — including the line saying what result would count as a blank.
Then it goes up here, whatever it said. A dull result is still a result, and it is the kind this site exists to publish.
Questions on the table
Empty so far. Yours could be the first, which is a good place in the queue to be.
Propose one
Sign in to propose a question or vote on one. It is the one way we can keep a single person from being forty of them, and we are sorry about it.
What we found
Nothing has been run yet, so this space is empty. You are looking at the honest version of it: a worked example dressed up as a result would fill the gap nicely and tell you less than the gap does.
The first will go up the day it finishes, whatever it found. A blank is the likeliest single outcome, and it is the one most sites quietly bin.
What a method looks like
Nobody has voted for this one. It is here so you can see the shape of what you are asking for when you propose a question. It is also a small, dull question, which is what makes a good experiment and a terrible headline.
Does asking an AI to write a complaint letter get a better reply than writing it yourself?
It is the single most common thing people bring to this site, and nobody anywhere has actually checked whether it changes what the shop writes back.
Written on 2026-09-30. Not running — nothing has started, so nothing is frozen yet.
What is being compared
Two letters about the same real problem — one drafted by a model and edited by the sender, one written by the sender alone — sent to the same kind of organisation.
How it is run
- Volunteers bring a real complaint they were going to make anyway. Nothing is invented for the test.
- Each person writes their own letter first, before seeing anything a model produced, so the order cannot flatter the tool.
- The drafted version is produced with our own Bureaucracy Bot and edited by the sender, who stays responsible for every fact in it.
- Pairs are matched by the kind of organisation and the kind of complaint, never by how the letters read.
- Replies are logged as they arrive, with the date, for eight weeks.
What counts as right
A reply that offers what was asked for, in writing, within eight weeks. Tone, length and politeness are recorded but do not count.
What would mean we found nothing
Fewer than thirty matched pairs come back, or the difference between the two groups is smaller than the difference between two halves of the same group. Either way we report that and stop.
Who checks
Every reply is read by two people who do not know which letter produced it.
What this cannot show, whatever it finds
- Whether this holds for anybody who did not volunteer. People who sign up to an experiment on an AI site are not a sample of the country.
- Whether a different model, or the same one next month, would do the same thing.
- Anything about complaints where the law is on a timer. Those go to Citizens Advice, not to us.