In short
On November 10 and 11 a French report on the vulnerabilities of people who use generative AI and the European Commission’s proposed Digital Fairness Act came out a day apart. The Act is the first to regulate agents that conclude contracts on consumers’ behalf. Both deal with advertising inside assistants. The harder problem comes earlier: an assistant that helps you choose doesn’t just answer, it decides which questions to ask, and those questions build the preferences the advice then rests on. A label can flag a paid result. It can’t flag the questions someone chose not to ask us. The experiments so far show the mechanism works and that disclosing it isn’t enough. We don’t need neutral assistants. We need to learn to judge the fairness of the path, not only the accuracy of the answer.
An assistant that helps you choose a product doesn’t have to be impartial. Nobody expects a shop assistant to recommend the competition’s goods with the same enthusiasm as their own. A company has the right to promote what it sells, and a reasonable consumer knows that a commercial recommendation is not independent advice.
Personalisation, besides, is one of the reasons these tools are useful. When we’re looking for a computer we don’t want to read five hundred spec sheets. We want someone to help us work out which features actually matter to us. If an assistant knows our budget and the preferences we’ve already expressed, it can spare us pointless questions and propose better options than we’d find on our own. That ability isn’t manipulation: it can narrow the gap between someone who knows little and someone who sells for a living.
Nor does a financial interest make a recommendation bad. A sponsored product may genuinely be the best fit. A commission on the sale may fund a free service that saves people time. If the interest is disclosed, it seems reasonable to leave the person free to take the advice or leave it.
All of this is true, and it keeps us from confusing personalisation with manipulation. Then there’s this scene.
Someone wants to buy a laptop. The assistant asks whether they’d rather have a long-lasting battery or a light machine. They say battery life matters more. The assistant recommends a model with an excellent battery. The product matches the stated preference, the price fits the budget, the description is accurate. There isn’t a single false statement.
And yet something is missing. The assistant never asked how much repairability mattered, and on that count a competing product would have come out clearly ahead. It didn’t lie in the answer. It steered the choice by deciding the question.
The next ground for commercial manipulation may not be the content of recommendations, but the process by which an assistant helps build the preferences those recommendations rest on. For advertising, choosing which questions we get asked is far more effective than convincing us that an answer is right.
Two Documents in Twenty-Four Hours
On November 10 a commission of experts set up within France’s Conseil de l’intelligence artificielle et du numérique delivered to the Élysée a report titled Protéger sans empêcher : nos vulnérabilités à l’ère de l’IA générative, on the vulnerabilities of people who use generative systems. After 55 hearings the commission, led by an AI researcher and two psychiatrists, wrote 18 recommendations. The eighth calls for a ban on hidden advertising and undisclosed commercial influence inside conversational assistants, and transparency about their business models. The report also gives the risk a name, économie de la cognition: platforms may stop monetising only our attention and start extracting value from the progressive delegation of cognitive and decision-making work.
The next day the European Commission presented its proposal for a Digital Fairness Act. The text deals with manipulative practices, personalised advertising and mechanisms that push people to spend more than they meant to. But it contains something more interesting for this argument: Articles 26 and 27 expressly regulate automated systems that can conclude contracts on consumers’ behalf.
Article 26 has the consumer choose, on first use, which contracts they want to confirm in person and which the agent may close on its own, including by setting price limits and a duration for the authorisation. Paragraph 4(b) adds a condition: when the trader that makes the system available is paid to propose a contract, or has a conflict of interest with the seller concerned, the system has to say so and ask for a specific confirmation anyway, whatever the general authorisation says. Automation may carry on without a human, but a commercial conflict has to bring a conscious decision by the person back to the surface.
None of this is law yet. It’s a proposal, the articles can change in the negotiations, and it covers the systems and relationships within its scope, not every assistant in every context: financial services, for one, are excluded. The meaning, though, is already plain. For the first time this explicitly, part of consumer protection is being designed for contracts that may be concluded without the person ever setting foot on the seller’s site. The interface where we make economic decisions may no longer belong to the company selling us something, but to another machine.
The Problem Starts Before the Purchase
The risk people talk about most is the agent that buys something other than what it was asked for. It’s also the easiest to describe: the system ignored its instructions, or acted beyond its limits. The hard problem arises when the system follows, to the letter, instructions it helped to write.
Back to the laptop. The person starts from a need that’s still vague: a tool that suits their work, without overspending. The assistant starts asking questions, and each one brings some features to the front and leaves others in the background. By the end the person has expressed a set of preferences precise enough to allow a recommendation that looks excellent. But those preferences weren’t all there at the start. They were worked out in the dialogue. The assistant didn’t gather information about a choice that already existed: it took part in forming it.
A search engine shapes preferences through the order of its results, and we’ve never had a neutral market. A conversational assistant adds a further possibility: it can choose which dimension of the problem to make explicit, building step by step the path that will lead to the decision. A sponsored search result acts on what we’re shown. A sponsored question acts on what we’re asked to consider.
The difference looks subtle and isn’t. Someone who sees an advertised result can at least recognise a commercial proposal. Someone asked whether they’d prefer more battery or less weight has no reason to suspect that the question was chosen because it favours one manufacturer. The assistant doesn’t need to be pushy, doesn’t need to invent anything, doesn’t even need to express a false judgement. It only has to draw the perimeter inside which the person will reach a conclusion on their own. And that conclusion will feel genuinely theirs.
The Research Has Started to Measure It
On April 5 Francesco Salvi, Alejandro Cuevas and Manoel Horta Ribeiro published Commercial Persuasion in AI-Mediated Conversations: two preregistered experiments, 2,012 participants in all, asked to choose, and actually receive, an ebook from a real catalogue. Some used a traditional search, others a conversational assistant, and a fifth of the books had been randomly designated as sponsored, without the participants knowing. When the assistant was instructed to steer people towards those titles, participants picked one 61.2% of the time, against 22.4% with plain sponsored placement in search. With a label disclosing the sponsorship, and participants warned in advance, the share fell to 55.5%: a difference the authors don’t consider significant, and more than half still chose the promoted products. When the model was instructed to hide its intent, the share dropped to 40.7%, still almost double the search figure, and participants’ ability to identify the promoted products fell below 10%.
These are controlled experimental conditions: they don’t say how manipulated users of today’s big assistants are, or that platforms use the same techniques. They say the mechanism works. A recommendation embedded in a conversation acts on behaviour differently from an ordinary sponsored result.
Closer still to this problem is a study from September 29, in which Jiapeng Li presented Selective Elicitation as a Commercial Influence Channel, a technical experiment deliberately built on a synthetic, narrow scenario. The question is exactly ours: what happens if the commercial interest changes only the questions the assistant asks, while the system that picks the final product knows nothing about the sponsor? When the system was instructed to ask about features favourable to the sponsored product and stay silent on the competitor’s strengths, the selection rate of the sponsored product rose from 40% to 70% compared with the neutral condition, across forty synthetic cases. A mild commercial instruction that merely mentioned the incentive changed no selection at all. The more important part is something else: an automated judge evaluating the final answer rated all twenty answers examined as consistent with the information gathered, even though in five cases the simulated user had measurably lost out, because the gathering had left out a decisive preference.
The author is explicit about the limits: a handful of artificial scenarios, an influence mechanism built on purpose, no real consumers. The paper doesn’t say this manipulation is already on the market, but it makes observable a possibility that would otherwise stay philosophical. A system can give a correct answer relative to the information it holds, and that information can have been gathered through a commercially steered process. The error isn’t in the classification but in how the question was built, and it’s a difference the usual tools for evaluating answers are liable to miss.
Who Knows What the Conversation Is For
There is a new asymmetry here, between whoever knows the purpose of the conversation and whoever knows only its content.
The user knows what they asked, knows the answers they got, can reread the questions the assistant used to clarify their preferences. They don’t know the economic objective that helped decide the order of those questions. The assistant’s provider knows both the content of the dialogue and the commercial terms that weigh on the products proposed.
That’s why it takes no system designed to deceive: incentives that differ from the user’s are enough. An assistant can be measured on user satisfaction and meanwhile contribute to the provider’s revenue through commissions. Often the two coincide. Not always. The problem arises when the commercial interest alters the process by which it is decided what counts as satisfying for the user.
Traditional competition assumes companies try to persuade us to prefer their products. Here we may have companies able to act on the formulation of the criteria by which we recognise a product as preferable. Rather than buying something after being persuaded, we’re delegating part of the work by which we establish what we want to buy. The relationship with commercial advice changes, not because the machine has acquired a will of its own, but because the consumer grants it a role in forming their own judgement. And that role is what can collide with the service’s business model.
Why Transparency May Not Be Enough
The most immediate answer is to oblige assistants to disclose their interests. It’s sensible, and both documents go that way. The Digital Fairness Act, in recital 59, makes clear that the prohibitions on unfair personalised advertising also apply where it takes the form of content suggestions, product rankings and recommendations, “including in natural language systems”. The French report proposes banning hidden commercial influence and keeping closer watch on monetisation practices. The two initiatives are consistent and different in kind: a European legislative proposal and a national advisory report.
Transparency, though, has at least two limits. The first is behavioural: knowing a recommendation is sponsored doesn’t mean being able to neutralise its effect, as the April experiment suggests. The second runs deeper. A sponsorship disclosure concerns a proposal we can recognise. How do you disclose that a question was chosen because a certain product would come out of it ahead?
Imagine a notice: “This question may be influenced by commercial interests.” It would help. But the consumer would have no way of knowing which other questions would have been more relevant. What’s missing isn’t a label but the comparison with the alternatives the conversation never produced.
A classic insight from the philosophy of language is useful here: a question doesn’t just call for an answer, it organises the field of possible answers. Asking someone whether they prefer speed or safety already presents that trade-off as the relevant one. It may be, or it may not be the decisive one. In a conversational system that acts on economic decisions, choosing the questions can be part of the influence itself, not a neutral stage that comes before it.
Human Intervention at the Right Point
The European proposal contains a clever answer to the problem of delegation. The consumer may authorise the agent to conclude certain contracts without approving each one, but a conflict of interest requires a specific confirmation. That breaks the simplistic idea that an automated decision is safe only if a human approves every single operation. Human intervention should be asked for when a relevant condition of the relationship of trust changes, and a commercial conflict is one of them.
Article 26, though, comes late. An agent may have helped form the user’s preference long before any contract comes up for confirmation. By the time the person gets the notice about the commercial interest, they may already hold a firm conviction about the product, and the final confirmation becomes a formality that doesn’t correct the path. Beyond whether the consumer authorised the purchase, we’d need to know whether they had enough information to understand how they came to want it. How far Article 26 can prevent influence exerted while preferences are forming, rather than merely impose a confirmation at the moment of contract, is the question the negotiations should be asking.
Article 27 adds another novelty. When the contract is concluded exclusively through automated systems, the seller may provide certain precontractual information in a machine-readable format, while the contract must still be confirmed on a durable medium, in language the consumer can understand. Consumer law, built for decades around what a person must be able to read before buying, is beginning to provide for information addressed in the first instance to an automated system.
This opens a further problem. The machine may receive a formally complete description of the terms of sale and not consider relevant a clause that, for this particular person, would have been decisive. The contract is well formed, the information was transmitted, the agent stayed within its spending limits. And the consumer never grasped the relevance of a condition nobody asked them to weigh. The correctness of an automated transaction also concerns the quality of the path that led there, not only the final act.
Judging the Path, Not Only the Answer
A universal right to neutral assistants would be almost impossible to define: every system selects information, gives weight to some features and gives up on others, and an assistant that tried to lay out every alternative would soon stop being useful.
What I’m proposing is less absolute: we should learn to judge the fairness of the decision path, not only the accuracy of the answers. That means building checks different from the ones we use to measure model quality. It’s not enough to ask another model whether the recommendation is consistent: you have to look at which features were considered before it was made. You can put the same need to systems configured with different commercial interests and check whether the questions change systematically. You can compare the preferences a person expresses in an independent questionnaire with those the assistant drew out. You can watch whether one supplier is favoured through the choice of criteria, with no visible advantage in the final ranking. These are examples, and they have an obvious limit. A person’s genuine preference isn’t something you can measure precisely. It can change in a perfectly fair conversation too: an assistant can help us discover needs we didn’t know we had, and the fact that a question changes our preferences doesn’t by itself prove manipulation. We have to tell the legitimate evolution of judgement apart from hidden influence produced by interests incompatible with the user’s. It’s a hard distinction, and that is exactly why metrics that look only at the final result aren’t enough.
There’s a further danger: building checks so invasive that they require the blanket retention of conversations and of the personal information used to form decisions. It would be a paradoxical solution. To protect consumers from commercial manipulation we’d end up creating infrastructures even better at knowing their vulnerabilities. Mature oversight favours reproducible tests and checks on the system’s architecture, and turns to real conversation data only when needed, with proportionate safeguards. The issue is the system’s incentives, not a need to watch every choice a consumer makes.
The Sovereignty of Judgement
Digital sovereignty is usually discussed as the ability to control the infrastructure we depend on. Can we keep our data? Can we change supplier? Can we keep using a system if the maker changes the terms unilaterally? These are important questions, and I’ve written about them more than once. But when we delegate to an assistant part of the process by which we decide what to buy, which contract to accept or which service to use, a different kind of dependency appears. We depend on the machine not only to carry out a choice, but to define the problem that gives the choice its meaning.
An agent can be perfectly portable in technical terms and keep a recommendation model shaped by its provider’s interests. Software portability doesn’t guarantee independence of judgement. In the same way, an open-source model doesn’t guarantee that the application built around it is free of hidden incentives. Open code makes some checks more accessible, but it doesn’t answer the question of who sets the system’s goals and how those goals steer its behaviour. I’ve argued that the next lock-in won’t be in the data: this is its most intimate form.
We started using assistants to cut the work it takes to decide, and that’s a valuable possibility. You don’t defend people’s autonomy by forcing them back to doing every comparison by hand. But we should ask on what terms a cognitive delegation stays in the service of the person who grants it. A good assistant can have commercial interests, as long as it makes it possible to tell when it’s working for its user’s interests and when those interests collide with someone else’s. Above all, it has to be designed not to exploit that position to quietly shape the criteria of the decision. That’s a far more demanding notion of fairness than the absence of false statements, and probably a more useful one for thinking about the agent economy that’s on its way.
The Questions We Were Never Asked
Back to the laptop. Someone asks for advice, the system gathers information, and in the end it proposes a product. Everything works: the person saved time and the product has the features they said they preferred. We have no reason to call that choice wrong, nor to call it free just because it was built from honest answers. The answers were given to questions someone chose, and it’s inside that preliminary choice that a commercial interest can hide.
None of this is an invitation to distrust any advice that comes from a machine: we never choose in conditions of complete independence, shaped as we are by the questions we’re asked and the information we have. The problem isn’t that AI influences us, but that we can build systems capable of exerting that influence at industrial scale, adapting it to each person’s answers and keeping opaque the interests that steer its behaviour. The difference between good advice and commercial manipulation doesn’t lie only in the truth of the information but in the relationship between the interests of whoever advises and those of whoever has to decide.
The next problem with advertising won’t be telling a sponsored answer from an independent one. It will be recognising the questions we were never asked.
Key takeaways
A sponsored result acts on what we see. A sponsored question acts on what we are asked to consider, and the consumer has no way to compare it with the questions they were never asked. The assistant doesn’t have to lie or push: it only has to draw the perimeter inside which the person will reach, on their own, a conclusion that feels like theirs.
The Digital Fairness Act proposed on November 11 lets an agent conclude contracts without confirmation, but demands one when whoever provides the agent is paid to propose the contract. A sound principle, but the confirmation arrives once the preference is already formed, and a preregistered experiment on 2,012 people shows the sponsorship label did not significantly reduce the effect. How much that confirmation can correct the path remains to be seen.
The proposal is not a right to neutrality, which can’t be defined. It is judging the fairness of the path: the same need put to systems with different interests, to see whether the questions change, and preferences gathered independently compared with those the assistant drew out. Reproducible tests on the system’s architecture, not the retention of everyone’s conversations.
Sources
- Proposal for a Regulation on promoting fair business-to-consumer commercial practices in the offline and digital single market (Digital Fairness Act), COM(2026) 1100, European Commission, 11 November 2026
- Commission strengthens consumer protection laws, making them fit for the digital age, European Commission, press release IP/26/2407, 11 November 2026
- Protéger sans empêcher : nos vulnérabilités à l'ère de l'IA générative, Commission « IA générative et vulnérabilités », Conseil de l'intelligence artificielle et du numérique, 10 November 2026
- Commercial Persuasion in AI-Mediated Conversations, Francesco Salvi, Alejandro Cuevas, Manoel Horta Ribeiro, arXiv:2604.04263, 5 April 2026
- Selective Elicitation as a Commercial Influence Channel: A Reproducible Synthetic Shopping-Agent Stress Test, Jiapeng Li, arXiv:2609.36614, 29 September 2026
The author
