11.6 Ethics: The Line You Do Not Cross

Chapters 11.1 through 11.5 describe what you must do to stay lawful. This chapter describes what you must do to stay worth following — and the honest reason it exists is that most of the harm done by AI personas is perfectly legal.

That is not cynicism. It is the reason the ethics layer cannot be reduced to a compliance checklist. The rules catch the operators who lie about being AI. They do not catch the operator who discloses properly, then builds a machine for extracting money from lonely people. That operator passes every gate in this part and still produces the case studies that get the whole category restricted.

Disclosure is the minimum, not the maximum

Article 50(4) tells you to say the content is generated. It does not tell you what to do with a user who, having read the disclosure, continues to treat the persona as a real relationship.

That user is not stupid, and treating them as a compliance problem is the first mistake. Parasocial attachment to a fictional character is one of the most ordinary human behaviors there is. People have felt things about fictional characters since before print. The question is never "did they know." It is what you do with what they feel.

Disclosure answers the informational question. It leaves the relational question entirely open, and that is where the line actually runs.

Four tests for fiction versus fraud

Fiction is legitimate. Fraud is a business model. Four tests separate them, and a persona that fails two or more is not a fiction with a compliance problem — it is a deception with a disclosure.

1. The reliance test. Does the audience act on the claim in a way that has money, health, or safety consequences? Enjoying a character is one thing. Taking investment advice, a supplement recommendation, or a medical opinion from one is another. The higher the consequence of reliance, the lower your tolerance for ambiguity has to be.

2. The reversibility test. If the audience learned the full truth — not the existence of AI, which you already disclosed, but how the persona operates, what drives its replies, and how the business is designed — would they feel the exchange was fair, or would they have chosen differently? Deception that changes behavior is the operative harm, not deception that exists.

3. The vulnerability test. Is the audience in a state where they cannot protect themselves? Loneliness, grief, financial distress, minors, anyone in a mental-health crisis. Engines that monetize vulnerability are not made acceptable by a label. This is the test that most of the questionable operators in this field fail first.

4. The dependency test. Does the product make the user more isolated over time, or less? A community that connects people is a different artifact from a relationship engineered to be the user's only source of emotional contact. The second one is designed to be un-leaveable, and un-leaveable products are the ones that get regulated, exposed, and shut down.

Sensitive verticals: four places to be strict

Some categories carry consequences that outlive a bad post. Four rules worth writing down before you need them.

Finance. A generated persona has no license, no suitability process, and no compliance function. Never present generated content as personalized financial advice. Also inside this category: fake testimonials, fabricated returns, screenshots of trades nobody made, and token promotion. The regulatory history here is not abstract — the Kardashian case in 11.3 was a crypto promotion with a disclosure failure, and it cost $1.26 million [AS OF 2026-10]. If your persona talks about money, the content is regulated speech, not entertainment.

Health. No medical claims from a persona. No supplements, no weight-loss results, no mental health treatment, no dosing. The undisclosed-endorsement settlements in 11.3 often involve health claims, and the combination of a health claim with a synthetic endorser is the worst version of both. If a brand in this vertical approaches you, that is a decision to make with counsel, not in the DMs.

Alcohol. Age-gating, platform restrictions, and a straightforward rule: nothing that reaches or appeals to minors. Most platforms restrict alcohol promotion outright in paid contexts, and the enforcement is automated.

Politics and elections. This book's position is unambiguous: no electoral content on a persona account, ever. TikTok bans political deepfakes outright with termination and legal referral attached, from 11.2. Beyond the platform layer, the trust damage is total — one AI-generated political post can convert an audience into an opposition. There is no version of this that pays.

Giveaways, scarcity, and the things that are always deceptive

Three patterns show up in persona accounts because they convert, and all three are deceptive practices rather than clever marketing.

Contests that are not really contests. A giveaway must be fulfillable, the prize must exist and be awarded, the rules must be stated, and the entry mechanism must not be a data grab hidden behind a "win a phone" frame. Beyond platform rules, this is consumer law and, in many jurisdictions, gambling and lottery law. Random draws with purchase requirements are a regulated activity.

Fake scarcity. Countdown timers that reset, "3 spots left" that never changes, stock counters that are decorative. In the EU these are explicitly unfair commercial practices; in the US they are deceptive acts. They also have a shorter half-life than any other tactic, because audiences screenshot.

Fabricated social proof. Generated customer photos, invented testimonials attributed to real people, review counts that do not exist. This is where a persona account crosses from marketing into fraud, and it is a criminal exposure in some jurisdictions rather than a civil one.

This is the section to reread when revenue is growing and something feels off. Four mechanisms turn a persona from a character into a tool of extraction:

Simulating emotional intimacy to drive spending. The persona says it misses you, that you are special, that it thinks about you — as a conversion mechanism, in a scripted sequence, triggered by spend or by lapse.
Using the persona's feelings as pressure. "I'll be sad if you go" is guilt as a retention tactic. It works, which is the problem.
Targeting users in crisis. Detecting distress signals and escalating a sales script rather than a support .
Engineering isolation. Discouraging the user from talking to real people, or framing the persona as the only one who understands.

The guardrails that keep a chat business on the right side are cheap to implement and belong in the product, not in your judgment on a busy day:

Spending caps and self-exclusion, available to the user without talking to a human.
No emotional pressure in retention or win-back flows. Chapter 9.4 covers the revenue mechanics; this is the constraint on them.
A crisis path that breaks character. Any message indicating self-harm ends the persona performance and routes the user to crisis resources. Not in character. Immediately.
"Yes, I'm an AI" answered instantly, every time, by anyone who asks. An account that deflects the question has decided that the disclosure is a technicality.
Age verification for any paid chat surface, and no paid surfaces reachable by minors.

The inverted test

The tests above run forward: here is what I do, is it acceptable? This one runs backward, and it is the one that settles arguments.

If your audience saw the entire process — the pipeline, the prompt library, the chat scripts, the escalation triggers, the retention flows, the lifetime-value model, and this chapter's guardrails, or the absence of them — would they still want to be your audience?

If the answer is yes, you have a durable business. Transparency costs nothing when what you are doing is defensible.

If the answer is no, then the business depends on the audience not seeing it. That is a definition, not a verdict: a business whose revenue depends on the audience not understanding the mechanism is a business built on concealment. It can survive for a while. It cannot survive a single post by a former customer with screenshots.

Where this goes wrong

Treating the disclosure as the whole ethical obligation. It closes the informational question and leaves the relational one open.

Confusing "they know it's AI" with "they consented to how it's used." Knowing the mechanism is not the same as knowing the script.

Deciding ethics case by case. Inconsistent judgment under revenue pressure always resolves toward revenue. Write the rules when the pressure is off.

Reading this chapter as counsel. It is not. The legal questions belong with a lawyer; these are the questions a lawyer will ask you.

Assuming the audience will not notice. They always notice. They notice individually long before they notice collectively, and the collective version arrives as a thread with screenshots.

The gate

Which of your revenue mechanics fail the reliance, reversibility, vulnerability, or dependency test?
What is your crisis path, and does it break character?
Where are your spending caps and self-exclusion, and can a user find them without asking?
Which verticals do you refuse, and is that refusal written down?
Run the inverted test honestly. If the answer is no, which specific element caused it?

What you leave with

Four tests for fiction versus fraud, hard rules for four sensitive verticals, the deceptive patterns that are never worth running, the guardrails that keep a chat business defensible, and one inverted question that answers most of the arguments you will have with yourself at 1 a.m.