Engagement or manipulation? Gamification in AI

English · Nederlands

Game Dynamics

AI products are under pressure to show engagement. Daily active users, streaks and session length are easy to report to a board, and easy to inflate with the wrong mechanics. In 2010 I wrote about bread and games: gamification in marketing. Reading it again in 2026, the test I proposed then, reward the behaviour you want and keep it voluntary, is turning into a legal standard.

What I saw then

December 2010 was the peak of check-in badges. Social gaming was sold as the new way to influence behaviour, partly on the back of location apps built on game dynamics. My argument was that the tool does not create loyalty: “all apps in the world not one new loyal customer make.” Running a campaign on a game platform was not enough. The real challenge was using game mechanics to reward the behaviour you actually want to change.

I was optimistic. Marketers would get to be creative again, business owners would run the bank, and customers would be influenced in “more subtle, engaging and most of all: voluntary ways” than ever before.

What is the same now

AI products face the same temptation, with better tools. Notifications, streaks, autoplay, an assistant that always offers one more suggestion: they all lift usage metrics. Whether they reward the behaviour the customer wants is a separate question, and regulators have started asking it.

In September 2025 the US Federal Trade Commission sent orders to seven companies running AI chatbots that act as companions, including Alphabet, Meta, OpenAI, Snap and xAI. It asked how they monetise engagement and which features they use to increase how often and how long people use them, especially children and teens (FTC). That is my 2010 question, asked by a regulator that can demand answers.

Europe went further. In February 2026 the European Commission preliminarily found TikTok’s addictive design in breach of the Digital Services Act, naming infinite scroll, autoplay, push notifications and its highly personalised recommender system, and said TikTok needs to change the basic design of its service (European Commission). A final non-compliance decision can bring a fine of up to 6% of worldwide turnover (Matheson). Since 2 February 2025 the EU AI Act has prohibited AI systems that use manipulative or deceptive techniques to distort people’s behaviour in harmful ways (Lewis Silkin). And in the US, Amazon agreed in September 2025 to a $2.5 billion settlement with the FTC over Prime sign-ups without consent and a cancellation process that was hard to complete (FTC).

Next up is the Digital Fairness Act, which the Commission’s work programme plans for the fourth quarter of 2026, aimed at dark patterns, addictive design and unfair personalisation (European Parliament). The DSA rules for very large platforms will not be the last word.

What is different

In 2010 the game was visible. A badge, a leaderboard, a mayorship: you knew you were playing. In AI products the mechanics are invisible and personal. A model can learn which tone, which nudge and which follow-up question keeps each individual user going. The “bank” I wrote about is now an optimiser running in real time, which makes “voluntary” harder to judge and easier to break.

To be fair, most B2B software is not TikTok. People use it because their work requires it, and not all engagement is bad. A streak that helps someone practise a skill they chose to learn serves their goal. The problem starts when the mechanic serves your metric and not their goal, and B2B teams borrow consumer mechanics faster than they borrow consumer safeguards.

What to do about it

  1. Define the customer’s outcome before the engagement metric. Write down what the customer wants to achieve, then choose the usage metric that tracks it. When the two diverge, the outcome wins.
  2. Apply the voluntary test. For every streak, notification, default and autoplay, ask: would users opt in if they saw how it works, and is switching it off as easy as switching it on? Amazon’s cancellation flow is the expensive counterexample.
  3. Audit your design against the regulators’ list. Endless feeds, autoplay, push notifications, personalised nudges, hard cancellation. Document why each one exists and what it does for the user. That record is the start of the risk assessment you will be asked for.
  4. Price on outcomes, not on time spent. If your revenue rises with compulsive use, your incentives point the wrong way. I made the case for AI pricing in Price AI on outcomes, not tokens.
  5. Measure regret. Track opt-outs, disabled notifications and churn reasons next to engagement. A mechanic that lifts sessions and opt-outs at the same time is spending trust you will need at renewal.

Better games

In 2010 I ended with a call for better circenses. The good mechanics delivered: they helped people build habits they wanted. The bad ones now attract regulators. Build the first kind, because the second is turning into a liability.

Sources