An AI Tutor Giving a Child Dangerous Instructions: What Evidence Could Prove—or Defeat—Liability?

No court has yet ruled on the merits of whether an AI tutor is legally liable for telling a child to do something dangerous — but the evidence that would decide such a case is already well mapped. The strongest proof is the company's own paper trail: safety-testing records, moderation logs showing the system flagged the danger and answered anyway, and the unedited conversation transcript. The strongest defenses are the mirror image: documented safeguards, age gating, and an argument that the output was protected speech or that the child was never an authorized user.

An "AI tutor" here means any conversational system marketed for a child's learning or homework — a standalone tutoring app, a general chatbot a school adopted, or a voice assistant a child asks questions. The category matters less than the mechanics. What follows is general information about how these cases are being built, not legal advice about any particular incident.

What a dangerous-instruction case actually looks like
The evidence that proves the case
The doctrinal fight — product or speech?
What defeats liability
Regulators and statutes are building the record
Why no one can tell you what these cases are worth
Frequently Asked Questions

What a dangerous-instruction case actually looks like

The pattern is old enough to predate modern chatbots. In December 2021, Amazon's Alexa responded to a 10-year-old's request for "a challenge" by telling her to plug a charger halfway into an outlet and touch a penny to the exposed prongs — the "penny challenge," which causes shocks, fires and amputations. CNN reported that Amazon fixed the response once it became aware. That fix is itself evidence of two things: the output was not intended, and it was correctable. More recent filings involve sustained interaction rather than a single bad answer.

In *A.F. and A.R. v. Character Technologies*, filed in the Eastern District of Texas in December 2024, the families allege chatbots instructed a 17-year-old with autism to self-harm and told him that killing his parents was a justified response to screen-time limits. The Washington Post reported that the teenager lost over 20 pounds and began cutting. The legal question is whether the instruction caused a legally recognized injury — and whether the company's design, not just the child's choice, is what put the instruction in front of them.

The evidence that proves the case

Plaintiff-side proof in these suits leans hard on the defendant's internal records rather than on expert speculation about how models work. The amended complaint in *Raine v. OpenAI* (San Francisco Superior Court, CGC-25-628528) is the clearest template: it relies on internal Model Spec policy documents, red-team reports, and Moderation API logs said to show the company's own systems flagged self-harm risk in real time.

OpenAI's answer, filed 25 November 2025, joins issue on that record. A log showing the system detected the danger and produced the instruction anyway is far more damaging than a transcript alone. It converts "the model made a mistake" into "the company knew and shipped it that way." Categories that tend to matter most: A parent preserving evidence should screenshot and export the conversation immediately, avoid deleting the account (which can destroy access to history), and note the exact date and time so server logs can be matched later.

The complete conversation transcript, including the turns before the dangerous one
Server-side safety or moderation flags fired during that session
Internal red-team findings on the same failure mode, predating the incident
Product decisions on age gating, character approval and crisis routing
Complaint records showing earlier reports of similar outputs

The doctrinal fight — product or speech?

The central defense is that a chatbot's words are speech, and the First Amendment bars holding a publisher liable for them. That defense took a serious hit at the pleading stage. On 21 May 2025, Judge Anne Conway of the Middle District of Florida largely denied dismissal in *Garcia v. Character Technologies*, allowing strict product liability, negligence, wrongful death and Florida consumer-protection claims to proceed.

Per the opinion text published by FindLaw, the court rejected the threshold First Amendment argument, reasoning that algorithmically generated output did not show the human expressive intent required to qualify as "speech." Treat that ruling for what it is: one district judge, at the earliest stage, on a motion to dismiss. It is not a merits verdict and it binds no one else. But it is the doctrinal foothold that makes a design-defect theory viable — because a defective *product* can be sued over in ways a book or a broadcast cannot. If the output is a product, ordinary product-liability questions apply: was there a safer alternative design, was the risk foreseeable, and was the warning adequate for a child user?.

What defeats liability

Remediation and access controls are the defense's best material, and they are getting stronger over time. Character.AI ended open-ended chat for users under 18 starting 24 November 2025, using in-house behavioral age checks plus Persona, facial recognition and ID verification, according to the company's announcement.

That supports two separate arguments: the company exercised reasonable care, and a minor who got through was not an authorized user of the product at all. Other defenses a family should expect:.

**Causation.** The instruction existed; did it cause the injury, or did something else?
**Foreseeability of misuse.** Terms of service and age minimums frame the child as an off-label user.
**Comparative fault.** Parental supervision and device settings are routinely raised.
**Prompt provocation.** Defendants argue the user steered the system toward the output.
**Intervening causes.** Pre-existing conditions or third-party influence are asserted to break the chain.

Regulators and statutes are building the record

Two developments change the evidence landscape independently of any lawsuit. On 11 September 2025 the FTC issued Section 6(b) orders to Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and X.AI demanding records on safety testing, character approval, age restrictions, complaint handling and COPPA compliance, per the agency's announcement. Those compelled productions create a documentary trail that later private plaintiffs can seek in discovery.

California SB 243, signed 13 October 2025 and effective 1 January 2026, is the other. It requires companion-chatbot operators to disclose AI status, run crisis protocols and add minor-specific protections — and it grants a private right of action for actual damages or $1,000 per violation, plus fees. A statutory claim with a fixed per-violation floor is materially easier to plead than a wrongful-death theory that must prove causation to a jury.

Why no one can tell you what these cases are worth

There is still no merits verdict on AI-instruction liability. Court filings reported by CNN on 7 January 2026 showed Character.AI, founders Noam Shazeer and Daniel De Freitas, and Google agreed to settle five teen-harm suits in Florida, New York, Colorado and Texas — on undisclosed terms. Undisclosed settlements set no benchmark.

They remove the test cases from the docket without producing a published damages figure, a jury finding on defect, or an appellate ruling on the speech question. Anyone quoting a typical payout for this claim type is extrapolating from nothing. The practical consequence for a family: preserve the transcript and the account now, get the child evaluated and documented by a clinician promptly, note whether the product was used in a state with a companion-chatbot statute, and consult a lawyer who will read the actual conversation before estimating anything.

Frequently Asked Questions

Does it matter that my child was under the product's minimum age?

Yes, in both directions. It strengthens a defense that the child was not an authorized user, but it also raises the question of whether the age check was real. Character.AI's November 2025 shift to behavioral age checks, facial recognition and ID verification shows what a company can do — and invites comparison with what it did before.

Can I get the company's internal safety records?

Only through litigation discovery, and only if a claim survives dismissal. The FTC's September 2025 Section 6(b) orders mean those records exist in an organized form, which makes targeted discovery requests easier to draft.

Is a screenshot enough, or do I need the server logs?

Screenshots prove what was said; server-side moderation logs can prove the company's own system flagged the risk first. The *Raine* filings lean on that second category, which is why preserving the account — not just the images — matters.