Student Teacher Arrested After Snapchat's AI Reported Her Private Joke To The FBI

Your private messages may not be as private as you think. AI is increasingly monitoring digital conversations, but it doesn't always understand context. So what happens when artificial intelligence mistakes sarcasm, frustration, or dark humor for a real threat? One Illinois student teacher found herself at the center of that reality after a private Snapchat message triggered an FBI tip. In January 2025, the Illinois State University student was sitting in a Washington elementary school writing a lesson plan when a third-grade student pressed the power button on her computer, wiping out her work. Frustrated, she vented in what she believed was a private Snapchat group containing only her boyfriend and two roommates. According to police reports, she wrote asking whether she should "shoot the child or just the whole school?" No one in the group chat reported her. Instead, Snapchat's automated systems flagged the message and alerted the FBI, which then contacted the Tazewell County Sheriff's Office. Within hours, officers arrived at the school, questioned the student teacher, searched her belongings, confiscated her phone, and arrested her on a disorderly conduct charge. During questioning, she immediately acknowledged sending the message but repeatedly explained that it had been a joke intended only for her private group chat. She called it "stupid" and insisted it was never meant to be taken seriously. Teachers at the school also described her as "very laid-back," "quiet," and friendly with students. Officers searched her classroom and belongings but found no weapons or evidence suggesting she intended to carry out violence. She was still handcuffed and taken to jail. At one point, after learning she would be arrested, she simply responded, "That's crazy." The charge was eventually dismissed after she completed Tazewell County's deferred prosecution program, but the consequences extended beyond the courtroom. She was removed from her student-teaching placement and never returned to the elementary school, although she later graduated from Illinois State University. Snapchat declined to explain exactly why the specific message was flagged but pointed to its Community Guidelines, which prohibit content expressing an intent to cause physical harm. The company says it contacts law enforcement when it believes a threat is "credible and imminent." An Eighth Grader Was Arrested Over a Joke An Associated Press investigation highlighted similar incidents involving AI-powered monitoring software. One Tennessee eighth grader was arrested after making what her mother described as a "wrong" and "stupid" joke in a school-monitored online chat. The girl had been teased by classmates and responded by writing, "on Thursday we kill all the Mexico's." Surveillance software flagged the message, and before the school day was over she had been arrested, interrogated, strip-searched, and spent the night in jail, according to her family's lawsuit. “Is this the America we live in?” Her mother questioned whether the technology had ignored the context entirely. "It made me feel like, is this the America we live in?" she said. "And it was this stupid, stupid technology that is just going through picking up random words and not looking at context." Across the country, thousands of school districts now use AI-powered monitoring programs such as Gaggle and Lightspeed Alert to scan students' online activity for signs of violence, self-harm, bullying, or abuse. These systems can automatically notify school officials and, in some cases, law enforcement. Supporters argue the software has helped prevent suicides and identify credible threats before tragedy strikes. But critics say the technology often struggles to distinguish real danger from sarcasm, jokes, creative writing, or conversations taken out of context. Elizabeth Laird, director of equity in civic technology at the Center for Democracy and Technology, warned that the systems have "routinized law enforcement access and presence in students' lives, including in their home." Even Gaggle CEO Jeff Patterson criticized how one school handled the Tennessee case. "I wish that was treated as a teachable moment, not a law enforcement moment," he said. Public safety and school security are legitimate priorities, especially given the rise in school violence across the country. But as AI monitors more of our digital conversations, these cases raise an uncomfortable question: if a private joke can trigger an FBI tip or land a teenager in jail, how much privacy do we actually have left?