Microsoft Bars Use of Student Data for AI Training Under New School Deal

Microsoft has agreed to legally binding restrictions preventing it from using student or teacher data to train its artificial intelligence models, as US teachers’ unions push technology companies for stronger privacy safeguards in schools.

The agreement follows New York City Mayor Zohran Mamdani’s decision to impose a one-year moratorium on AI use for public school students up to eighth grade.

Microsoft Accepts Binding AI Privacy Rules

Microsoft signed the National AI Safety & Privacy Standard with the American Federation of Teachers and the United Federation of Teachers. The agreement was announced in New York on September 9 by Microsoft Vice Chair and President Brad Smith, AFT President Randi Weingarten and UFT President Michael Mulgrew.

Under the agreement, Microsoft cannot use student or teacher data to train its AI models or track students. AI systems also cannot make high-risk decisions without human review.

School districts can incorporate the protections directly into Microsoft contracts. If the company violates the terms, districts can terminate the contract and seek damages.

The standard contains 10 enforceable protections. Microsoft cannot sell student data or use it for advertising, must limit the amount of information it collects and must explain in plain language how its tools operate. It is also required to report data breaches within 72 hours, address security flaws within a set deadline and undergo annual certification and audits.

AI companion chatbots for students are prohibited under the standard. Smith said the 30-page agreement could be reduced to three central principles: “privacy, safety, and transparency.” He said the terms would apply to every school district Microsoft works with from November 1.

Weingarten described the arrangement as an “iron-clad privacy agreement with real teeth”.

New York and Los Angeles Restrict AI in Schools

The agreement comes after Mamdani announced a one-year AI moratorium on September 2 with Schools Chancellor Kamar Samuels. The measure covers nearly 600,000 students up to eighth grade.

Companion chatbots and mainstream AI tools such as ChatGPT and Claude are blocked in every grade. High school students will have access to five AI pilots and two mandatory 45-minute AI literacy modules. Teachers may use AI for lesson planning but not for grading.

Los Angeles Unified has imposed a broader restriction for students. District officials disclosed a moratorium preventing students in all grades from using generative AI on district devices during the school year. Google’s AI Mode has also been disabled on student accounts.

The decision followed the board’s June vote to restrict screen time. Board member Nick Melvoin did not object to the measure but said the decision “didn’t really come from the board.”

Teachers’ Unions Seek Similar Deals With AI Companies

Teachers’ unions are now seeking similar AI privacy agreements with OpenAI and Anthropic. Weingarten said the two companies, which along with Microsoft fund the unions’ $23 million National Academy for AI Instruction, had “expressed willingness to do this kind of agreement.”

An Anthropic spokesperson said the company had been working with the AFT on “shaping a gold standard for safety and privacy.” OpenAI had yet to comment, according to the report.

The unions are pursuing contractual safeguards amid concerns that existing US laws have not kept pace with generative AI. The report said COPPA requires parental consent before companies collect information from children under 13, but predates generative AI by more than two decades and does not address model training.

At least 37 states have issued AI guidance for schools, but there is still no national consensus on how schools should teach AI literacy. Weingarten said existing protections such as HIPAA and FERPA had not anticipated the arrival of AI, arguing that technology companies must take responsibility for products marketed to students, educators and communities.