AI Governance in Schools: Privacy, Security, and Student Data

Artificial intelligence is already changing how K–12 schools teach, plan, communicate, and operate. But adopting new technology responsibly requires more than choosing the right AI tools. Schools also need clear AI governance that protects privacy and security, establishes appropriate uses, and keeps educators involved in decisions that affect students.

For school and district leaders, the goal isn’t to prevent innovation. It’s to create the safeguards that make responsible innovation possible.

What Is AI Governance?

AI governance is the system of policies, roles, processes, and safeguards an organization uses to guide how AI is selected, implemented, monitored, and evaluated. In schools, that means deciding who can use AI, what information can be shared with it, which products are approved, and who is responsible for oversight.

An effective AI governance framework for schools connects technology decisions to educational goals. Instead of treating every new AI product as a separate technology purchase, districts can establish a consistent process for evaluating AI systems based on factors such as:

• Student privacy and data protection policies
• Cybersecurity
• Instructional value
• Accessibility and equity
• Accuracy
• Potential bias
• Human oversight
• Vendor practices
• Legal and regulatory requirements

Governance policies should involve instructional, technology, privacy, security, and leadership perspectives rather than relying on one department. Schools should use their governance frameworks to establish clear ownership, create an AI policy, inventory existing tools, evaluate new products for educational value and risk, train educators, and periodically review approved uses.  

What Does AI Governance in Schools Look Like?

One good but frustrating thing about AI is that it is continually innovating and changing. As a result, most K–12 leaders want to create a repeatable governance framework for AI adoption rather than having to start over every time a new tool appears. To do that, you can build a simple cycle like this:

Govern → Evaluate → Approve → Educate → Monitor → Improve

1. Govern: Establish ownership, principles, and an AI policy.
2. Evaluate: Review the educational value, privacy, security, accessibility, and risks of proposed AI tools.
3. Approve: Decide where and how the tool may be used.
4. Educate: Build AI literacy among educators, staff, students, and relevant stakeholders.
5. Monitor: Review AI use, incidents, vendor changes, and emerging risks.
6. Improve: Update policies, training, and safeguards as technology changes.

School and district leaders who want to develop the skills to guide decisions on AI can explore WGU’s AI for K–12 Education Leaders Certificate. The certificate focuses on responsible AI strategy, organizational guidelines, risk identification, and AI implementation in K–12 settings.

Why AI Governance Matters in K–12 Schools

AI governance in K–12 schools matters because educational technology can affect students’ privacy, learning, and digital safety. Clear governance policies allows districts to explore useful applications of AI while establishing boundaries around higher-risk uses.

Artificial intelligence can support educators with tasks ranging from lesson planning and creating instructional materials to communication and administrative work. But convenience shouldn’t eliminate human review.

Without a consistent process, teachers and staff may independently adopt products with different privacy practices, security controls, or terms of service. As a result, the school district may struggle to know which applications are being used and what information is entering them. Governance helps replace that uncertainty with shared expectations. 

Responsible AI integration is centered on teaching and learning, student well-being, safety, privacy, and other district and school board priorities. Educational leaders should design systems that help school systems move from simply exploring generative AI toward more deliberate implementation. Effective AI adoption therefore requires both innovation and accountability.

Privacy Risks and FERPA Considerations

Does FERPA apply to AI? Yes, FERPA applies when the use of an AI product involves education records protected by the law. Using artificial intelligence does not create an exception to existing student privacy obligations.

The U.S. Department of Education’s Student Privacy Policy Office administers and enforces FERPA and provides technical assistance to schools and districts on safeguarding student information. That makes FERPA AI considerations an important part of product review.

For example, imagine a teacher wants to use a generative AI application to help summarize student writing. Before uploading identifiable student work, the district needs to understand whether the information is protected, whether disclosure is permitted under FERPA, and how the vendor will collect, use, retain, and protect that information.

A student data privacy AI review should ask questions such as:

• Does the product require personally identifiable student information?
• What information does the vendor collect?
• Why does the vendor need it?
• How is information stored and secured?
• Is submitted information used to train or improve AI models?
• Is information shared with other parties?
• How long is the information retained?
• Can the district request deletion?
• What happens to the information when the contract ends?
• Do applicable FERPA requirements permit the disclosure?
• What additional federal, state, or local privacy requirements apply?

FERPA isn’t necessarily the only consideration. Depending on the tool, students’ ages, the type of information involved, and how the product is used, other laws and state requirements may apply. District leaders should work with their privacy, legal, procurement, and technology professionals rather than assuming that a vendor’s claim that a product is “education safe” resolves the issue.

UNESCO’s guidance for generative AI in education and research similarly identifies data privacy protection and age-appropriate use as important considerations for educational institutions.

The simplest rule for educators is also one of the most useful: Don’t enter identifiable or confidential student data into an AI product unless your school or district has approved that use.

Cybersecurity and AI Tools

AI security in education should be part of a district’s existing cybersecurity program, not a separate policy. Every new application can introduce accounts, integrations, stored information, vendor access, or other risks that deserve review.

K–12 organizations may face cybersecurity threats, including data breaches, ransomware, business email scams, and other attacks. K–12 cybersecurity guidance recommends prioritizing protections and developing approaches suited to schools’ limited resources and complex technology environments.

When reviewing AI tools, technology and security teams can consider:

• Authentication and account security
• Data encryption
• Access controls
• Vendor incident-response practices
• Data retention and deletion
• Third-party integrations
• Security certifications or independent assessments

You may also want to ask whether unnecessary information is being collected, how administrators can disable accounts or revoke access, and how the vendor communicates security incidents.

Automation also deserves attention. An application that only drafts text presents a different risk profile from one that can automatically send messages, change records, make recommendations, or trigger actions in connected systems. 

The greater the potential impact, the stronger the oversight should be.

Creating an AI Governance Committee

Who should oversee AI in a school district? AI governance works best when seen as a shared responsibility rather than something assigned entirely to the IT department or one administrator. A district AI governance committee might include representatives from:

• District and school leadership
• Information technology and cybersecurity
• Curriculum and instruction
• Classroom educators
• Student services or special education
• Data privacy
• Legal or compliance teams
• Procurement
• Communications
• Students or families, when appropriate

The exact structure will depend on district size and resources. What’s important is stakeholder engagement: People who understand instruction, technology, student needs, operations, and risk should have a voice.

A useful governance committee meeting agenda can be simple:

Sample AI Governance Meeting Agenda

1. Review newly requested AI products.
2. Discuss instructional or operational use cases.
3. Identify privacy, security, accessibility, and equity concerns.
4. Review incidents or questions involving existing products.
5. Approve, restrict, pilot, or reject proposed uses.
6. Identify needed staff training or communication.
7. Assign owners and deadlines.
8. Schedule policy or vendor reviews.

By focusing meetings on decisions and ownership, you can prevent a governance committee from becoming a discussion group without authority to act.

Building an AI Acceptable Use Policy

What should an AI policy include? An AI policy for schools should explain approved and prohibited uses, privacy expectations, human oversight, academic integrity, accountability, and the process for approving new products.

The goal is to make expectations understandable enough that teachers and students can actually follow them.

Sample AI School Policy Outline

A practical AI school policy can address:

• Purpose and scope: Why the district permits the use of AI and who the policy covers.
• Approved tools: How educators determine which applications can be used.
• Student information: What information may never be entered into unapproved systems.
• Instructional expectations: When AI use is appropriate in the classroom.
• Academic integrity: When students must disclose or cite AI assistance.
• Human review: Which decisions require educator or administrator judgment.
• Accuracy and bias: Expectations for verifying AI-generated information.
• Security: Account, access, and incident-reporting requirements.
• Accessibility and equity: Expectations for inclusive implementation.
• Review process: How often guidelines will be revisited as technology and requirements change.

A responsible-use policy should also distinguish between different contexts rather than simply declaring AI “allowed” or “banned.”

Teacher Training and AI Literacy

An AI policy won’t accomplish much if educators don’t understand how to apply it. AI education and professional development help turn written rules into everyday practice.

Teachers don’t need to become machine learning engineers. They do need enough AI literacy to understand what artificial intelligence can and cannot reliably do.

Training can help educators:

• Understand the basic capabilities and limitations of generative AI.
• Recognize inaccurate or fabricated outputs.
• Identify potential bias.
• Protect private and confidential information.
• Follow district approval processes.
• Design appropriate student AI use.
• Evaluate whether AI adds instructional value.
• Keep professional judgment central when using AI.

AI literacy should also evolve. A single professional development session at the beginning of the year won’t prepare educators for every new product or feature.

Districts can instead integrate AI education into existing professional learning, digital citizenship initiatives, curriculum conversations, and technology training. If teachers are aware of how AI works, they can better guide students on how to use it as well. 

For educators who want structured professional development, WGU offers an AI for K–12 Teachers Certificate designed to develop practical skills for AI use, including AI-supported lesson planning, responsible student AI use, evaluating outputs for accuracy and bias, protecting privacy, and keeping human judgment central.

Common Mistakes Schools Should Avoid

The biggest AI governance mistakes often come from moving too quickly—or refusing to move at all.

Mistake 1: Banning everything indefinitely.

A blanket prohibition may seem simple, but it doesn’t build AI literacy or give educators a framework for evaluating future technology.

Mistake 2: Allowing every tool until there’s a problem.

Unmanaged AI adoption can make it difficult to know which products have access to school information or whether appropriate safeguards are in place.

Mistake 3: Treating AI as only an IT issue.

AI affects curriculum, assessment, privacy, accessibility, operations, and professional practice. Governance policies should reflect that.

Mistake 4: Writing policy without training.

Educators need examples and scenarios showing how rules apply to their work.

Mistake 5: Ignoring vendor changes.

Products, features, terms, and data practices can change. Approval now shouldn’t necessarily mean approval forever.

Mistake 6: Focusing only on student AI use.

Staff use of AI matters too. District policy should consider how teachers, administrators, and other employees use these systems.

Mistake 7: Assuming AI output is objective.

AI-generated information can be inaccurate or biased. Human review remains essential, particularly when decisions could affect students.

AI Governance Checklist

Use this checklist as a starting point when developing responsible AI in education practices for your school or district.

• Identify an individual or committee responsible for AI governance.
• Inventory AI products already being used across the district.
• Define approved and prohibited AI use cases.
• Establish a formal process for requesting new AI tools.
• Review privacy and FERPA considerations before sharing protected information.
• Evaluate vendors' model-training practices and data collection, retention, and deletion standards.
• Include cybersecurity review in product approval.
• Establish rules for handling student data.
• Require appropriate human oversight for consequential decisions.
• Define expectations for student academic integrity and disclosure.
• Train teachers and staff in AI literacy, and plan regular training going forward.
• Give educators practical examples of appropriate and inappropriate use of AI.
• Include stakeholder engagement in major governance decisions.
• Document approved tools and communicate changes clearly.
• Establish a process for reporting AI-related privacy, security, or accuracy concerns.
• Periodically review vendors, policies, and approved use cases.
• Update AI data governance practices as technology and requirements evolve.

A Simple Decision Tree for Approving AI Tools

When someone requests a new tool, a district can navigate the decision using five questions:

1. Does it solve a real instructional or operational need?

If no, don’t add technology simply because it uses AI.

2. Does it require student, employee, or confidential district information?

If yes, complete the appropriate privacy and legal review.

3. Does it meet district security requirements?

If no, stop and make sure to meet these requirements before proceeding.

4. Can people meaningfully review important outputs or decisions?

If no, consider whether the use presents unacceptable risk.

5. Does the educational value outweigh the remaining risks?

If yes, approve or pilot it with documented conditions and a review date.

This approach keeps districts from focusing only on whether a product is impressive. The more useful question is whether you can integrate AI in a way that supports learning while maintaining appropriate safeguards.

Build the Skills to Lead Responsible AI Adoption

AI governance isn’t about choosing between innovation and student protection. Strong governance helps schools pursue both.

When districts establish clear expectations, evaluate technology before deployment, protect student information, strengthen AI literacy, and keep people accountable for important decisions, educators can make more thoughtful choices about where artificial intelligence belongs in schools.