Learning Analytics Guiding Principles

This document includes the university's definition for learning analytics, guiding principles, in scope/out of scope learning activities and data, as well as roles and responsibilities for using learning analytics.

This is excerpted from a longer document that includes additional details about data governance and the purpose and background for the creation of the principles. Refer to The Appropriate Use of Data for Learning Analytics Guiding Principles on the vice provost’s page for Learning Analytics at UW-Madison. These guiding principles were created by the Learning Analytics Data Use Subcommittee, endorsed by the Learning Analytics Roadmap Committee on May 13, 2019, and were approved by the Data Governance Council on May 28, 2019.

In this document:

Definition of Learning Analytics
Guiding Principles / Values
Scope: In Scope & Out of Scope
Roles & Responsibilities

UW-Madison’s Definition of Learning Analytics (LA)

Learning analytics is the undertaking of activities that generate actionable data from the learning environment intended to improve student outcomes by informing structure, content, delivery, or support of the learning environment.

— Contextually defined for UW-Madison by the Learning Analytics Roadmap Committee

Guiding Principles / Values

Students are real and diverse individuals, and not just their data or information. These principles — beneficence, transparency, privacy and confidentiality, and minimization of adverse impacts — aim to uphold the dignity of students while ensuring learning analytics are used to improve educational outcomes.

Beneficence

Use LA to benefit student learning and success, while potentially improving teaching.
Ensure good evidence and pedagogical reasons are the backbone for data collection and interpretation.
Engage students as active agents in designing LA interactions that will support them.

Minimization of Adverse Impacts

Promote inclusivity and equity to mitigate bias.
Ensure that data are not exploited for unexpected or unwanted uses or outcomes.
Continually evaluate and monitor LA tools, practices, and interventions to assess differential impact and overall efficacy.
LA should be only one data point for consideration when taking action; it should not be the only input.

Transparency

Be transparent regarding the policies and procedures for collection, access, and use of student learning data for LA purposes.
Ensure LA are well defined and visible to stakeholders, including methods used to initiate interventions.

Privacy and Confidentiality

Follow existing campus privacy, data governance and security standards and practices, including those pertaining to institutional data access and stewardship.
Share data/information only with those who are authorized.
Use data in the most anonymous format possible that is still useful.

Scope: In Scope & Out of Scope

What is included in teaching and learning data use and learning analytics activities?

As an institution, we are required to collect and retain data as part of the student record. Data Governance Policy classifies LA data as “restricted” because the unauthorized disclosure of personal student information could cause potential harm to individuals affected or a significant level of risk to the university. The Learning Analytics Data Use Subcommittee establishes a clear delineation of what is in scope and what is out of scope for both LA data and LA activities, detailed in these tables.

Learning Analytics Activities
In Scope	Out of Scope
Course improvement Improvement of learning activities Personalized individual student learning support Course materials development and improvement Instructional improvement and self-evaluation Program review, evaluation, and accreditation Advising, including communication among advisors and with instructors, about interventions to improve student outcomes Analysis and reporting about targeted student cohorts	Research requiring oversight by the IRB (Institutional Review Board) Instructor evaluation by the institution, including tenure review
Data Used for Learning Analytics Purposes
In Scope	Out of Scope
Personal data provided by students at application and/or enrollment, including age, race/ethnicity, and gender Data from face-to-face, online, and blended courses, and other learning experiences, such as: student work learning and student engagement assessment (grades, rubrics, direct assessment of outcomes) attendance and participation formative and summative assessment course evaluations (for instructors personal use only) library use tutoring center use conference/workshop participation Data from University-supported systems, including SIS (Student Information System), Canvas, TopHat, G Suite, Piazza, and Atomic Assessments; publisher tools included as required course materials, and the Unizin Data Platform Supplemental data collected by instructors within course context Program level data (outcomes, graduation rates, retention metrics)	Student-disclosed mental health information University Health Services records and other HIPAA-protected data Data on student appeals, misconduct, or complaints Students’ financial aid data Disability status Religious, political, or union participation

Roles & Responsibilities

The LA guiding principles apply to everyone. Every stakeholder group has a responsibility to understand this document. Specific stakeholder roles and responsibilities are provided for students, instructors, advisors and leadership.

All stakeholders have the responsibility to:

Refer to data governance, cybersecurity, and other university and UW System policies around data access, use, and retention.
Be aware of opportunities to engage with LA
Generate and use LA data ethically
Make inquiries when LA activities are unclear or produce real or perceived negative impacts on students

Students have the responsibility to:

Be aware of how LA data are being used for their benefit
Respect the privacy of other students’ data
Consider insights or interventions presented to them via LA activities
Generate their LA data ethically

Instructors have the responsibility to:

Understand FERPA and their responsibility regarding systems and data subject to it
Understand the LA data/results and their implications before acting
Consult with the appropriate supervisor or support person when needed
Consider insights provided by LA and act in a sensitive manner appropriate to the instructional role
Inform students about centrally-available or instructor-created resources and training to help students understand LA data/results
Use de-identified data whenever possible, especially with longitudinal datasets
Only use enterprise systems unless authorized by the institution
Consult with Academic Technology when considering the use of teaching and learning technology or products not supported by UW-Madison
Track instructional interventions triggered by LA

Advisors (Academic/Career Services and Student Affairs) have the responsibility to:

Understand FERPA and their responsibility regarding systems and data subject to it
Understand the LA data/results and their implications before acting
Consult with the appropriate supervisor or support person when needed
Inform students about centrally-available or instructor-created resources and training to help students understand LA data/result
Consider insights provided by LA and act in a sensitive manner appropriate to the advisor role
Use de-identified data whenever possible, especially with longitudinal datasets
Track advising interventions triggered by LA, and ensure appropriate communication among academic, student affairs and career services advisors

Program and Institution Leadership have the responsibility to:

Understand FERPA and their responsibility regarding systems and data subject to it
Develop and support the technical infrastructure underpinning LA data use
Develop, review, and strive to ensure compliance with student intervention policies, procedures, and standards
Provide LA resources and training to help data users understand data/results
Clearly communicate how students can engage with LA presented to them
Safeguard data access
Collect and incorporate LA data for reporting and accreditation purposes as required
Use de-identified data whenever possible, especially with longitudinal datasets
Practice non-maleficence (do no harm)