Desire2Learn - Data Purge Tool
NOTE: This document is expressly intended for campus D2L Administrators. Please contact your campus administrator or help desk if you have questions.
The D2L Data Purge tool provides the function to create "non-restorable" archives containing course materials, files, and user activity. It should only be used to create course archives and shouldn't be used to purge courses.
This document provides instructions on how to use the tool and access the output archives. The process to access and download the archives is similar to the Bulk Course Export process.
Please note that the process is scheduled to run daily, so it may take up to 24 hours for the output files to become available. If you run into any difficulties with the process, please complete and submit a Problem Reporting Form.
The Data Purge tool can be used to extract course data to meet the course retention policy requirements. For all instructional courses run on D2L after June 30, 2019, the Data Purge Tool should be run for the course and the output saved locally on your institution's storage. The output from the Data Purge tool should be deleted one year after the course end date as there is no requirement to retain the data longer than this period.
The D2L Data Purge tool has the ability to archive the following information, if those tools are in use within the course:
- Org unit
- Org unit enrollments
- Org unit components and user activity for:
- Content modules and topics
- Course files
- Group contributions such as group dropbox folders
- Group Locker
- Quizzes, Question Library, Surveys, and Self-Assessments
- Release Conditions
- For org unit data sets: only queue 1 course per set, otherwise, courses will be merged into a single output folder. This will ensure compliance with the UW System retention policy.
- Put the course OU in the name of the data set; do not use the upload .csv option.
- Store data and files in an archive, don't purge anything.
- Create an archive of all course materials, user activity, and files in the course path.
- Download the archives when they become available as the output files will be automatically removed 7 days after creation.
- Log into your site as an administrator
- Click the Gear Icon and choose Data Purge from the list
- Click Create Data Set and choose Org Unit data set from the dropdown
- In the Purge Set Name field, enter the name you'd like to give the data set, including the Course OU - e.g. Utility Support Course - 117171
- Click Add Org Units and choose Enter Org Unit IDs from the dropdown
Note: do not use the Import a CSV file option
- Type in the Course OU of the course you would like to create an archive for and click Add
- Click Save
- This will place the data set in a "Draft" state - if edits need to be made, do so now by clicking the downward arrow next to the name of the data set and choosing Edit
- Once you are ready to queue the data set for processing, click the downward arrow next to the name and click Queue for processing
- You will see the status changed to "Queued"
You have the option to abort the processing of the data set during this time, if you no longer require an archive of the course; this can be done by clicking the downward arrow next to the name of the data set and selecting Abort this data set from the dropdown - it will then display the status as "Aborted"
- When the process run, the status will change to "Processing" and will show the percentage completed so far
- The process runs daily; once completed, it will change to "Completed"
If it is displaying as "Paused", it may not have completed processing; continue to monitor the status of the data set until it resumes processing and is completed
Note: If the status shows as "Failed", please submit a Problem Reporting Form detailing the issue
- Log into your site as an administrator
- Double check that your data set has completed (within Admin Tools > Data Purge, the data set will display as "Completed")
Note: If a data set had begun processing but is showing as "Paused", a folder will still exist for it; do not download this folder until the data set has completed processing as it is incomplete
- Navigate to your Data Archive course
- Click Edit Course
- Click Manage Files
- Click on the output folder
Note: Each data set is assigned a number; if there are many folders, you can determine which data set is yours, or is associated with a specific course, by clicking through the folder tree until you find an info.xml file. Downloading and opening this in a program such as Notepad will display the name of the data set, which should contain the Course OU number.
- Download the entire folder that the associated info.xml file is located in (the folder will be named in the format of a timestamp - e.g. 2019-07-09T20_09_51.030Z) by checking the box next to the folder and clicking the Download button
- This will open a new window that generates a .zip file; once it's complete, click the link to download the .zip file of the archive
- Rename the file to something useful (such as the course name and OU) - your course is successfully archived!
- If you would like to keep the course files organized, you can delete the folder once you have downloaded and saved it from the Manage Files area of the Data Archive course
Note: The archives will be purged 7 days after creation
When you first open the .zip file, you'll see the info.xml file, which contains an overview of org unit data that was archived. It does not contain any data that will be relevant to course materials that an instructor may want to have had archived.
There is also a manifest.xml file, which contains a list of database tables. This file can be ignored.
Expand the folder with the string of numbers (this is your University Org ID number). You will be presented with something that looks similar to the following:
Each of these numbered folders represent an 'org' - an org can be a merged course section (e.g. History 101 Sec 01), a group (e.g. Team 1), or the course as a whole (e.g. History 101). The groups or sections will mostly be useful for group Dropboxes or Discussions.
There will also be a folder titled "Org Files" which will contain only the Content files. The course as a whole is typically the smallest number, numerically, and contains the majority of the course materials.
You will also periodically see folders titled "Users". Within these folders will be numbered folders - each folder corresponds to a specific user.
Notes about additional resources available in the Data Archive course File Manager:
- A CSV formatted file, named users.csv, contains a list of your D2L users (names and OrgunitID) can be used determine which numeric identifier belongs to which user listed in the archived data.
- A CSV formatted file, named CourseSections.csv, lists all D2L course OUs and the corresponding sections available in the course.
- These two files are automatically generated every Sunday.
If you'd prefer to watch this process, screen recordings are included below. They do not involve looking at the index.xml file or navigating the zipped folder post-download.
To submit a course for archive:
To download that data archive: