Course Quality Assurance Rubric
Last updated by neal.legler 3 years ago CIDI’s Quality Assurance rubric defines basic standards for usability of instructional content posted online. Some standards apply to all course delivery methods, and others apply primarily to fully online courses, as indicated. All course delivery methods, however, can benefit from following these guidelines.
Accessibility | Online | Other | |||
☐ | A1 | Files: Files are accessible and mobile friendly (Prefer HTML format). | ✔ | ✔ | |
☐ | A2 | Images: Non-decorative images have descriptive text (Checked by Ally). | ✔ | ✔ | |
☐ | A3 | Captions: Videos with spoken audio are captioned. | ✔ | ✔ | |
☐ | A4 | HTML: Text headings use properly nested heading tags. Links are descriptive. | ✔ | ✔ | |
☐ | A5 | Color: Foreground and background colors pass the WCAG AA standard for contrast. | ✔ | ✔ | |
Presentation | |||||
☐ | P1 | Links: All links direct to the intended, functional, published location (No broken links). | ✔ | ✔ | |
☐ | P2 | Tidiness: No extraneous items (navigation, files, assignments) are visible to students. | ✔ | ✔ | |
P3 | Layout: No overly distracting visual layout issues exist. | ✔ | ✔ | ||
☐ | P4 | Video: No video display or playback problems exist. | ✔ | ✔ | |
☐ | P5 | Organization: Content is organized in a logical way for students. | ✔ | ✔ | |
Content Readiness | |||||
☐ | C1 | Currency: All content items reflect the correct semester. | ✔ | ✔ | |
☐ | C2 | Grading: Grading methods and key student and teacher expectations are posted. | ✔ | ✔ | |
☐ | C3 | Instructions: Assignments have adequately detailed instructions. | ✔ | ||
☐ | C4 | Progression: The final grade is not based solely on a few high stakes assessments. | ✔ | ||
☐ | C5 | Completeness: Students have enough content to learn and succeed. | ✔ | ||
Bonus | |||||
These items are not required, but may present areas for improvement and suggestion. | |||||
☐ | B1 | Template: The course uses a styled, organized home page and content pages. | ✔ | ✔ | |
☐ | B2 | Policies: USU institutional policies are included. | ✔ | ✔ | |
☐ | B3 | Orientation: The course contains an introduction and orientation | ✔ | ||
☐ | B4 | Objectives: Course-level and module-level objectives are included. | ✔ | ||
Common QA Criteria Problems
The Quality Assurance Rubric is written in positive terms describing what a course should look like. However, the quality assurance process involves marking off the criteria for which a course does not meet the standards. For this reason, each of the criteria are listed below with descriptions of the most common problem areas. When you see one of these problem areas in a course (or something of a similar nature):
In the status comments, write a brief note or two about the nature of the problem for the person who will attempt to fix it.
Accessibility Section
This section takes first priority in the QA process. The criteria in the accessibility section present the greatest usability impact for the most possible users. Federal mandate also requires online materials to be accessible.
The Files, Images, and Captions criteria in this section are associated with ongoing accessibility projects . For these criteria, the standard “Fix” option is replaced with an option to add the course to one of these accessibility project boards.
A1 - Files: Files are accessible and mobile friendly (Prefer HTML format).
Selecting “ + Conversion ” adds the course to the File Conversion Courses project board, where staff for CIDI and Digital Accessibility Services will work to convert the course’s inaccessible files into accessible formats–usually in the form of Canvas pages.
Select “+ Conversion” if you see 5 or more files–particularly PDFs–in use that are inaccessible and/or good candidates for file conversion (creating an HTML Canvas page of the content). We recommend using TidyUp to identify a list of files that are in use.
PDFs of scanned documents are the most inaccessible. If you see any of these in use (even less than 5), and they don’t meet the criteria below as “bad candidates for conversion,” select “+ Conversion.”
PDFs, Word documents, and other files can be of high quality–readable and not scanned–but still be a good candidate for file conversion. Consider how usable the file would be on a mobile device or screen reader, and consider whether a converted HTML version would be noticeably more usable. If the file could be converted without much trouble, and if the converted version would be more usable, then it is probably a good candidate for file conversion.
The QA/Accessibility board includes course data to help identify courses that are good candidates for file conversion. The “PDF,” “PDF in Use,” and “PDF no OCR” columns are good references. (OCR stands for optical character recognition. Files without OCR are most likely scanned.) The “Files Ally” column provides the overall Ally report score for file accessibility and can also be a great, at-a-glance indicator of whether a file needs file conversion help.
Some types of files are not good candidates for file conversion. Do not count the following (as noted in the DAS file conversion documentation :
Files with more than 30-35 pages (unless there are multiple chapters/articles that would be easy to break up).
Files that are primarily visual (significant/meaningful graphic design) such as brochures, flyers, or graphic design portfolios where the layout is crucial that would be difficult to replicate in HTML.
Outdated documents (e.g. a syllabus from 2012. You may need to look into the course and see how it's being used.)
A2 - Images: Non-decorative images have descriptive text.
Select “+ Images” when you see a course with images–particularly those used for instruction–that do not have informative descriptions in the HTML <img> tags.
Image descriptions appear in the alt attribute of the image tag: <img href=”gw-delaware-crossing.png” alt=”George Washington standing regally on a boat, filled with bedraggled soldiers, crossing the icy Delaware river”>.
In Canvas, images are nearly always given alt tags, but the filename is used for the alt tag description: <img href=”important-info.png” alt=”important-info.png”>. This is not an informative description. Either fix it on the spot (if permission is given), or select the “+ Images” option.
Images that are purely decorative should have an empty alt tag: <img href=”decorative-image.png” alt=””>. If decorative images do not have an empty alt tag, either fix them on the spot (if permission is given), or select the “+ Images” option.
Courses with image description problems can be quickly spotted by referring to the “Images no Alt” column in the QA/Accessibility board. Be sure to also refer to the “Images in Use” column to get a sense of whether the
You can also open the course’s Ally report to drill down to a list of images that do not have appropriate descriptions. You can find a series of Chrome bookmarklets, including one to readily access a course’s Ally report, on the Digital Accessibility Services bookmarklets page .
A3 - Captions: Videos with spoken audio are captioned.
Selecting “+ Captions” adds the course to the Captioning project board for further work adding closed captions to the videos. Select “+ Captions” if you see:
Videos that do not have a “CC” icon in the video player, allowing the user to turn on closed captions.
A4 - HTML: Text headings use properly nested heading tags. Links are descriptive.
Mark an X if you see:
Seemingly unintended differences in font size from one element to another element of the same type, such as a paragraph or list item. This is usually the result of copying/pasting.
Text headers that are created by simply increasing font size and bolding text rather than using header tags (H1, H2, H3, etc.)
HTML heading tags are not used in their proper sequence and hierarchy. Some heading levels are skipped, or they are used out of order.
A5 - Color: Foreground and background colors pass the WCAG AA standard for contrast
Mark an X if you see:
Color contrast issues: The color of an important foreground element, usually text, doesn’t stand out enough from its background color. The WebAIM WAVE accessibility browser extension is a great tool for checking color contrast issues. https://wave.webaim.org/
Colors, such as red or green, used as the only way to differentiate items, such as correct and incorrect, from each other.
Presentation (P) Section
Approach the QA check of this sections as follows:
Review all of the criteria in the Presentation and Content Readiness (P/C) sections, described below.
Next, decide if, taken altogether, they pose a sufficient challenge for students and faculty that one or more should be fixed.
P1 - Links: All links direct to the intended, functional, published location (No broken links).
Broken Links:
The easiest way to check for broken links is to run the link validator in the Canvas course settings. Check broken links for the following:
Is the link in active use? Is the item it appears in published? Listed in the course modules? Related to current content?
Is the target content still available? For example, is it a link to a website that no longer exists? Or is a link to a file or assignment that is no longer part of the course? Has the content moved or failed to copy properly? Or has the Canvas link become corrupted?
Mark an X in this category if you see broken links that are meant for active use.
Make a note in the status comments on whether the target of the broken link is still available for re-linking or needs to be located.
If broken links appear in unused content, consider whether the content is extraneous and should be removed. If so, mark an X in the P7 category instead.
P2 - Tidiness: No extraneous terms (navigation, files, assignments) are visible to students.
Run the TidyUp tool to check this category. Mark an X if you find:
Non-duplicate, but old or otherwise unnecessary and/or empty files, pages, or assignments that are visible to students.
Empty pages, assignments, headings, and other unfinished items left out in plain view with no explanation.
It’s possible for a course to have a noticeable amount of unused content that is not visible to students. This isn’t necessarily a QA problem for the student experience, but it can result in maintenance problems. If you see this, mark an X and make a note of it.
P3 - Layout: No overly distracting visual layout issues exist.
Mark an X if you see :
Mark an X in this category for any of the above, and add notes in the status comments describing the issue.
P4 - Video: No video display or playback problems exist.
Mark an X if you see:
Embedded videos showing errors, such as unauthorized access, content that is unavailable or still processing, or an altogether blank box.
Three or more videos embedded on the same page that aren’t listed together in a playlist. (In addition to often appearing unpleasant, this can also slow down page load time by too much.)
P5 - Organization: Content is organized in a logical way for students.
Problems in this category cannot be defined comprehensively or even objectively. When you first access a course, think as if you were a student trying to find the syllabus, a course schedule, course readings, assignment descriptions, and course expectations. If this does not come easily to you, mark an X in this category.
Common issues may include (and are not limited to):
Inconsistent presentation of content. This can happen when content from multiple courses and teachers gets combined without enough alignment and clean-up.
Content Readiness (C) Section
Approach the QA check of this section as follows:
Review all of the criteria in the Presentation and Content Readiness (P/C) sections, described below.
Next, decide if, taken altogether, they pose a sufficient challenge for students and faculty that one or more should be fixed.
C1 - Currency: All content items reflect the correct semester.
Mark an X if you see:
C2 - Grading: Grading methods and key student and teacher expectations are posted.
Mark an X if you see:
No explanation of teachers expectations of their students (i.e. late work policy, communication, etc.)
No explanation to students as to what they can expect from their teachers (i.e. response to emails, etc.)
C3 - Instructions: Assignments have adequately detailed instructions.
If the course is an online course without regular synchronous meetings, mark an X if you see:
C4 - Progression: The final grade is not based solely on a few high stakes assessments.
Mark an X in this category if the course is fully online and appears to present students with a high stakes exam or assignment without any lower-stakes activities that will give them a chance to practice, receive feedback, and prepare in advance of the high-stakes activity.
Automated feedback is passable, although make a note if you see no assignments where an actual instructor provides feedback.
C5 - Completeness: Students have enough content to learn and succeed.
If the course is an online course, mark an X if you see:
Assessments or assignments that don’t seem to be addressed adequately by the lectures, assigned readings, and other content.
Unpublished assignments or quizzes that have current due dates and instructions, no published duplicates, and no other reason to be unpublished.
Rare: Navigation items that are hidden from students but contain content that students aren’t given any other way to access. (For example, a hidden Assignments navigation link when there are assignments available that are not linked to elsewhere in the course.)
Bonus Section (B)
Criteria in the bonus section nearly always improve the student experience. However, they require additional work and instructor approval and are not required for the basic quality assurance review. However, making note of where bonus improvements can be made will provide opportunities for further course improvement if time and resources allow. If you feel like changes in one or more of these areas would greatly enhance the student experience to the point that they should be done, if at all possible, then be sure to select “+ Recommend” in the “+ Bonus Update” column.
B1 - Template: The course uses a styled, organized home page and content pages.
Mark an X in this category if:
B2 - Policies: USU institutional policies are included.
Mark an X in this category if the syllabus contains very few, if any, of the USU standard policies language.
B3 - Orientation: the course contains an introduction and orientation
Mark an X in this category if the course is fully online and:
B4 - Objectives: Course-level and module-level objectives are included.
Mark an X in this category if the course is fully online and:
