Perhaps the most important aspect of quality assurance (QA) when developing courseware is to validate the curriculum’s documentation. This might seem challenging at first but it’s actually easier than it seems. I use a validation chain to validate curriculum documentation.
The chain begins with the project charter. The first documentation created for most curriculum development projects is a project charter. The charter specifies the outcomes the organization would like to occur as a result of offering the new curriculum to learners. These outcomes are not learning objectives — they are organizational-level objectives. It’s critical to document the desired outcomes at the onset of the project to ensure that everything developed helps support their realization.
The desired outcomes are documented before development even begins but are not realized ‘til well after the curriculum is put into production. With everything that transpires between these two points in time, a validation chain is needed to ensure that the development team does not lose sight of the desired outcomes. To start the chain, document each desired outcome with its own unique identifier (which could be as simple as a letter or number).
After creating the project charter, the next step is to conduct a needs analysis (or study of practice). The purpose of this activity is to identify the competency gaps that the curriculum should fill. The needs analysis report documents all of the competency gaps. As with the desired outcomes, I recommend assigning a unique identifier to each competency gap. In addition, identify which outcome in the project charter each competency gap will support achieving when the learner develops the competency.
If a competency does not help the organization to achieve a desired outcome, then the curriculum should not try to develop the competency, even if there’s a gap. A different curriculum developed at a later time can develop the competency. There will always be more competency gaps in every organization than can be addressed by a given initiative. If a curriculum tries to address every competency gap that cannot be linked back to the project charter, it will get too long and developing it will waste organizational resources on issues that are not designated as priorities. Therefore, the needs analysis should focus only on the outcomes desired for the project.
After reporting the results of the needs analysis, the next step is the instructional design (ID). The ID documentation specifies each terminal and enabling learning objective. To continue the linkage in the validation chain, the ID documentation identifies which competency gap in the needs analysis report each learning objective is intended to help close. Again, if a learning objective does not develop a competency specified in the needs analysis, it should be eliminated from the curriculum. Keep the curriculum focused only on learning objectives that can be linked to the findings from the needs analysis.
The ID of elearning also documents all of the functional specifications of the courseware to be developed. A functional spec documents how an element of the courseware responds to an action taken by a learner (e.g. video plays when learner reaches screen, URL opens in browser when learner clicks hyperlink, etc.). The ID documentation should assign an identifier to each functional spec so that the specifications can also be linked into the validation chain.
The final document linked into the validation chain is the test plan. There are two QA activities that link the test plan into the validation chain: unit testing and beta testing.
Unit testing is performed on each unit of the courseware as it is developed. A unit of courseware is typically a lesson or, in the case of SCORM courseware, a Sharable Content Object (sometimes referred to as a “SCO”). The test plan documents the quality controls (QC) with which each unit is tested. Link each QC that tests a function to its corresponding functional specification in the ID documentation. Likewise, every functional spec in the ID documentation should have a corresponding QC documented in the test plan.
Beta testing is performed on the entire curriculum when all the courseware is developed. The beta test delivers the full curriculum just as it would be delivered in production. Beta testers are from the audience the curriculum targets, so the members of the development team that perform other QA activities are typically not appropriate for beta testing. However, a key customer will often provide learners to act as beta testers or other learners are typically willing to volunteer for beta testing because of the opportunity to get the training early and free.
The purpose of beta testing is to determine how well a curriculum meets the needs of the target audience and helps learners to achieve the learning objectives. Therefore, each learning objective should have a corresponding evaluation criterion in the beta test. The evaluation criteria are tested using a survey of the beta testers upon completion of the curriculum. The test plan documents the survey form and the evaluation criteria for the beta test and refers back to the ID documentation for each learning objective evaluated.
By linking all the curriculum documentation together in this manner, I can trace the QA back from the test plan through the ID documentation and needs analysis report to the project charter. This chain validates all the curriculum documentation and maintains consistency throughout it. This helps ensure that the curriculum put into production will support the realization of the outcomes the organization wanted from the curriculum at the outset.