Return to the list of Formative Evaluation IDAs | Return to the IDA Library Home Page

Instructional Design Activity: Formative Evaluation

 

Overall Instructor Rating: Satisfactory

Ratings explanation:

  • Exemplary - A model answer in almost every way (this is given out very rarely).
  • Satisfactory - Very well done; you've met the expectations of the assignment. There are some minor problems, so read my feedback well.
  • Marginal Pass - You pass, but there are lots of issues to consider. Read my feedback very carefully and be sure you understand the points/issues I raise.
  • Not satisfactory, redo and resubmit - The assignment was not completed appropriately. I am concerned that you do not understand the process well enough yet. To get credit for the assignment, you need to redo it, most probably on another topic. Read and consider my feedback very carefully before redoing.

Instructor's Overall Feedback:

(Feedback given by Younghee Woo; concurred by Lloyd Rieber.)

Overall, I think you understand the process of formative evaluation well. Here are a couple of things to think about however. So please read that feedback below carefully.

 

The objective of this activity is to plan for the formative evaluation of your instruction. As stated by Dick, Carey, and Carey (2001): "Formative evaluation is the process designers use to obtain data that can be used to revise their instruction to make it more efficient and effective. The emphasis in formative evlauation is on the collection and analysis of data and the revision of the instruction" (p. 284-285).

Using the lesson you have described in the IDA on lesson design, complete the Formative Evaluation Planning Worksheet. Using the table below, describe:

  • Who will do the ongoing evaluation;
  • What is being evaluated;
  • When the evaluation should occur; and
  • How the evaluation will be conducted (i.e. tools you will use)

Be sure to refer to Dick, Carey, and Carey Chapter 10 for specific information that can be gathered from experts (e.g. subject-matter, learning, and learner) and learners in different formative evaluation situations (e.g. one-to-one, small-group, and field trials).

 

Formative Evaluation Planning Worksheet

Write the instructional objectives of your lesson here (it can be more than one):

Your final response:

Given a problem, SWBAT demonstrate the ability to create conditional statements in a scripting language to solve that problem.

The instructor's feedback to this step:

This is well stated.

Write your objective using the correct form. Click here for advice on writing objectives.

HTML formatting reminder: You can embed HTML tags directly in your responses. This will be help your buddies and the instructor view your information in a more readable fashion. For example, if you want a paragraph break somewhere, just enter <p>:

I will have the experts review the lesson on three levels:
<p>
1. content; 2. technical accuracy; 3. quality of instruction.
<p>
The content review will help ensure that no mistakes were made in the information provided to the learners.

Who
What
When
How
Experts (subject-matter, learning, learner)

Your final response:

Experts will review content for accuracy and correctness. This will help ensure that code examples work properly when implemented and that other content is accurate.

Also, interface experts will be utilized to review the design concepts used in development of the medium used.

 

Your final response:

Two evaluations will be done. An initial one after a rough draft of course content has been developed will be done near the beginning of development.

Another evaluation of final content will be completed after final draft is completed.

An evaluation of the interface and design concepts will take place between the rough and final drafts to ensure usability and intuitive design do not impede the instruction process.

Your final response:

Checklists and panel interviews will be conducted to evaluate content.

Evaluation of the interface and design concepts will be completed by a testing and quality assurance (TQA) and Human-Computer Interaction (HCI) consultant team. The developer's will be debriefed on the consultant team's findings.

The instructor's feedback to this step:

Your design is excellent. You seem to understand the expert review process quite well.
Learners: one-to-one

Your final response:

5 sample learners will be selected who will cover the range of potential learners from basic to expert.

Your final response:

These one-on-one sessions will be conducted after the final draft phase of development. These one-on-one sessions will specifically be used to examine the developed medium of instruction. This will be to verify that the design is intuitive and does not retract from the learning process or transfer of information from medium to learner.

Your final response:

A room with a computer and various cameras (hidden) as well as screen capture software will be used. The learner will go through the lesson and the designers will review the video footage and screen capture footage to find possible problems. A survey of the learner's experience will also be utilized as well as a debriefing session with all sample learners and designers present.

The instructor's feedback to this step:

Excellent! I just recommend that you add the criteria for selecting sample learners. And you mentioned you will use cameras -- learners may feel uncomfortable being filmed, so establishing rapport with learners will be an important issue. There are ethical considerations about filming learners without their consent.
Learners: small-group

Your final response:

A small group will be used to evalutate the content and examples to be used.

Your final response:

This small group session will be conducted after the rough draft phase of content development. This session will not examine the medium to be used for the instruction only the content.

Your final response:

This will be administered in a classroom situation with an instructor. A test and survey will be given at the end of the lesson to validate learner response.

The instructor's feedback to this step:

Good job! If you add the elements to be evaluated, it would be better.
Learners: field trial

Your final response:

The instruction will be test run in a single course section.

Your final response:

This will be the final evaluation before the instruction goes "live".

Your final response:

The evaluation will take place in the exact environment where it will be utilized.

The instructor's feedback to this step:

I recommend that you add the specific evaluation method such as interview or observation and so on, and the elements to be evaluated in field trial stage.