Overall Instructor Rating: Exemplary
Ratings explanation:
- Exemplary - A model answer in almost every way (this is given out very rarely).
- Satisfactory - Very well done; you've met the expectations of the assignment. There are some minor problems, so read my feedback well.
- Marginal Pass - You pass, but there are lots of issues to consider. Read my feedback very carefully and be sure you understand the points/issues I raise.
- Not satisfactory, redo and resubmit - The assignment was not completed appropriately. I am concerned that you do not understand the process well enough yet. To get credit for the assignment, you need to redo it, most probably on another topic. Read and consider my feedback very carefully before redoing.
Instructor's Overall Feedback:
(Feedback given by Younghee Woo; concurred by Lloyd Rieber.)
You did an excellent job here. You seem to understand the process very well. I have some suggestions about your formative evaluation plan, so please read that feedback below carefully.
The objective of this activity is to plan for the formative evaluation of your instruction. As stated by Dick, Carey, and Carey (2001): "Formative evaluation is the process designers use to obtain data that can be used to revise their instruction to make it more efficient and effective. The emphasis in formative evlauation is on the collection and analysis of data and the revision of the instruction" (p. 284-285).
Using the lesson you have described in the IDA on lesson design, complete the Formative Evaluation Planning Worksheet. Using the table below, describe:
- Who will do the ongoing evaluation;
- What is being evaluated;
- When the evaluation should occur; and
- How the evaluation will be conducted (i.e. tools you will use)
Be sure to refer to Dick, Carey, and Carey Chapter 10 for specific information that can be gathered from experts (e.g. subject-matter, learning, and learner) and learners in different formative evaluation situations (e.g. one-to-one, small-group, and field trials).
Formative Evaluation Planning Worksheet
Write the instructional objectives of your lesson here (it can be more than one):
Your final response:
Lesson 1: Types of Paint - Glossy and Flat
Lesson 1 Objective: Employees will be able to classify paint as either flat or glossy by being able to correctly interpret paint can labels and ST product literature.
(Dr. Rieber, I know I have probably "overshot" the mark (again)in this IDA, but it is very difficult to take an ADDIE component like evaluation, which is a rather comprehensive component geared toward the whole of the instruction, and make it a lesson evaluation. Besides, I needed to work it out this way to make sure that I understood and would be able to perform such a task on the full scale. For all intent and purpose, I could select applicable portions of the Who, What, When, and How and apply them to an individual lesson.)
The instructor's feedback to this step:
Looks good!
Write your objective using the correct form. Click here for advice on writing objectives.
HTML formatting reminder: You can embed HTML tags directly in your responses. This will be help your buddies and the instructor view your information in a more readable fashion. For example, if you want a paragraph break somewhere, just enter <p>:
I will have the experts review the lesson on three levels:
<p>
1. content; 2. technical accuracy; 3. quality of instruction.
<p>
The content review will help ensure that no mistakes were made in the information provided to the learners.
Who |
What |
When |
How |
| Experts (subject-matter, learning, learner) |
Your final response:
A variety of experts have been carefully chosen to assist us in conducting the following:
1.Content Review
2.Technical Accuracy Review
3.Quality of instruction Review
The Content Review will help ensure that no mistakes have been made in the general information provided in the instruction.
This review will also provide feedback on how current the information is in the instruction -- including the currency of the information in the ST videos, ST product information guides, manufacturer's literature, and the new ST Intranet website.
The Technical Accuracy Review will help us to determine if our technical references to flat and glossy paint, paint characteristics, and paint uses are correct.
On the Quality of Instruction Review, the experts will rate the instruction on its, relevance, effectiveness, and efficiency.
|
Your final response:
All of our experts will receive a packet containing all of the instructional materials that will be used during training, including the training manual, training CD (games, videos, exercises, etc.), quick reference materials, product materials, and assessments.
The experts will also receive an agenda one week prior to the review sessions.
Content, and Technical and Quality of Instruction Reviews will take place during three separate sessions with the experts.
All three reviews with the experts will be completed prior to the one-on-one reviews.
|
Your final response:
All reviews will be conducted through the use checklists, questionnaires, evaluations, interviews, and group discussions with the experts.
At least one expert for each review will be needed for follow-up interviews.
|
The instructor's feedback to this step:
This looks very good. You have well defined the elements to be evaluated and clearly described the evaluation process. I just recommend that you make clear who are experts for reviewing the content, technical accuracy, and quality of instruction. You might need different experts for each part. |
| Learners: one-to-one |
Your final response:
Three sample learners will be selected for the evaluation of the “learnability” of the instruction.
We will have a new Paint Department employee with less than one month of experience, an employee with six months of experience, and an employee with two years of experience.
|
Your final response:
The one-on-one learner sessions will occur as soon as the subject matter experts have completed the Content, Technical, and Quality of Instruction Reviews.
|
Your final response:
The designer will sit with each learner as they go through the introduction to the training, pre-tests, the training manual, the training CD (containing practice exercises, games, self-paced quizzes, timed quizzes, and role-plays), quick reference materials, videos, the Intranet website, product information guides, manufacturers'literature, and posttests.
The learner will mark, circle, and make notations in the instruction where the information is confusing, incorrect, redundant, etc. The designer will answer questions from the learners about the instruction.
Indepth one-on-one interviews will be conducted after each learner has gone through the instruction. The learner will answer questions that will aid the designer in clarifying information.
In addition to clarity, the one-on-one sessions will help the designer address issues related to the impact and the feasibility of the instruction.
|
The instructor's feedback to this step:
Excellent! I felt everything was very well designed. |
| Learners: small-group |
Your final response:
The small-group learners will be evaluating the learnability” of the instruction.
A group of 5-6 learners from the Paint Department from different ST store locations will be selected for this evaluation.
|
Your final response:
The small-group learners' session will occur after the subject matter expert reviews, and after the one-on-one reviews.
This session will take place once all feedback and revisions from the two prior sessions have been properly documented and incorporated into the prototype of the instruction.
|
Your final response:
For the most part, we will follow the same format as we did with the one-on-one learners. However, posttests and post-instruction interviews will be of greater importance as we try to revise the instruction one last time before the field trial.
This may be the final opportunity to measure the effectiveness of the changes made after the prior sessions.
This may also be the last opportunity to identify any remaining problems with the instruction.
We will have these learners complete an attitude questionnaire.
|
The instructor's feedback to this step:
Looks good! But, I recommend that you add the elements to be evaluated in the 'what' section. |
| Learners: field trial |
Your final response:
The learners in the field trial will be evaluating the “learnability” of the instruction.
The learners in the field test will be 6 new hires who will be using the instruction once it is completed.
Since the field learners are the actual learners, they will follow a schedule and be placed in an environment appropriate for learning this material.
|
Your final response:
Based on the feedback received from prior sessions, all changes and revisions have been made. This instruction is in the final format for these learners.
Minor changes may need to be made to the instruction and will be made as needed and as time permits.
|
Your final response:
The field trial will take place in the actual setting with actual learners.
As with the small group, we will use posttests and post interviews to make an assessment of the instruction.
We will also use questionnaires as needed.
|
The instructor's feedback to this step:
Looks good!. However, I recommend that you add some different evaluation methods for the field-trial. When collecting data in a real setting, observation of the instruction would be very valuable.
|
|