Overall Instructor Rating: Exemplary
Ratings explanation:
- Exemplary - A model answer in almost every way (this is given out very rarely).
- Satisfactory - Very well done; you've met the expectations of the assignment. There are some minor problems, so read my feedback well.
- Marginal Pass - You pass, but there are lots of issues to consider. Read my feedback very carefully and be sure you understand the points/issues I raise.
- Not satisfactory, redo and resubmit - The assignment was not completed appropriately. I am concerned that you do not understand the process well enough yet. To get credit for the assignment, you need to redo it, most probably on another topic. Read and consider my feedback very carefully before redoing.
Instructor's Overall Feedback:
This is a very nicely-done IDA! You seem to understand every stage of the formative evaluation well. The objective is well-stated. The What sections of each evaluation stage are very specific and thorough, giving clear ideas about the exact things to be assessed about the instruction. Your distinction of what to be evaluated at different stages also indicates your understanding of the overall evaluation process. Moreover, you included “who” information in these sections too. Good job!
I also like your explication of learner selection considerations in the one-to-one and small group learner trials. That was very thoughtful and helps readers of your IDA to better understand your rationale behind the things you would do. The timeline to carry out various evaluative steps looks logical and feasible as well. More importantly, you did include the assessment of learning in small group learner trial and field trial, which is truly crucial but easy to neglect when doing this IDA.
However, I do have several notes for you to consider:
In the expert review, your focus of assessing quality of instruction is to address the question “Did the students learn what they were supposed to learn as outlined in the objective?” I suggest you reword this item or examine quality of instruction using alternative indicators because expert review happens quite early (often is the first step) in formative evaluation. At the time an expert review is conducted, the instruction should not have been tried out on learners yet.
The second thing is about When to conduct the expert review. I’m wondering whether it’ll be a little too late for expert review if the development of lesson has already been completed by that time. In that case, a lot of development effort, time (and sometimes money) would have been invested before major problems are identified by experts. A better time for this would be completion of the sketch or initial prototype. But if the instruction process itself will be a major focus of evaluation, then it could be appropriate for expert review to occur after completion of development. Just something for you to consider.
In the one-to-one and small group trial, one of your evaluation methods is post-test. What exactly will be tested? [Note from Greg: And what form will the post-test take? Will it be a multiple choice test, or something else?] A little more details will make this section perfect. The same applies for the How part of your field trial.
Regardless of the above stated minor issues, I believe you clearly grasped the important things about formative evaluation and deserve an exemplary grade for this IDA. Thank you for the hard work!
Ying
[Note from Greg: Along with my note above, I approve of Ying's feedback. Well done!]
The objective of this activity is to plan for the formative evaluation of your instruction. As stated by Dick, Carey, and Carey (2001): "Formative evaluation is the process designers use to obtain data that can be used to revise their instruction to make it more efficient and effective. The emphasis in formative evlauation is on the collection and analysis of data and the revision of the instruction" (p. 284-285).
Using the lesson you have described in the IDA on lesson design, complete the Formative Evaluation Planning Worksheet. Using the table below, describe:
- Who will do the ongoing evaluation;
- What is being evaluated;
- When the evaluation should occur; and
- How the evaluation will be conducted (i.e. tools you will use)
Be sure to refer to Dick, Carey, and Carey Chapter 10 for specific information that can be gathered from experts (e.g. subject-matter, learning, and learner) and learners in different formative evaluation situations (e.g. one-to-one, small-group, and field trials).
Formative Evaluation Planning Worksheet
Write the instructional objectives of your lesson here (it can be more than one):
Your final response:
SWBAT generate an appropriate interview ensemble (outfit) according to guidelines on the WSU Career site.
The instructor's feedback to this step:
No specific feedback given on this step.
Write your objective using the correct form. Click here for advice on writing objectives.
HTML formatting reminder: You can embed HTML tags directly in your responses. This will be help your buddies and the instructor view your information in a more readable fashion. For example, if you want a paragraph break somewhere, just enter <p>:
I will have the experts review the lesson on three levels:
<p>
1. content; 2. technical accuracy; 3. quality of instruction.
<p>
The content review will help ensure that no mistakes were made in the information provided to the learners.
Who |
What |
When |
How |
| Experts (subject-matter, learning, learner) |
Your final response:
No final draft
Content, Technical Accuracy and Quality of Instruction: The design team will have two or three experts review these three elements of this instruction.
Content:
- The content review will consist of ensuring the material is presented in a logical format
- Are the quotes sited as "thought provoking" relevant to the topic
- Are the guidleines for dress on the WSU site clear
- Can this topic be effectively taught/learned in a web-based format?
- Is the website attractive/interesting?
- Are all assignments relevant to the topic?
- Are the site photos relevant to the topic?
- Was the content free of spelling and grammatical errors?
Technical Accuracy:
- Does the site link provided to students work?
- Do all links to assignments and documents contained on the page work?
- Are the instructions clear and easy to follow on the assignments?
- Are instructions for submitting assigments clear to the students?
Quality of Instruction:
- Did the students learn what they were supposed to learn as outlined in the objective?
|
Your final response:
No final draft
As soon as the development of this lesson is completed, the experts will be provided a link to access the instruction. Since they will have personal and individual access to the instruction, they can complete it in the manner the students would. Therefore they can return feedback at their own convenience. They will be provided a structure feedback form for collecting qualitative data.
|
Your final response:
No final draft
The experts will use a structured feedback form with questions that allow rating as well as open spaces to provide qualitative feedback.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
| Learners: one-to-one |
Your final response:
No final draft
Three learners that are representative of the target group will be selected to complete the instruction for evaluation. They will be selected based on their prior knowledge level to ensure it aligns with the prior knowledge level of the target group.
The following will be reviewed:
- Is material presented in a logical format that is easy to understand for the target grade levels and ages (9th and 10th graders; 14-16 yrs old).
- Can students individually complete the instruction unaided?
- Does this age group understand the quotes sited as "thought provoking", do they understand the relevance to the topic?
- Do they find the lesson intersting? Enjoyable?
- Are the instructions clear on all assignments?
- Are the guidleines for dress on the WSU site clear to the students?
- Is the vocabulary suited to a general audience of 9th and 10th graders?
- Is the website attractive/interesting enough to hold the attention of this target group?
- Are the site photos relevant to this age group?
|
Your final response:
No final draft
One-to-One sessions will occur after the experts have given their feedback and preliminary adjustments and corrections have been made.
|
Your final response:
No final draft
Students will be asked to complete the online portion of the assignment using the PCs at the school so they can be observed. However, they will be allowed to complete the activity in which they assemble the outfit and take a photo at home to simulate how they would actually complete the assignment.
A structured observation guide will be used, and an interview guide will be used to get feedback directly from the students once the entire lesson is completed.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
| Learners: small-group |
Your final response:
No final draft
To test the viability of instruction of this lesson through online media, a group of 7-8 learners similar to the target group will be selected to complete the lesson.
The qualities of the learners to be considered for selection are: Age, grade level, employment status (prefereably never having been employed), knowledge level on the topic of interviewing for a job.
From this group, we will be able to determine:
- The usability of the instruction for this age group
- The clarity of the instructions
- Did the learners acquire the skills?
- Did they like it?
- Did it hold their interest and attention? What parts did or did not?
|
Your final response:
No final draft
This will occur after adjustments and corrections have been made in reponse to the one-to-one trials.
|
Your final response:
No final draft
This group will be instructed to complete the lesson in the same manner the target learners will. They will be given instructions to complete it individually from their home PC, or have the option to use a school PC, or other PC to which they have personal access.
Their assignment submissions (the interview outfit photo) will be evaluated, and they will be given a post-test of their learnings to confirm that they learned what is stated in the lesson objectives.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
| Learners: field trial |
Your final response:
No final draft
This group of learners is the group for which the instruction has
been developed. Through this evaluation we will determine how effectively the lesson format and the selected media (online)was in "teaching to the objectives". In others words, "Was the lesson's objective reached by the students?"
|
Your final response:
No final draft
This test will occur after the instruction has undergone any
necessary adjustments/corrections/changes and is considered to be complete.
|
Your final response:
No final draft
The evaluation will be conducted in the setting where the instruction will be
delivered (online, individually). A post test and a post-
instruction questionnaire focusing on the impact of the
learning environment to the learner outcomes will be administered. It will also focus on their reaction to the learning activity.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
|