Overall Instructor Rating: Satisfactory
Ratings explanation:
- Exemplary - A model answer in almost every way (this is given out very rarely).
- Satisfactory - Very well done; you've met the expectations of the assignment. There are some minor problems, so read my feedback well.
- Marginal Pass - You pass, but there are lots of issues to consider. Read my feedback very carefully and be sure you understand the points/issues I raise.
- Not satisfactory, redo and resubmit - The assignment was not completed appropriately. I am concerned that you do not understand the process well enough yet. To get credit for the assignment, you need to redo it, most probably on another topic. Read and consider my feedback very carefully before redoing.
Instructor's Overall Feedback:
Hi Brent:
You effectively addressed the issues in your first draft. Excellent work.
Dr. Law
Hi, Brent
Your IDA contains a concrete and specific formative evaluation plan; however, the focus of some evaluation is not headed in the right direction. Please see my feedback below:
1. You’ve got a good plan for the expert review part. I liked the idea of having expert formative evaluation simultaneously with the design process. In many cases, you need SMEs during your design process and their formative evaluation really occurs throughout the design process.
2. Your one-to-one, small group, and field trial evaluations focus on evaluating learners, not your instruction. The purpose of evaluation is to see whether your “two-hour training on managing a bike recycling program session” works as you intended. Learners are supposed to help you to improve the training program. In your learner-related evaluations, you plan to evaluate whether they learn from the training and whether they can perform correctly. The data regarding their learning and performance as a result of learning could be used as part of your evaluation data. For example, in your one-to-one evaluation, the purpose of the interviews is to assess learners’ ability regarding the learning contents. If you want to conduct interviews with individual learners, the interviews should focus on what they think about the training, what their suggestions for improvement are and whether they learned what they need to learn. In your field-testing, the instructor should teach the training, rather than let learners manage a BRP session assuming that they already had the training. Does my feedback on focus and purpose of formative evaluation make sense to you?
3. In that sense, the timing of your learner formative evaluation does not quite fit the purpose of formative evaluation. Usually, the earliest trial in the actual setting would be your field-test, even though it is still a part of test. A field-test is conducted in the most similar situation to your actual training. Your one-to-one and small group evaluations usually help you be ready for your first trial. It is hard to have your one-to-one evaluation after you already have your first instruction. Without actual interaction with potential learners, how would you know that your training program is ready for your learners?
For your reference, I would like to recommend revisiting the IDA example of formative evaluation and the Impatica presentation. They will help you reconceptualize this IDA.
I hope my feedback helps you. If you have any questions, please feel free to ask me.
Thanks.
Best,
Eun Jung
The objective of this activity is to plan for the formative evaluation of your instruction. As stated by Dick, Carey, and Carey (2001): "Formative evaluation is the process designers use to obtain data that can be used to revise their instruction to make it more efficient and effective. The emphasis in formative evlauation is on the collection and analysis of data and the revision of the instruction" (p. 284-285).
Using the lesson you have described in the IDA on lesson design, complete the Formative Evaluation Planning Worksheet. Using the table below, describe:
- Who will do the ongoing evaluation;
- What is being evaluated;
- When the evaluation should occur; and
- How the evaluation will be conducted (i.e. tools you will use)
Be sure to refer to Dick, Carey, and Carey Chapter 10 for specific information that can be gathered from experts (e.g. subject-matter, learning, and learner) and learners in different formative evaluation situations (e.g. one-to-one, small-group, and field trials).
Formative Evaluation Planning Worksheet
Write the instructional objectives of your lesson here (it can be more than one):
Your final response:
Following a two-hour training on "Managing a Bike Recycling Program Session,"
1) LWBAT assess condition of bicycles needing repair, utilizing a standardized checklist;
2) LWBAT describe how to assign bicycle repair tasks to available volunteers, based on volunteer skill level;
3) LWBAT develop achievable Bike Recycling Program session goals, based on available volunteer skills and the condition of bicycles requiring repair;
4) LWBAT coordinate a Bike Recycling Program session such that session production goals are met.
The instructor's feedback to this step:
No specific feedback given on this step.
Write your objective using the correct form. Click here for advice on writing objectives.
HTML formatting reminder: You can embed HTML tags directly in your responses. This will be help your buddies and the instructor view your information in a more readable fashion. For example, if you want a paragraph break somewhere, just enter <p>:
I will have the experts review the lesson on three levels:
<p>
1. content; 2. technical accuracy; 3. quality of instruction.
<p>
The content review will help ensure that no mistakes were made in the information provided to the learners.
Who |
What |
When |
How |
| Experts (subject-matter, learning, learner) |
Your final response:
Lesson content will be evaluated by two individuals: an advanced bicycle mechanic and the Director of the Bike Recycling Program.
The bike mechanic will evaluate the bike repair checklist for thoroughness and clarity. S/he will also assess the workshop tools/facility orientation information for logical progression, safety considerations, and technical accuracy.
The BRP Director will review and modify volunteer skill assessment instruments, goal setting guidelines, and effective role playing scenarios used during instruction.
Lesson content will be evaluated by two individuals: an advanced bicycle mechanic and the Director of the Bike Recycling Program.
The bike mechanic will evaluate the bike repair checklist for thoroughness and clarity. S/he will also assess the workshop tools/facility orientation information for logical progression, safety considerations, and technical accuracy.
The BRP Director will review and modify volunteer skill assessment instruments, goal setting guidelines, and effective role playing scenarios used during instruction.
|
Your final response:
Expert formative evaluation will occur simultaneously with the instructional design process. From the beginning, SME perspectives will be critical as strategies are explored and articulated for developing an effective BRP Coordinator training lesson.
Expert formative evaluation will occur simultaneously with the instructional design process. From the beginning, SME perspectives will be critical as strategies are explored and articulated for developing an effective BRP Coordinator training lesson.
|
Your final response:
Both SMEs will provide verbal and written feedback to lesson designers during design work sessions. Handouts emphasizing the most salient information will be distributed to learners as well.
Both SMEs will provide verbal and written feedback to lesson designers during design work sessions. Handouts emphasizing the most salient information will be distributed to learners as well.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
| Learners: one-to-one |
Your final response:
Learners, i.e. prospective BRP session coordinators, will be interviewed by the BRP Director for their ability to describe:
1) how bicycles are quickly evaluated using the repair checklist,
2) how volunteer skill sets are identified,
3) how volunteers are matched with to-be-repaired bicycles,
4) how to establish BRP session goals, based on the bicycles condition and available volunteer skill sets, and
5) how to resolve commonly encountered volunteer & BRP session management issues
Learners, i.e. prospective BRP session coordinators, will be interviewed by the BRP Director for their ability to describe:
1) how bicycles are quickly evaluated using the repair checklist,
2) how volunteer skill sets are identified,
3) how volunteers are matched with to-be-repaired bicycles,
4) how to establish BRP session goals, based on the bicycles condition and available volunteer skill sets, and
5) how to resolve commonly encountered volunteer & BRP session management issues
|
Your final response:
The BRP Director will conduct one-to-one interviews with learners in the week immediately following initial instruction. Allowing for a delay of a day or more will permit the BRP Director to assess retained knowledge better than if performed immediately following instruction.
The BRP Director will conduct one-to-one interviews with learners in the week immediately following instruction.
|
Your final response:
Evaluation will consist of an informal, face-to-face interview. The BRP Director will take notes on learner reactions as needed. These notes will assist the Director in covering issues where further instruction is needed.
Evaluation will consist of an informal, face-to-face interview. The BRP Director will take notes on learner reactions as needed.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
| Learners: small-group |
Your final response:
All target learners, i.e. prospective BRP session coordinators, will engage in a focus-group style conversation with the bicycle repair SME and the BRP Director.
First, the SMEs will present a brief review seminar of the major points covered in the training units.
Next, the SMEs will follow a Q & A format to address any lingering questions learners may have about effectively managing a successful BRP session.
All target learners, i.e. prospective BRP session coordinators, will engage in a focus-group style conversation with the bicycle repair SME and the BRP Director.
First, the SMEs will present a brief review seminar of the major points covered in the training units.
Next, the SMEs will follow a Q & A format to address any lingering questions learners may have about effectively managing a successful BRP session.
|
Your final response:
The small-group formative evaluation will occur immediately after the delivery of instruction. In this way, learners can seek clarification of issues covered in instruction, while the questions are fresh in their minds.
The small-group formative evaluation will occur simultaneously with the delivery of instruction.
|
Your final response:
Evaluation will be conducted through in-person focus group discussion and a question-and-answer session with the two SMEs.
Evaluation will be conducted through in-person focus group discussion and a question-and-answer session with the two SMEs.
|
The instructor's feedback to this step:
No specific feedback given on this step. |
| Learners: field trial |
Your final response:
Learners will manage a BRP session under the guidance and supervision of the BRP Director. The BRP Director will observe learners' familiarity with the workshop facilities, ability to assess bicycle conditions and establish session goals, and mastery of volunteer management techniques in practice.
Learners will manage a BRP session under the guidance and supervision of the BRP Director. The BRP Director will observe learners' familiarity with the workshop facilities, ability to assess bicycle conditions and establish session goals, and mastery of volunteer management techniques in practice.
|
Your final response:
The guided, supervised, learner-run BRP session will occur within two weeks of the training instruction.
The guided, supervised, learner-run BRP session will occur within two weeks of the training instruction.
|
Your final response:
The BRP Director will evaluate learner performance through direct observation, note-taking, and follow-up discussion with the learner(s).
The BRP Director will evaluate learner performance through direct observation, note-taking, and follow-up discussion with the learner(s).
|
The instructor's feedback to this step:
No specific feedback given on this step. |
|