Case Study: Program Evaluation
GOVT 342. Professor Judith Owens-Manley, Hamilton College
Please provide the course description as it appears in the catalog and the typical enrollment for the course.
Program evaluation is an integral component in the setting of public policy and its implementation. This course will build on knowledge and skills developed in initial research methods courses and acquaint students with a range of methodologies used to evaluate programs in various settings.
Typical enrollment is 4-12 and has been as high as 18 in extenuating circumstances when students needed the course to fulfill a requirement. There were 4 students in this particular course.
What are the learning goals of the media assignment(s) in the course. If your course assignments contain both analysis and creative production components, describe the learning goals of each.
- Students able to operate the video camera and understand basic principles of filming both landscape and individuals during interviews.
- Students understand and demonstrate logging video material.
- Students understand and demonstrate creation of codes for analysis and then coding pieces of video data.
- Students edit portion of video data.
What are the relationships of the media assignment(s) to the other aspects of the course?
Media assignments were the community-based research portion, their on-the-ground implementation of what they understood from reading the background literature. In other words, they read reports about other HOPE VI Projects in other U.S. cities, then they attended live interviews, visited the housing site that was torn down, were in the homes of people who had been displaced by a razed housing project, and thought about how to tell that story.
Describe your assignment design/structure.
Students began videotaping or audio taping resident interviews after the first few weeks so that material could be logged and coded as we went along. Assignments began to be divided early on as the visits to the community were time intensive and did not need to involve everyone. Also there were different levels of skills with students involved and prior exposure to videotaping and editing; also two students, although interested, were not willing to commit the time that it would have taken to develop the expertise. The design and structure evolved as we went along, as interviews were canceled, unexpected delays occurred, and we had to be flexible in our timeline.
To what extent does your assignment design address issues of visual/aural literacy?
The assignment design was to have students prepare material for visual display on a website, and this was not addressed as fully as we originally had hoped due to lack of time.
To what extent does your assignment design address issues of information literacy?
The assignment included responsibility for logging video and audio-taped material and then coding it for analysis.
To what extent does your assignment design address issues of technology skills?
The assignment did address technology skills for taping and editing. Students were to spend time in the lab learning these skills by editing portions of tape or particular tapes that they took responsibility for.
What are the resources necessary for your assignment (content/materials, institutional support, equipment)?
Institutional support was absolutely necessary, as I had no background myself for having the students videotape and edit interviews, preparing the material to be featured on the website. The equipment for videotaping and audiotaping interviews as well as the software for editing was critical. I would not have been able to do this course in the way that it was designed without the considerable participation of ITS support.
Describe how you evaluated the project outcomes? Did you evaluate process? Outcomes? Both?
- Process was evaluated by considering what the student experience was and was included in a poster session.
- Process was also evaluated by considering the experience of community residents who had been interviewed. They were invited to a luncheon that was attended by approximately 60 people, most residents of the former housing project and the "subjects" of the community-based research. Their feedback was that they were grateful and very interested in the final product, a DVD that was given to each resident that told the story of their lives in Washington Courts before it was torn down.
Estimate the time invested in the project by you, your students, and academic support staff.
The time investment was huge for all of us, and it could probably be streamlined some by understanding from the beginning how to organize the project.
How many times have you taught this course/assignment? What would you do differently next time?
I only taught it once, and it could be set up on a different topic to be repeated - a different set of interviews. Next time I would take it on with a population that I had more direct access to rather than rely on a community partner as I did this time - and a community partner that had not been entirely reliable in the past.
What is your level of expertise with respect to media technologies and scholarship?
Beginner!
The website for the 5-year project study including student work from this semester is available here.
