Evaluation for Astrobiology in Secondary classrooms Teacher Workshop
Summer Workshop held at Tennessee State University June 19-23, 2006

The evaluation methodology looks at the outcomes for students as teachers emphasize:

The evaluation instruments are linked to the anticipated program outcomes listed in the Logic Model based on the Benchmarks for Science Literacy from the American Association for the Advancement of Science Project 2061 (1993), research on effective professional development (Guskey, Kirkpatrick) and research based strategies for teaching science to minority students (CREED.)  Basic satisfaction surveys will be used to determine how effective the teachers and students view the curriculum. The ASC Learning Community Collaboratory will be used to track all of the evaluation information. The analysis of the information will be used for the formative evaluation needed to revise the curriculum. The information gathered from the target teachers is essential for the modification to be successful. The ultimate goal is that the ASC curriculum meets the needs of the teachers in other formal and informal settings in addressing state and national standards.

The evaluation strategies will include retrospective surveys and classroom observation protocols. The retrospective survey pre-test method assesses teachers pre- and post knowledge that is directly attributable to the professional development associated with the science content and the teaching strategies that are suggested in the ASC curriculum. (Campbell and Stanley, 1963; Siegel & Yates, 2007) The teacher observation rubric translates the science and practice standards into observable behaviors. It will be used as a guide for teacher self-reflection as well as the basis of the classroom observation protocol. The evaluation methodology combines a strategy of teacher self-reflection with confirmation from an outside observer.  For example, the classroom observation protocol observes if a teacher “Uses themes in astrobiology to teach complex thinking and challenges students with current issues and investigations from developing scientific areas,” an observable behavior based on science standards and project outcomes,” a project goal and instrument item that is directly based on the AAAS science standards.

The evaluation strategies to measure student learning will be through pre- and post tests embedded in the teaching modules and reported by the teachers through the Collaboratory feedback. To evaluate students growth in the processes of science, a strategy of self reflection combined with outside observation is used the Hands-on Science Performance Assessment instrument. Students use the instrument to assess their own performance, and then the ASC team will compare their ratings to the ratings of the teacher.  The teacher and student engage in conversation based on this instrument in a strategy that provides evidence for the evaluation, feedback to the teacher, and guidance to the student in a manner consistent with a strategy that supports science education for minority students.

Research Supports the Design of ASC
The modules under development are based on recent research-based strategies designed to diminish achievement gaps and improve the participation of underrepresented groups in STEM activities. Our curriculum will be developed with the specific purpose of engaging minority students in scientific thought, helping to decrease the achievement gap between mainstream and non-mainstream students. Research supports the use of astrobiology as a framework for increasing science literacy (Astrobiology Design Project Team, 2002; Carrapiço, et al. 2001; Rodrigues & Carrapiço, 2005; Slater, 2006; Staley, 2003; Tang, 2005), and that cultural awareness is key for successful, engaging science teaching (Lee & Luykx, 2006; Aikenhead, 2001; Lynch, et al., 2005).
The design rationale has also been sculpted by key educational theorists such as John Bransford and his work on how people learn (Bransford, et al. 1999) and the backward design principles of Wiggins and McTighe (Wiggins & McTighe, 1998). To ensure that the modules meet the needs of the ethnically diverse populations the project is using the standards proposed by the CREDE project (Tharp, et al., 2003) that characterize the dynamics of learning in diverse cultures, Five Standards for Effective Pedagogy developed by the Center for Research on Education, Diversity and Excellence (CREDE). The evaluation of the modules through field-testing in both formal and informal settings informs the modification of the curriculum to better meet the goals of the project.