Journal of Personnel Evaluation in Education 11:139±165, 1997 # 1997 Kluwer Academic Publishers, Boston ± Manufactured in The Netherlands

Can State-Mandated Teacher Evaluation Ful®ll the Promise of School Improvement? Events in the Life of One School GLENDA G. LOFTON Assistant Professor, Department of Curriculum and Instruction, College of Education, Southeastern Louisiana University, Hammod, LA 70402 FLO HILL Principal, Livingston School District JOSEPH G. CLAUDET Assistant Professor of Educational Leadership, Texas Tech University

Abstract This study reports a chronology of events, related issues, and concerns from research in one school involved in a year-long study designed to pilot the use of a state-mandated teacher evaluation system as a tool for professional development. The study coincided with the ®rst year of implementation of a controversial, statewide, on-the-job evaluation of beginning and experienced teachers for the purpose of issuing or validating a renewable professional certi®cate. Analysis of teachers' professional development within the context of such a volatile, political setting resulted in the identi®cation of eight principles that can be used to guide future school improvement efforts grounded in classroom-based assessments of teaching and learning. The study documents the dif®culties of using policy-mandated teacher evaluations for school improvement and points out the importance of going beyond the assessment system itself to examine issues related to using that system to improve teaching and learning and to effect changes in the everyday life of the school.

Facilitating meaningful change and improvement at any level in schools is not an easy task (Fullan, 1993). In the context of a volatile political and emotional setting where teachers' previously granted lifetime certi®cates are at stake, the process becomes even more complex. What happens in the life of a school when confronted with an emotionally laden, mandated statewide teacher evaluation system? Can such a system become the stimulus for professional development and school improvement? This article shares observations and experiences gleaned from intensive work in one school during the ®rst year of implementation of a new, performance-based, certi®cation policy. It is an attempt to understand what happens in the everyday life of the school and if it is possible for a school to improve. Traditionally teacher evaluation systems have ful®lled two purposes: accountability and professional growth (Bridges, 1990; McLaughlin & Pfeiffer, 1988; Natriello, 1990). More recently teacher evaluation has become a cornerstone of educational reform initiatives and

140

G.G. LOFTON, F. HILL & J. CLAUDET

a vehicle for overall school improvement (Millman & Darling-Hammond, 1990). During the 1980s and 1990s a variety of states have developed and implemented comprehensive, on-the-job assessment programs for the purposes of beginning teacher certi®cation, induction and support for new teachers, career ladder placements, and more recently in Louisiana for the professional, renewable certi®cation of all (45,000) teachers (Ellett, Wren, Callender, Loup & Liu, 1996). Many of these programs and their accompanying instrumentation have been grounded in ®ndings from the process-product literature on effective teaching (Brophy, 1986, 1988; Porter & Brophy, 1988) and within their time represented state of the art assessment frameworks. The prevailing view of policymakers in these policy and implementation efforts has been that improving the quality of teaching is the key to improving the overall quality of education. While policymakers and citizens view teacher evaluation as playing a major role in promoting accountability and improving instruction, teachers protest that such appraisals actually work against these objectives by undermining their professional status, decreasing autonomy, and disregarding the complexity of the teaching and learning process (McLaughlin & Pfeiffer, 1988). These contrasting views have refueled the debate among educators about the role of formative versus summative evaluation procedures in improving educational outcomes, a distinction originally highlighted by Scriven (1967). Some, like Popham (1988), have argued that the two cannot coexist, while others, like Hunter (1988), believe that the two can work together to enhance teaching effectiveness. Ellett (1987) took the issue a step further when he sugggested that those charged with the summative evaluation of teachers are morally obligated to assist and support teachers through the formative evaluation process. Regardless of one's position on the purpose and form of teacher evaluation, the fact still remains that teacher evaluation systems, for the most part, have not lived up to their avowed promise to change and improve schools. This ®nding has raised new issues and opened up new lines of inquiry for framing and informing future evaluation efforts (Ellett, 1997). Nationally, there has been a concerted effort toward developing sound procedures for evaluating teachers and upgrading the overall quality of the evaluation process. Noteworthy among these are (1) the efforts of the National Board of Professional Teaching Standards in developing, piloting, and implementing assessment procedures, (2) the development and federal funding for activities of the Center for Research on Educational Accountability and Teacher Evaluation (CREATE) at Western Michigan University, (3) the publication of The Personnel Evaluation Standards (Stuf¯ebeam, 1988), and (4) analyses of the quality of evaluation systems conducted at the district and state levels (Ellett et al., 1996; Loup, Garland, Ellett & Rugutt, 1996). While efforts to upgrade the quality of instruments and procedures are essential, they are not enough. For teacher evaluation to make a difference in the everyday life of a school, inquiry must extend to newer understandings of (1) the nature of the teaching and learning process, (2) the implementation and maintenance of educational reform initiatives, (3) research ®ndings and theories about change and the change process, (4) studies of schools as complex social organizations, and (5) the school culture. The New Handbook of Teacher Evaluation (Millman & Darling-Hammond, (1990) and more recent works (Stronge, 1997) have explored these issues. Duke and Stiggins (1990) and Iwanicki

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

141

(1990), for example, discussed issues and procedures to be considered in making evaluation a means to professional growth and school improvement. However, those wishing to use summative evaluations in formative ways, for the most part, have not appreciated or utilized growing bodies of literature pointing to the importance of establishing structures and processes that lead to a climate and culture for meaningful change in schools (McLaughlin, 1990). Furthermore, there have been few intensive, school-based studies on implementing high-stakes evaluation systems and analyzing their impact on the day to day life of the school. Purpose The purpose of this year-long case study was to (1) pilot the use of a state-mandated teacher evaluation system as a tool for professional development and improvement at both the school and classroom level, (2) identify factors that serve to impede or facilitate the use of a summative evaluation system for formative evaluation purposes ( professional growth), and (3) understand events in the everyday life of the school associated with implementation of an externally imposed innovation (state-mandated assessment). Historical Context The System for Teaching and Learning Assessment and Review (STAR) (Ellett, Loup & Chauvin, 1989) was developed and implemented in the state of Louisiana as a basis for assessing the quality of teaching and learning in Louisiana's classrooms and to meet requirements of the Louisiana Teaching Internship (LTIP) and Statewide Teacher Evaluation (LTEP) Programs. Like many other policy-based initiatives, the programs were a major part of a new governor's educational reform package, the Children First Act enacted by the Louisiana legislature in 1988. The Act mandated comprehensive, on-thejob assessments of beginning teachers for the purpose of certi®cation and experienced teachers for the purpose of obtaining or validating a professional, renewable certi®cate. While both initiatives were linked to certi®cation, the avowed ultimate goal was to use the data for the enhancement of teaching and learning and school improvement. The STAR was grounded in ®fteen years of development in teacher assessment and evaluation systems in other states, the literature pertaining to effective teaching and learning, and in timely conceptions of teaching expertise and knowledge suggested by Berliner (1986, 1989) and Shulman (1986, 1987). More than a teacher-evaluation and licensing procedure, the STAR re¯ected a new generation of contextually based classroom assessments designed to capture the essence of teaching practices that enhance student learning, extending from the assessment of novice teachers to those that might be considered superior, expert, or master teachers (Ellett, 1990). The assessment framework consists of a total of twenty-two Teaching and Learning Components distributed across four performance dimensions (I, Preparation, Planning, and Evaluation; II, Classroom and Behavior Management; III, Learning Environment; and IV, Enhancement of Learning).

142

G.G. LOFTON, F. HILL & J. CLAUDET

The number of assessment indicators comprising the teaching and learning components varies. The assessment framework and organization of the STAR can be found in the appendix. At the heart of the system was the belief that a teacher assessment system should be part of an ongoing improvement process at the school and classroom level in which teachers re¯ect on their own practice (Schon, 1987) and work collaboratively with their principal, colleagues, and students to enhance teaching and learning. Meaningful, classroom-based learning was viewed as occurring in a context where both teacher and students are learners, engaged in re¯ection, interaction, and collaboration as each constructs his or her own understandings. If the STAR's potential for assisting teachers and administrators in making ongoing professional development part of the everyday life of the school was to be achieved, two major obstacles had to be overcome: (1) making a shift from the emphasis on the summative aspects of the evaluation to the formative aspects; and (even more challenging (2) overcoming the anxiety and open hostility of teachers, regarding the policy initiative, that was being fueled by external groups such as the state's two large teacher unions. In a state with a long history of lifetime certi®cation, even the 25 to 30 per cent increase in salary accompanying the new evaluation requirements did little to appease its opponents. To live up to its promise, the STAR, as part of a politically motivated, rapidly implemented assessment program, had to be ®ltered through a context of teachers' heightened personal concerns and considerable negativism. The case study described here documents the events in the life of one school during the ®rst year of the program's implementation. The heightened concern expressed by teachers within the school were not atypical of those echoed by other teachers state-wide. Eventually the intense controversy surrounding the evaluation program led a new governor, sympathetic to teachers' unions, to halt the program, and replace it with a new program that included local control for teacher assessment, and deleted removal of lifetime certi®cation from the Children First Act. Case Study Context The school district in which the study was conducted was unique in size, setting, and resources. Although small and rural, it surprisingly represents one of the wealthiest districts in Louisiana. The new superintendent was perceived as a dedicated innovator, committed to change and improvement. Both the superintendent and his instructional facilitator were extremely supportive of the school's involvement in the study, met with the principal, and accompanied project staff on the ®rst visit to the school. The elementary school in which the study occurred was likewise unique in size, setting, and resources. Nestled among scenic and picturesque hills, the school was situated in a remote area of the district and state. Because of its remote location, the school had a population of approximately 180 students, pre-K through grade 6, in contrast to the only other elementary school, which had approximately 900 students. The student population was relatively stable although many students had only limited experiences outside the immediate geographical area. Unlike the other elementary school in the district, which had

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

143

been actively involved in a nine-year statewide school improvement effort, this school, because of its limited accessibility, had very little external support. The principal appreciated the new opportunities afforded him and his staff and pointed out that he had implemented many of the principles of site-based management in his school before the concept evolved in the literature. Teachers, however, operated within a strong autonomy norm, and the history of collaboration with the intent of improving teaching and learning was very limited. Though outspoken in his resistance to the assessment instrument (STAR) as a summative evaluation tool, the principal had a good understanding of the teaching and learning model on which it was based and saw its potential for staff development and the improvement of teaching and learning. The school professional staff consisted of fourteen teachers, three itinerant teachers, and a number of teaching aides who assisted in every classroom. Class sizes were rather small, partially because of the student population, and partially because of the district's desire to maintain small class sizes at the primary level. Since many of the faculty drove long distances to teach at the school, the principal tried to keep after-school meetings to a minimum and provided released time for staff development and collaborative planning. Teachers in this school, like other teachers in the state, were extremely apprehensive about the new state-mandated teacher evaluation program and were adamant that their lifetime teaching certi®cates should not be at issue with implementation of the new legislative mandate. Case Study Procedures Both qualitative and quantitative procedures were used in this mixed-method study. Quantitative assessments conducted at the class level provided baseline data for analyzing the quality of teaching and learning and for implementing an ongoing improvement process at the school level. Throughout the improvement process, teachers' responses, perceptions, and beliefs were recorded and qualitatively analyzed to provide insight into factors facilitating and impeding use of the assessment system as a means for professional growth and improvement. To provide a framework for analyzing and discussing quantitative and qualitative data, a collaborative, re¯ective professional growth model was designed. the model included preparation, implementation, and integration activities and encouraged teachers to (1) think systematically about their practice and its impact on student learning, (2) add to their repertoire of proven practices by trying out new techniques, (3) collaborate with others, and (4) learn from these experiences. To support teachers in implementation of this model, a number of staff development resources were designed including inservice modules on conducting professional development conferences, a professional development Ideabook for designing in-building, classroom-based, learner-focused professional growth activities, and a module on developing students' thinking skills (the area identi®ed by a pilot study of 6,000 teachers as the greatest need statewide) (Lofton, Ellett, Hill, Claudet, Chauvin & Loup, 1991a; 1991b; 1991c).

144

G.G. LOFTON, F. HILL & J. CLAUDET

Three members of a university-based research and development team served as researchers and facilitators in implementing and monitoring the model for change and improvement. Two members worked with the school on a regular basis and one periodically. One had extensive experience in facilitating change and staff development based on participation in a nine-year, statewide school improvement project, and the two other staff members had some thirty years of combined classroom teaching experience. All three had been involved in the development of the STAR system and the training of evaluators. Collectively the group provided approximately 250 hours of on-site assistance and support to the school and collected data on the formative use of the evaluation system and its impact on the everyday life of the school. The principal and an administrative intern within the school also assisted in collecting information and feedback from the staff. Quantitative and qualitative data collected by the researchers included (1) fall and spring STAR assessments of classroom teaching/learning and learning environments, (2) teacher lesson plans, (3) teacher personal journals containing written entries cataloguing perspectives of the process throughout the study, (4) assessors' classroom observation notes, (5) interview notes from lesson plan conferences, and (6) assessors' written notes from postobservation conferences, interviews, and videotaped teaching segments. Interviews were typically semistructured, using the results of STAR assessments, teacher lesson plans, and personal journals to frame interactions and dialogue. Data was ®rst synthesized in written narratives by each member of the research team. Weekly meetings were held by two members of the team, and the third member participated monthly. Members met to share and triangulate core observations, concerns, and trends in the data throughout the three phases of the improvement modelÐ preparation, implementation, and integration. Dialogue focused on the following: *

* * * * * * *

Baseline and subsequent observations of the quality of teaching and learning individually and collectively, Strategies for facilitating growth and improvement among emerging teacher types, Critical events indicative of change during each of the three phases, Changes in classroom teaching and learning activities, Changes in collaborative, re¯ective practices of teachers, Changes in school culture, Analysis of what works, what doesn't work, and why, and Formulation of conclusions or key principles to guide future improvement efforts.

Change Activities In keeping with the collaborative, re¯ective improvement model on which the study was based, three phases of change activities were provided and monitored: (1) preparation activities that encouraged teachers to think systematically about their practice and its impact on student learning, (2) implementation activities that helped teachers add to their repertoire of proven practices by trying out new techniques in collaboration with others, and (3) integration activities that were designed to help teachers re¯ect on and learn from

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

145

these experiences and use this knowledge in seeking new and better ways of enhancing the learning of students. Consideration was given to creating a context for change during all phases. This section highlights extended activities resulting from work with teachers and a brief synopsis of observations associated with each phase. Preparation Activities Preparation activities included the following: * *

*

*

An initial meeting with the faculty to explain the project and answer questions, Individual teacher assessments with the new observation and assessment system (STAR) to obtain baseline data for generating individual and school improvement plans and a brief follow-up conference to collaboratively identify strengths and target areas for improvement, Meeting to explain and distribute individual and schoolwide assessment pro®les and to prepare teachers to their role in collaborative, re¯ective conferences, and Individual conferences with teachers to review assessment pro®les and develop improvement plans.

Consistent with statewide pilot assessment results, the following STAR Teaching and Learning Components surfaced as common areas of need schoolwide: lesson initiation, content accuracy and emphasis, monitoring and feedback, and thinking skills. Because staff development was already scheduled to address ways to better enhance students' development of higher-order thinking skills, most teachers targeted lesson initiation and the related component of content accuracy and emphasis in their individual improvement plans. Plans of action for professional growth typically included reading or reviewing an assessment component, planning and implementing related teaching and learning activities, re¯ecting on what was learned, and sharing results. Implementation Activities Implementation activities began with the development of loosely structured improvement plans for individual teachers and an eight-hour staff development series on the enhancement of students' thinking skills. The remainder of the school year was devoted primarily to the thinking skills component, but teachers were able to experiment with a variety of professional development activities in the process. Including the following: *

*

*

Participation in the staff development session on thinking skills (two hours of released time provided weekly for a period of four weeks), Informal one-on-one meeting with each teacher to followup on staff development sessions, Small-group meetings during teachers' planning periods to share ideas, successes, and plan collaboratively,

146

*

*

* *

*

*

G.G. LOFTON, F. HILL & J. CLAUDET

Classroom observations of a thinking skills lesson with feedback and coaching (voluntarily engaged in by one-third of the teachers), Team teaching of thinking skills lessons ( participated in primarily by pre-K teachers accustomed to teaming and collaborative planning districtwide), Observing colleagues or teaching a colleague's class, Videotaping lessons and assessing them using the observation and assessment system (voluntarily done by three teachers with the encouragement of the principal), Using students as a resource to enhance teaching and learning (collaboration with students to solicit feedback on what helped them learn and what let them know that the teacher cared about them and their learning), and Using a personal log as a tool for re¯ecting, building understanding, and venting frustrations.

As the school year drew to an end, project staff began to see some evidence that collaborative, re¯ective behaviors were becoming integrated into the daily life of the school. For example, when the case study ®rst began, observation notes made by members of the research team indicated little discussion among teachers about teaching and learning. As the school year progressed, however, teaching and learning were often the object of conversation among teachers on the playground and in the lunchroom. Integration Activities Activities designed to foster integration of the process included * * * *

Postassessments to assess and analyze growth over the school year, End-of-year conferences with individual teachers, A ®nal meeting with all teachers to assess outcomes and get feedback, and A ®nal meeting with the principal and the school planning team to discuss how results might be integrated into the school's improvement plan for the following year.

STAR postassessment results revealed that teachers were beginning to integrate many of the concepts and principles that were the basis of the assessment system into their teaching. Particularly noteworthy was the emphasis on the integration of students' prior knowledge and the use of graphic organizers to help students see relationships among essential elements of the content. In the postassessment conferences and the ®nal staff meeting almost every teacher re¯ected on and shared successful thinking skills activities such as (1) second-graders assessing their progress in physical education using a performance checklist, (2) ®rst-grade money changers using critical analysis and problem solving as they developed concepts about money, (3) ®fth-graders building on prior associations as they developed new vocabulary, (4) sixth-graders developing a critical attribute chart on energy and formulating hypotheses about which soft drink would stay cooler in summer (cola or uncola?), (5) kindergartners' use of clothing and other concrete objects to develop the concept of seasons, (6) a class of four-year-olds verbalizing

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

147

principles to explain why some objects ¯oat and some do not, and (7) a kindergartner giving speci®c feedback to peers in a cooperative learning task. Even though classroom assessment was part of a state mandate that targeted summative evaluation decisions (teacher certi®cation), there was evidence of teachers' beginning to use elements of the STAR assessment system for more formative evaluation purposes ( professional development). For example, a common statement among teachers who really had worked to incorporate the diagnostic information from the assessments into the everyday life of the school was, ``It really makes a difference for students.'' One commented, ``Can you just picture what these ®rst-graders will be able to do in the sixth grade if all of us continue to emphasize thinking skills?'' They were developing the vision. ``But,'' they quickly added with a smile, ``we still don't want to be evaluated.''

Observations and Voices from the Field From the multitude of data recorded and synthesized by the research team, key events were identi®ed that most poignantly tell the school's story. These events relate to facilitating change, building understanding of the teaching and learning process, enhancing the learning of teachers and students, and changing school culture. Facilitating Change Personal concerns (How does this affect me?) are a major obstacle to be overcome in response to adapting or adopting any new innovation (Hord, Rutherford, Hurling-Austin & Hall, 1987). This was especially the case with the state-mandated evaluation system in Louisiana. As would be expected, teachers in this school had many personal concerns. They were not receptive to an externally imposed teacher evaluation system and the possibility of having their lifetime teaching certi®cates called into question. Normal resistance to change was further compounded by teachers' rather limited understanding of the new assessment system and the conceptual teaching and learning framework on which it was based and by considerable misinformation and rumor. In the initial meeting with teachers concerning their involvement in the study, most remained quiet. The few questions that surfaced focused primarily on personal concerns such as ``extra effort'' and additional demands that might be placed on them. One teacher, Mrs. D, did remain after the meeting, however, to voice her concerns about the assessment system. Her facial expressions and voice revealed that the issue was a highly emotional one for her. The new assessment program for her was viewed as a certi®cation and licensure procedure, a staged ``dog and pony show,'' rather than as a process with potential to enhance the quality of everyday teaching and learning. Like many other teachers in the state, Mrs. D harbored frustration at the lack of information about the new teacher evaluation program and assessment process. While assessors had received seven days of intensive training on the assessment system, teachers had only received a six-hour orientation program and felt ill-prepared to fully grasp and

148

G.G. LOFTON, F. HILL & J. CLAUDET

cope with the wealth of information about teaching and learning practices contained in the new, comprehensive assessment system. Mrs. D saw each assessment indicator used to operationalize the larger components of teaching and learning as a separate entity to be addressed. When the research team pointed out the relationship among the ideas and how a great many indicators could be integrated and addressed simultaneously, she voiced her concern that there was a great deal of variation among assessors in their understanding and that some were just listening for ``key words'' embodied in the script rather than observing teaching and learning processes. Information alone was not enough to alleviate Mrs. D's concerns. It was not until she planned, implemented and videotaped a lesson designed to develop thinking skills that she began to see how many components and indicators of those components could be addressed by a single activity. (Mrs. D will be revisited throughout this article to demonstrate a teacher in transition from considerable negativism about a new innovation to seeing its possibilities for enhancing teaching and learning.) The principal of the school proved to be an ``inside strategic champion'' (Mazzoni, 1991) in creating a context for change. He not only critiqued the video with this teacher (Mrs. D), but he also personally assessed and held conferences with each teacher in the school. He assisted in scheduling, coordinating, and monitoring all activities. An excellent climate watcher, he provided objective and constructive feedback to project staff on teachers' responses to the various activities and intervened when necessary to maintain a positive learning environment in the school.

Building Understanding of the Teaching and Learning Process The interactive process of teaching and learning on which the STAR assessment system was based focused on teachers and students as learners (Ellett, 1990). For most teachers in the school who saw themselves as dispensers of knowledge, this view represented a new set of conceptions about the role of the teacher in the teaching and learning process. Just as students had been conditioned not to think for themselves, some teachers had become dependent on textbooks and other resources and wanted a ``prescription'' for teaching and learning. They repeatedly requested, ``Just tell me what to do.'' Most desired speci®c activities for their content area and grade level so that they could go back to the classroom and use them with their students. A ®rst-grade teacher who was selected for assessment during the initial year captured the essence of these new conceptions best in one of her log entries. After describing the frustrations of not having someone to ``just tell me what to do and how to do it,'' she described a conference with the principal that was critical to her growth and understanding: Conferenced with principal. Told him my examples and ideas to see if I was on the right track. His comments and questions led me to think a little differently, try a different technique or approach. He has a way of helping without giving you a direct answer or solution. Gives you enough to help you solve your own questions. Now I see this is what it's all about. I have to think critically, solve problems, develop concepts of my

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

149

own, or I won't grow and improve as a teacher. There are no set answers, examples to show. There are many, many ways and techniques to use.

Enhancing the Learning of Teachers Consistent with the ®ndings of Berliner (1986, 1989), teachers in the school demonstrated varying levels of content and pedagogical knowledge and expertise. These differences were obvious in comparing classroom observations of teachers and teachers' response to other activities. Some teachers had a repertoire of techniques readily available. In a manner that appeared almost effortless, these teachers skillfully integrated and applied principles of learning and made adjustments as needed based on the responses of students. By contrast, the novice or less skilled teachers appeared somewhat awkward and stilted in the classroom and demonstrated less ¯exibility in responding to students, perhaps because they lacked a deep and rich understanding of teaching and learning as highly integrated processes. This seemed evident particularly by teachers who referred to the classroom assessment process as simply a ``dog and pony show.'' In re¯ecting on syntheses of the observation data for the fourteen teachers in the school, and after considerable discussion, members of the research team identi®ed four teacher types. These teachers were similar not only in their initial and ®nal scores on STAR assessments but also in the way lessons were structured including content, emphasis on higher-level thinking skills, student interest and engagement, teacher-student interactions, and student-student interactions. They also reacted similarly to the observations, assessment, and professional growth activities. Interestingly, none of the four teacher types are representative of high levels of teaching expertise as described by Berliner (1989). These teacher types are described metaphorically as follows: Teacher Type I: The activity director. This teacher was student-centered, creative, innovative, and willing to expand a great deal of personal time and energy to improve teaching and learning. Lessons, however, were described as activities for activities sake. Activities, though content related and interesting to students, were not clearly focused on essential content elements or lacked substantive content. Students were actively involved, but there were missed opportunities to enhance students' thinking and subsequent learning. Often, this type of teacher had only a super®cial understanding of key teaching and learning components re¯ected in the assessment system (STAR) and tended to rationalize or blame any problem areas on the assessment system, time elements, or some external factor. As teachers became more knowledgeable of the content and expectations relative to components of the assessment system, they tended to respond more positively and actively sought additional ideas and resources. Excellent teaching and learning activities often resulted. Teacher Type II: The content specialist. This teacher, like the Type I teacher, had many strengths. In contrast to Teacher Type I, this teacher's lessons were characterized

150

G.G. LOFTON, F. HILL & J. CLAUDET

by a strong content focus but limited student interaction and involvement. Well-structured lessons were comprised of activities that just scratched the surface. Students were not challenged to dig deeper, resulting in minimal involvement and littler higher-level thinking (analysis, synthesis, evaluation). This teacher type seemed very sincere and eager for new ideas and suggestions for involving students. With minimal support, the content specialist created lessons that were content and learner focused. Teacher Type III: The manager. In this teacher's classroom, maintaining a quiet and well-behaved environment took precedence over learning experiences that required more active involvement and interaction. Lessons focused on attending activities rather than processing activities. Students were attentive, completed assigned tasks, responded brie¯y (but with correct answers) when called upon, and appeared to have mastered the objectives. Objectives and activities, however, placed minimal cognitive demands on students and provided limited opportunities for students to think at higher levels. Teachers were pleased with the lessons as long as students were paying attention and completed ``exercises'' showing they had achieved the objectives. This teacher type was conscientious and willing to try new things but had dif®culty generating ideas. Teacher Type IV: The automaton. The Type IV teacher planned traditional activities aimed at covering the content. This teacher type seemed programmed to the curriculum and text. The content was often too dif®cult and lacked personal meaning or relevance for students. For this teacher, lessons ranged from well structured to poorly structured, but students' individual needs were not accommodated. Although most students were attending in class, they appeared to be only passively involved in learning. Interactions between teachers and students were minimal, and interactions among students were almost nonexistent. The teacher seemed to care about students but clearly lacked knowledge of pedagogy. This teacher type typically thought the lessons had gone well even though nothing had been done to monitor student learning processes and outcomes. Attending was equated with learning. Despite resolutions to experiment with ways for accommodating students' interests and needs, this teacher type quickly returned to the status quo. Assessment results on the new state-mandated evaluation for these teachers were extremely low, and for some demoralizing. Teachers in this group were the only teachers that did not improve from the pre- to postassessment. Enhancing the Learning of Students: Observations on Change Understanding of change in the school required observations of changes in students' learning as well as in teachers' attempts to enhance students' learning using results of the assessment process. Important indicators of change were also included in teachers' personal journals. A memorable journal entry came from a prekindergarten teacher: ``It is now four o'clock in the morning. I'm awake. You'd be awake too if you were having to teach a lesson developing the higher-level thinking skills of four-year-olds.'' This teacher later, however, expressed amazement at the ability of four-year-olds to actively engage in

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

151

inductively developing reasons why some objects ¯oat and some do not. While isolated events and re¯ections such as this occurred throughout the study, the real insight about the potential impact of the teaching and learning components on students' learning came during the last week of the school year. Mrs. D, the same teacher who had remained after the initial meeting to voice her concerns about the assessment system, stayed after the ®nal meeting to debrief and talk about her personal experiences. For the ®rst time, she appeared con®dent and secure in the knowledge of the assessment system. The researchers lamented that her ®nal lesson on wind, judged as superior by the assessor, was not on videotape. She offered to plan another lesson for taping, but it was the last week of school, and the research team felt that would impose a burden on her. Then it was suggested that she teach the same lesson to another second-grade class. This class was also studying weather and had not yet had the lesson on wind. She knew all the children; it seemed like the perfect solution. The research team would have a video of an almost perfect lesson, and it would not involve additional planning by the teacher. Two days later, with cameras rolling, Mrs. D retaught the same lesson to her colleague's second-grade students. The activities were the same: students were asked to visualize, to relate examples from their personal experiences, to use prior knowledge in developing a concept map, to develop principles and generalizations from concrete, relevant, and meaningful experiments, to discuss ®ndings, and to work in cooperative groups. Though there was an attempt to replicate an effective lesson, there were some surprises. For example, and in the words of Mrs. D, ``The lesson was a disaster.'' The students were mostly nonresponsive throughout the lesson, and Mrs. D did not attempt very often to probe or rephrase questions. When she did try to do so, the students seemed to become even more withdrawn from the lesson. For example, when asked to brainstorm ideas for the concept map, only simple associations were generated by students. When placed in cooperative groups designed to complete a common task by working together, students worked independently. Some students even chose to hide their work from other students. The research team surmised that the students were uncomfortable with a different teacher or perhaps the video camera, but Mrs. D knew better. With tears in her eyes, she discussed the lesson with the research team: ``I've seen these same responses, these same blank expressions before. They were just like my class at the beginning of the school year!'' For the ®rst time Mrs. D fully understood the interactive nature of the teaching and learning process and the potential of the assessment system to enhance students' thinking and learning. Mrs. D learned that the quality of teaching and learning is more than just the teacher delivering the lesson. Establishing the context to support quality interaction and teaching and learning processes includes more than knowledge of content and the implementation of activities. Clearly, in this instance, students did not understand their roles as learners and their relationship to Mrs. D as a teacher. These understandings of the nature of teaching and learning are clearly re¯ected in the conceptual framework of the assessment system. This framework had been explained to Mrs. D before. However, it was not until this critical incident occurred that Mrs. D began to internalize and value the formative, professional development elements that were integrated within the new state assessment system.

152

G.G. LOFTON, F. HILL & J. CLAUDET

Mrs. D then began to share with the research team the ``secret'' things she had done throughout the school year to help students understand their roles in the teaching and learning process. She confessed that although her initial motive was to get students to help her do well on the summative (certi®cation) evaluation, the outcomes had been much greater. She began to prepare her students by bringing in articles from the newspaper about the assessment system. She discussed with students changes she was trying to make in her classroom. She showed students a copy of the STAR assessment manual that gave detailed descriptions of each teaching and learning component and each assessment indicator. She explained to students that this ``book told what teachers and students were expected to do.'' She actually read them several assessment indicators each day. (See the appendix for a sample of two assessment indicators and their accompanying annotations that were read and explained to the students.) The children were totally fascinated, often asking her to read the items over again. ``Do you feel this is important?'' she would ask, and together they analyzed why. She also helped students to understand the vocabulary in the assessment manual so that they understood what they were expected to do. For example, several of the assessment indicators related to using questioning to enhance thinking. Mrs. D had students analyze questions that made them think and dig deeper. For example, ``What kind of question would you have to ask to make your friend talk to you?'' she asked. Would you say, ``What color is the girl's hair, or would you ask your friend to describe the girl's hair?'' The children began to develop their own questions and to question each other. She explained wait time and probing for the basis of misunderstanding. Prior to understanding about probing, students often acted frightened and afraid to answer (not unlike the class described above that Mrs. D tried to teach). They thought they were being reprimanded or that the answer was wrong; no one had ever probed about right answers before. (That was why, she explained, she had not probed more during the taping of the lesson. She saw again that look of fear in students' eyes.) Now, she said, her students were so accustomed to probing and elaborating on their ideas that they went right on without her. She credited ``wait time'' with being highly important in helping the students with low self-esteem become more actively engaged as learners. ``Just give me my time to think,'' they would say, ``and I'll come up with the right answer.'' Students even devised a red stop sign to hold up to indicate they were thinking and a green sign to indicate they were then ready to share. Students learned the value of learning with and from each other. Mrs. D recalled that prior to discussing collaboration with her students, they, like the children in the taped lesson, thought working together was cheating. When placed at round tables to complete an activity, they put up barriers to hide their papers. They learned about feedback and became quite good at giving speci®c and positive feedback to their peers. In mathematics and reading students even began to teach minilessons, approximately 10 minutes in duration. They planned with the teacher and then taught the lesson to the class. Only those students who accepted and ful®lled their role as student participant, elaborating and responding, could have their day as teacher. Students, she discovered, had so much information to share; it was important to give them opportunities to interact and share with each other. ``The pace of our schools is much

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

153

too fast,'' Mrs. D added. ``Students must have time to experience and share their thoughts. We need to know what they know and how they know. Testing must become ®ll my blank.'' She seemed to know her students as never before. ``At that ®rst thinking skills workshop,'' she reminded project staff, ``you challenged us to `dig deeper.' Now I'm obsessed with ®nding a way,'' she said, as tears again ®lled her eyes. ``Now I'm determined to dig a tunnel.'' As the research team left the school that day, it didn't have the perfect videotaped lesson for future use. What the team did have was a new appreciation for the interactive nature of teaching and learning and the clear importance of educating students in their role as active learners. The team returned to the school one more time, to meet with the principal and collaborate with Mrs. D on the development of another staff development module, ``Re¯ecting in Action'' (Lofton, Ellett, Hill, Claudet, Chauvin & Loup, 1992) aimed at helping other teachers learn with and from their students. The transformation of Mrs. D clearly indicates how the assessment system might be used to bring about changes in individual teachers and how it might be used to facilitate the development of students in their roles as learners. However, building a schoolwide culture to support professional growth and improvement requires the involvement of the entire school. Changing School Culture Fostering meaningful collaboration and re¯ection proved dif®cult given teachers' limited experiences with collaborative, re¯ective practice and time constraints within the school. Initially, teachers, like students, were uncomfortable with and resistant to learning activities requiring re¯ection and collaboration. Instead they wanted recipes and ``quick ®xes.'' One teacher commented, ``We had to generate all the ideas and do all the thinking. I don't know why the research team even came!'' Members of the research team responded to these concerns by guiding teachers through the re¯ective planning cycle incorporated in a professional development Ideabook (Lofton et al., 1991c), a resource book designed to assist teachers in an ongoing improvement process. Of all the activities, teachers consistently rated monthly, small-group sessions in which they had opportunities to discuss STAR components, ask questions, and share ideas with peers as the most valuable learning activity. When teachers requested that demonstration and modeling of desired teaching and learning activities become a part of these sessions, members of the research team were hesitant, fearing that these activities might encourage mimicry and dependence. Based on Joyce's (1990) belief that mimicry is the ®rst step in the adapting of any behavior, examples modeling speci®c teaching and learning components were added and proved to be extremely effective in helping teachers become re¯ective practitioners, analyzing what worked, and why. As teachers began to see the bene®ts of collaboration and re¯ection on their own learning, they requested more time for interacting and sharing with peers. However, they didn't want to give up their planning time, they didn't want to take the time away from their students, and they didn't want to stay after school in order to have that additional time.

154

G.G. LOFTON, F. HILL & J. CLAUDET

Collaborative, re¯ective conferences among teachers and assessors, using assessment data, likewise seemed to have excellent potential for enhancing the learning of teachers, but posed similar problems in regard to time. Time was required to help teachers understand their roles and to structure productive conferences. In working with teachers in structuring lessons to facilitate students' thinking and re¯ection, the research team discovered that the same principles applied in structuring conferences to facilitate the thinking and re¯ection of teachers. Good conferences like good lessons involve critical analysis, generating examples, engaging in problem solving and creative thinking, using mental imagery, developing concepts and principles, and extending learning to new contexts. All of these ideas were contained and explained in the new observation and assessment system. Using these principles as the basis of a framework for structuring the professional development conference to encourage re¯ective thinking proved extremely bene®cial for teachers because it built on their prior knowledge on developing higher-level thinking as out-lined in the assessment system, and it gave focus and direction to meaningful dialogue about learning. Despite the dif®culties in sharing observations, the research team agreed that the use of collaborative, re¯ective processes did result in change and improvement. For example, in one of the small-group planning sessions, teachers realized that they were teaching the same concepts and principles at different grade levels with little variation in the challenge or complexity of the learning tasks. They brainstormed ideas about how to integrate content throughout the curriculum and came up with a logical sequence of activities for enhancing learning and extending thinking across successive grade levels. Change and improvement were also re¯ected in the quantitative data from the spring STAR assessments. As collaborative, re¯ective practices become embedded in daily practice, the beliefs, norms, and values of the group began to change. Evidence of this was ®rst seen among preK teachers. They had been involved in collaborative planning at the district and school level for some time and were initially much more open to observation and suggestions from researchers than other faculty members. They observed and gave each other feedback and planned and evaluated lessons together. As a result, these teachers showed greater gains from STAR pre- to postassessments than other teachers who were not involved in collaboration and sharing. As an outgrowth of formal, structured activities, informal collaboration and sharing gradually emerged from all teachers. Playground and lunchroom talk increasingly became opportunities for collaboration and re¯ection on teaching and learning. However, changing the deep underlying structures and culture of schools is no easy task and takes time. Almost an entire year was spent in creating readiness for changeÐbuilding trust and establishing rapport, and understanding and seeing a need for incorporating the teaching and learning components included in the assessment system into the everyday life of the school. At the end of the year, one of the teachers commented, ``Can't you come back next year? We're just now ready to begin.''

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

155

Conclusions From an analysis of the data highlighted in this case study, members of the research team identi®ed eight key principles that can be used to guide future school improvement efforts grounded in classroom-based assessments of teaching and learning. A careful examination of these principles suggests that change and improvement are in¯uenced more by learning processes (for example, the role of prior knowledge) than they are by measurement and evaluation issues related to an assessment system. 1. Prior Knowledge and Development Seem to be Directly Related to Growth and Improvement The extent to which gains in teaching effectiveness (as measured by the assessment system and supported by informal methods) were evidenced at the end of a year of professional development seemed to be related to a teacher's (1) baseline level of professional knowledge, (2) observed level of effectiveness in the actual teaching and learning process, and (3) successful progression through the normal stages of concern associated with any change ( personal concerns, task concerns, and impact concerns) (Hord et al., 1987). 2. Individual Differences Should be Considered in Planning Meaningful Learning Activities Learning experiences must build on teachers' prior knowledge so that the information is personally meaningful and relevant. It is important to build on a teacher's strengths as well as areas of relative weakness. Likewise, activities should be planned that assist the teacher in moving through the stages of concern. It is only when personal concerns and task concerns have been alleviated that teachers can focus on the impact of new methods and techniques on the learning of students. Assessment pro®les, considered in light of the four teacher types previously identi®ed, provide data for planning and interventions. However, pre- and postassessments of inservice sessions, observations, sharing sessions, log entries, conferences, informal conversations, all become opportunities for listening and responding to individual needs. Even when teachers verbalize what they need, helping them to translate new learnings into the contexts in which they work is not an easy task. 3. Collaborative, Re¯ective Learning Experiences Enhance Learning and Create a Culture for Change and Improvement The assessment system proved an effective means for building a common knowledge base and initiating dialogue around a common language about teaching and learning. Through carefully structured activities during the school day, teachers came to value and see the bene®ts of re¯ection and collaboration for themselves and for their students. Particularly

156

G.G. LOFTON, F. HILL & J. CLAUDET

effective were (1) learning logs that encouraged personal re¯ection before, during, and following the implementation of teaching and learning activities; (2) small-group sessions in which teachers had opportunities to discuss, ask questions, share ideas with peers, and use demonstration lessons; and (3) videotaped teaching segments that modeled desired teaching and learning behaviors to generate discussions about what works, and why. Collaborative, re¯ective conferences using assessment data showed potential for facilitating growth and improvement. Time must be taken to help teachers and principals understand their roles and how to structure productive conferences. The principles used in structuring a lesson to facilitate students' thinking and re¯ection proved an effective framework for structuring the professional development conferences to facilitate teachers' thinking and re¯ection (Lofton et al., 1991b). The time required for formal opportunities for re¯ection and collaboration proved to be a major deterrent. As an outgrowth of formal activities, informal collaboration and sharing gradually emerged. However, changes in the deep underlying structures of schools take time. As noted here, one year is not enough.

4. Growth and Development Seem to be Directly Related to the Level of Individual Commitment and Amount of Engaged Time Gains from STAR pre- to postassessments re¯ected teachers' levels of personal commitment and willingness to spend additional time during and outside the school day. For a variety of reasons, very few teachers used personal time for pursuing schoolrelated, professional growth activities. It was interesting to discover that many teachers were willing to use personal time and exert a great deal of energy to take college courses, even those they characterized as boring and irrelevant. However, the motivation for attending college seemed to be to get a higher degree, to pursue additional career opportunities, and to receive a salary increment rather than engaging in a professional activity to improve the quality of teaching and learning in the everyday life of the school. While all teachers expressed a desire to improve, teachers, for the most part, viewed school-based professional development activities as ``extra work'' rather than a part of their ongoing responsibilities. One wrote in her log, ``I have enough to do just planning my daily lessons without all this extra work.'' This observation raises two essential questions: How do we balance work requirements and the need for professional development so teachers do not feel that professional development is extra work? How do we get teachers to view ongoing assessment and staff development as a professional responsibility? While professional development can be required, it begs the larger issue of building a school culture in which it is valued. As Fullan (1993) reminds us, ``You can't mandate what matters.''

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

157

5. Follow-up and Support, Balanced with Suf®cient Pressure to Improve, is Essential if Content is to be Internalized and Applied It is easy for the demands of everyday living to take precedence over professional improvement. Even when teachers were provided support in the form of released time, coaching, and scheduled follow-up by the principal and members of the research team, many teachers failed to complete assigned activities. Given this ®nding or insight, what is suf®cient and reasonable pressure to change? Many complained that the new assessment system put too much pressure on teachers; yet when given an opportunity to grow professionally without the threat of evaluation, many chose to ``worry about change and improvement when my name is drawn for assessment.'' Unfortunately, the promise of enhanced student learning was not viewed as a suf®cient reason for expending time and effort on professional development, given other job demands. 6. Focusing on Learning Motivates Growth and Improvement One of the most depressing yet enlightening ®ndings of the study was the realization that learning is not always the focus of teachers and students. For example, when teachers were asked to comment on a lesson, the most common response was, ``The children seemed to enjoy it.'' Likewise, students commonly focused on the interest of the activity rather than what was learned. Making a good grade or ``getting credit'' took precedence over learning. In the context of a mandated statewide teacher assessment program, it was easy for concern about scoring well on the assessment to take precedence over learning for teachers as well. On the initial performance assessment, monitoring of student learning and providing feedback about learning was an area of need school wide. As teachers (such as Mrs. D) really began to focus on evidence that learning was occurring in their classrooms, the motivation for professional growth and the additional time it required seemed assured. The single most important question for both teachers and students became, ``What did I learn today?'' As teachers individually and collaboratively re¯ected on learning (what worked and why), staff development became more deeply embedded in the everyday life and in teachers' sense of who we are and what we do around here. 7. Understanding One's Role in the Interactive Process of Teaching and Learning Builds Individual and Organizational Ef®cacy As evidenced by Mrs. D's classroom experiences, teacher and student ef®cacy seemed to be enhanced when each understood his or her role in the teaching and learning process and when the goal of learning became a shared adventure of re¯ection and collaboration, motivated by pursuit of the next learning challenge. The role of the teacher in this context is not only structuring the task and the talk to maximize the learning of each student but also helping students acquire the skills needed to explore, expand, and control their own

158

G.G. LOFTON, F. HILL & J. CLAUDET

learning. The challenge to the student is assuming a proactive role in monitoring and re¯ecting on personal learning. Students are valuable and untapped resources in enhancing teaching and learning. Mrs. D's work during the school year suggests the bene®ts of educating students about their roles and responsibilities in the learning process. It does not just happen. As noted earlier, a staff development module, Re¯ecting in Action was developed with Mrs. D's input to assist teachers in understanding and developing the students' role in the teaching and learning process.

8. Each Individual Must Build His or Her own Understanding Common statements from teachers particularly at the beginning of the year included such statements as, ``Just tell me what to do,'' or ``Just give me a list of thinking skills activities for my grade level and my content area.'' In the complex world of teaching and learning, however, there are no quick ®xes. Substantive changes that focus on the teacher as decision maker and re¯ective practitioner require time and energy. Support can be provided, but as the ®rst-grade teacher's log entry previously noted, each teacher must work through the process alone: Now, I see what it's all about. I have to think critically, solve problems, develop concepts on my own, or I won't grow and improve as a teacher. There are no set answers, examples to show. There are many, many ways and techniques to use.

Discussion and Implications The purpose of this study was to identify factors facilitating and impeding the use of a state-mandated teacher assessment program for the purposes of professional development and school improvement. Based on the data, eight principles were identi®ed and proffered as guidelines for future improvement efforts. Examination of these principles suggests that growth and improvement were in¯uenced more by factors in¯uencing learning in general than by traditional measurement and evaluation issues. For example, Principle 3, which notes the value of collaborative, re¯ective learning experiences, is consistent with the ®ndings of Slavin (1990) on cooperative learning. This in no way negates the importance of a valid and reliable assessment process and the use of resources such as The Personnel Evaluation Standards (Stuf¯ebeam, 1988) in upgrading the overall quality of personnel assessment and evaluation systems. What it does imply, however, is the need to go beyond the assessment system itself to examine issues related to using that system to improve teaching and learning and to effect changes in the everyday life of the school. The common theme of learning that permeates the eight conclusions is consistent with current views such as Fullan's (1993) that de®ne educational reform and change as continuous learning and acquiring the learning habit. The eight conclusion statements seem to apply

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

159

equally to teachers and students. In an assessment system originally designed to focus on teachers and students as learners, it is ®tting that the same principles seem to apply in both areas. In planning for learning or change, this study suggests that three interrelated factors must be considered: the individual, the task, and the context. Traditionally, in the use of large-scale teacher evaluation systems, implementers have focused primarily on the task and have largely ignored the literature on the individual, change, and context (Hord et al., 1987). It seems important to recognize that institutions do not change; people do. In the initial phases of this study, teacher's personal concerns and limited knowledge about the assessment system were serious deterrents to its use as a means for professional growth and improvement. Many of the concepts and principles on which the assessment system were based (Ellett, 1990) were new to teachers, and teachers were more concerned about ``passing the evaluation'' than using the assessment process and information to enhance teaching and learning. This study suggests an inverse relationship between the degree of personal concerns fueled by high-stakes evaluation (loss of one's teaching credentials) and the ability to bring about change. Learning cannot occur in an atmosphere of fear and misinformation. Once personal concerns were somewhat alleviated for teachers in this study, they began to focus on the task, understanding and applying the assessment system components and the interactive model of teaching and learning, to improve their classroom learning environments. If an evaluation system is to be used as a means for enhancing the quality of teaching and learning, it seems apparent that the primary focus of teacher evaluation systems must shift to (1) encompass both teaching and learning and the interaction between teacher and students and among students as well, (2) encourage re¯ective practice, re¯ecting ``on'' action and ``in'' action (McCleod, 1981; Schon, 1987; Yinger, 1977), (3) build on cognitive views of learning (Taylor, Dawson & Fraser, 1995; Tobin, 1993), (4) emphasize higher-level thinking (Jones, Palinscar, Ogle & Carr, 1987; Marzano, Brandt, Hughes, Jones, Presseisen, Rankin & Suhor, 1988), and (5) make assessment decisions based upon the responses of students and their learning, as well as teachers' knowledge of content, curriculum, and pedagogy (Ellett, 1990; Shulman, 1987). Distinctions between evaluations and assessments must also be made. According to Ellett (1997), evaluations occur when assessment data are compared to an established standard to make summative decisions such as annual teacher evaluations, certi®cation and licensing of teachers, and so on. In contrast, classroom-based assessments are not used to make decisions about individuals relative to an established standard. Assessment is viewed as an ongoing process designed to provide information to individuals for the purpose of continuous growth. Assessment is consistent with but not the same as models of formative evaluation. Unlike formative evaluation, the information does not have to be tied in any way to summative evaluation. Ellett refers to such assessments as ``thermometers that can be used to make inferences about the quality of teaching and learning and general life in classrooms.'' As such they can provide valuable feedback to teachers in the quest for continuous improvement. However, as shown here, explaining these conceptions to teachers, particularly in a high-stakes evaluation environment, in a manner that in¯uences meaningful professional growth, is not easy.

160

G.G. LOFTON, F. HILL & J. CLAUDET

The effectiveness of any new assessment system to bring about change and improvement directly depends upon the development and rich understanding of its philosophy, procedures, and the underlying principles of teaching and learning on which it is based. When teachers achieve these deeper understandings, they are more willing to exert the extra effort needed for continuous professional development. This ®nding has been supported in other reform efforts where the goal is implementation and maintenance of an ongoing improvement process (Lofton, Ellett, Hill & Rugutt, 1996). Even when teachers understand and want to improve, achieving a balance between pressure and support to change is important (Mann, 1983). Expectations for ongoing professional development and use of the assessment process to enhance teaching and learning should be clearly communicated. However, support in the form of released time, additional training, resources, modeling, collaboration, coaching, and mentoring should be provided to assist teachers in applying new knowledge and skills (Joyce, 1990: Rodrigues & Johnstone, 1986). As new knowledge and skills become integrated into the teachers' daily practices, their focus seems to shift from managing the task to observing and analyzing impact on student learning. In this study the application of the assessment system and integration of assessment data directly into classroom activities made observable differences in the involvement and learning of students, and encouraged teachers to invest more time and effort in planning and providing meaningful teaching and learning activities. As found in previous studies, the perceived bene®ts of an improvement effort, particularly in relation to student learning, became a powerful motivator and sustainer of change (Lofton, Ellett, Hill & Chauvin, 1997; McLaughlin & Marsh, 1978). A ®nding of major signi®cance in this study was the importance of educating students on their roles in the teaching and learning process. Students appear to be a valuable untapped resource in the improvement process. Building of both teacher and student ef®cacy ( persistence in the face of obstacles and barriers to success) seems to be an important element related to confronting new challenges and viewing these challenges as continuous learning opportunities (Bandura, 1977, 1982). In this study, ef®cacy seemed enhanced when teachers and students worked in the spirit of collaboration and learned the importance of personal re¯ection on teaching and learning. If assessment systems are to foster continuous learning (Fullan, 1993; Joyce, Wolf & Calhoun, 1993) resulting in greater individual and collective ef®cacy (Bandura, 1977, 1982), both collaborative and re¯ective processes and structures must become imbedded in the school culture. Given the traditional strong norms of teacher autonomy in schools, many teachers are not accustomed to collaboration and re¯ection. Collaboration associated with site-based management and restructuring is not necessarily synonymous with systematic collaboration for the purpose of enhancing teaching and learning. Procedures alone such as developing plans for improvement and integrating these with steps in a teacher evaluation cycle as proposed by Iwanicki (1990), and others, while perhaps useful, are clearly not suf®cient to achieve these purposes. If teacher evaluation is to be used for school improvement, what seems needed, as suggested by Wheatley (1992) is to build cohesive, process-oriented school organizations and cultures that foster individuality and community.

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

161

In conclusion, for assessment and evaluation systems to become vehicles for continuous learning, three things must be considered in planning and decision making: the individual, the task, and the context. In an environment deliberately structured to foster collaboration and re¯ection, researchers, administrators, teachers, and students learned that quality teaching is not following a static script. Instead, it re¯ects (1) building an understanding about the learner, (2) adapting to preexisting knowledge, (3) making spontaneous decisions about how to verbally mediate learning, (4) questioning, listening, and providing substantive feedback to help individuals construct new understandings, and (5) re¯ecting on what learners already know and their own learning. When all role groups are viewed as learners and engage in genuine dialogue (Senge, 1990) about the enhancement of teaching and learning, individual and collective ef®cacy seem enhanced, and a collaborative, re¯ective work culture open to seeking better ways is developed (Fullan, 1993). The study reported here documents transitions in learning in one school. These transitions were associated with implementation of a state-mandated, summative evaluation system for certi®cation. The use of the system for more formative evaluation and assessment purposes ( professional development) was highlighted in view of individual and organizational change processes. Selected case data from teachers and students was used to illustrate factors facilitating and impeding individual change and the cultural supports necessary to sustain meaningful change and continuous learning. Considered collectively, the results of the study document the dif®culties of using policymandated teacher evaluations for school improvement. As Fullan (1993) reminds us, what matters in schools cannot be mandated. Neither can meaningful change. Meaningful change as re¯ected in this study requires attention to context, time, new learning, and individual and cultural change. These factors are highly important concerns if statemandated teacher evaluation programs, and district administered teacher evaluation efforts as well, are to ful®ll their promise of improving the quality of teaching and learning in schools. Accomplishing formative goals (those concerned with assessments for professional development and continuous improvement in teaching and learning), as shown here, is no easy task.

Appendix: STAR (System for Teaching and learning Assessment and Review) System Framework, 1990±1991 STAR: System for Teaching and learning Assessment and Review: Teaching and Learning Components (TLCs) Performance Dimension I: Preparation, Planning, and Evaluation (26)a A. B. C. D. E. F.

Goals and objectives (4)b Teaching methods and learning tasks (4) Allocated time and content coverage (4) Aids and materials (4) Home learning (3) Formal assessment and evaluation (7)

162

G.G. LOFTON, F. HILL & J. CLAUDET

Performance Dimension II: Classroom and Behavior Management (23) A. B. C. D. E.

Time (6) Classroom routines (4) Student engagement (1) Managing task-related behavior (6) Monitoring and maintaining student behavior (6)

Performance Dimension III: Learning Environment (13) A. Psychosocial learning environment (10) B. Physical learning environment (3) Performance Dimension IV: Enhancement of Learning (55) A. B. C. D. E. F. G. H. I.

Lesson and activities initiation (8) Teaching methods and learning tasks (6) Aids and materials (6) Content accuracy and emphasis (6) Thinking skills (11) Clari®cation (4) Monitoring learning tasks and informal assessment (6) Feedback (4) Oral and written communication (4)

STAR: Teaching and Learning Component 11A: Timec Comments. Teaching and learning activities reasonably re¯ect allocated time, begin promptly, proceed ef®ciently with smooth transitions and no undesirable digressions and allow for maximum opportunities for student engagement in learning. Activity refers to all things teachers and students do in the classroom.

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

163

Assessment Indicators

Annotation

II.A.1 Learning activities begin promptly

This indicator focuses on the beginning of the lesson. Learning activities begin with little time spent on organizational activities such as roll taking and the distribution of materials and supplies. The ef®ciency with which organizational activities are handled is always a concern. IF A SIGNIFICANT AMOUNT OF TIME IS WASTED AT THE BEGINNING OF THE LESSON, THE INITIAL USE OF TIME IS UNACCEPTABLE.

II.A.2 Expectations for maintaining and completing timelines for tasks are communicated to students

As initial tasks begin and as tasks change throughout the lesson, the teacher clearly communicates to students when tasks are to be completed. Cautions about the persistence needed to complete tasks on time are elements of effective communication of expectations. IF EXPECTATIONS ARE NOT ADEQUATELY COMMUNICATED TO STUDENTS, THE USE OF TIME AVAILABLE FOR LEARNING IS UNACCEPTABLE.

a. Number of assessment indicators comprising performance dimension. b. Number of assessment indicators comprising teaching and learning component. c. Only two of six assessment indicators are shown here for the STAR TLC component of TIME.

References Bandura, A. (1977). Self-ef®cacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191±215. Bandura. (1982). Self-ef®cacy mechanism in human agency. American Psychologist, 37, 122±147. Berliner, D. (1986). In pursuit of the expert pedagogue, Educational Researcher, 15(7), 5±13. Berliner, D. (1989). Implications of studies of expertise in pedagogy for teacher education and evaluation. New directions for teacher assessment: Proceedings of the 1988 ETS Invitational Conference (pp. 39±65). Princeton, NJ: Educational Testing Service. Bridges, E. M. (1990). Evaluation for tenure and dismissal. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher evaluation: Assessing elementary and secondary school teachers (pp. 147±157). Newbury Park, CA: Sage. Brooks, J. & Brooks, M. (1993). In search of understanding: The case for constructivist classrooms. Alexandria, VA: Association for Supervision and Curriculum Development. Brophy, J. (1986). Teacher in¯uences on student achievement. American Psychologist, 41(10) (October), 1069±1077. Brophy, J. (1988). Research on teacher effects: Uses and abuses. Elementary School Journal, 89(1), 3±22. Duke, D. I., & Stiggins, R. J. (1990). Beyond minimum competence: Evaluation for professional development. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher education: Assessing elementary and secondary school teachers (pp. 116±132). Newbury Park, CA: Sage. Ellett, C. D. (1987). Emerging teacher performance assessment practices: Implications for the instructional supervision roles of school principals. In William Green®eld (Ed.), Instructional leadership: Concepts and controversies (pp. 302±327). Boston: Allyn & Bacon.

164

G.G. LOFTON, F. HILL & J. CLAUDET

Ellett, C. D. (1990). A new generation of classroom-based assessments of teaching and learning: Concepts, issues and controversies from pilots of the Louisiana STAR. Baton Rouge, LA: Louisiana Teaching Internship and Statewide Teacher Evaluation Projects, College of Education, Louisiana State University. Ellett, C. D. (1997). Classroom-based assessments of teaching and learning. In J. Stronge (Ed.), Evaluating teaching: a guide to current thinking and best practice. (pp. 107±128). Thousand Oaks, CA: Corwin Press. Ellett, C. D., Loup, K. K., & Chauvin, S. W. (1989). System for Teaching and learning Assessment and Review (STAR). Baton Rouge: Louisiana Department of Education, College of Education, Louisiana State University. Ellett, C. D., Wren, C. Y., Callender, K. E., Loup, K. S., & Liu, X. (1996). Looking backwards with the Personnel Evaluation Standards: An analysis of the development and implementation of a statewide teacher assessment program. Studies in Educational Evaluation, 22(1), 79±113. Fullan, M. G. (1993). Change forces: Probing the depths of educational reform. Bristol, PA: Falmer Press. Hall, G. E., & Hord, S. M. (1987). Change in schools: Facilitating the process. New York: State University of New York Press. Hord, S. M., Rutherford, W. L., Hurling-Austin, L., & Hall, G. (1987). Taking charge of change. Alexandria, VA: Association for Supervision and Curriculum Development. Hunter, M. (1988). Re¯ecting a reconciliation between supervision and evaluation: A reply to Popham. Journal of Personnel Evaluation in Education, 1, 275±280. Iwanicki, E. F. (1990). Teacher evaluation for school improvement. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher education: Assessing elementary and secondary school teachers (pp. 158±171). Newbury Park, CA: Sage. Jones, B., Palinscar, A., Ogle, D., & Car, E. (Eds.). (1987). Strategic teaching and learning: Cognitive instruction in the content areas. Alexandria, VA: Association of Supervision and Curriculum Development. Joyce, B. (1990). Changing school culture through staff development. Alexandria, VA: Association for Supervision and Curriculum Development. Joyce, B., Wolf, J., & Calhoun. (1993). The self-renewing school. Alexandria, VA: Association for Supervision and Curriculum Development. Lofton, G., Ellett, C., Hill, F., & Chauvin, S. (1995). The role of the school district in implementing an ongoing improvement process. Paper presented at the Southwest Education Research Association. Lofton, G., Ellett, C., Hill, F., Claudet, J., Chauvin, S., & Loup, K. (1991a). Learning to think: thinking to learn. Baton Rouge: Louisiana Department of Education, College of Education, Louisiana State University. Lofton, G., Ellett, C., Hill, F., Claudet, J., Chauvin, S., & Loup, K. (1991b). STAR professional development conference module. Baton Rouge: Louisiana Department of Education, College of Education, Louisiana State University. Lofton, G., Ellett, C., Hill, F., Claudet, J., Chauvin, S., & Loup, K. (1991c). Ideabook. Baton Rouge: Louisiana Department of Education, College of Education, Louisiana State University. Lofton, G., Ellett, C., Hill, F., Claudet, J., Chauvin, S., & Loup, K. (1992). Re¯ecting in action: motivating, monitoring, and maintaining learning. Baton Rouge: Louisiana Department of Education, College of Education, Louisiana State University. Lofton, G., Ellett, C., Hill, F., & Rugutt, J. (1996) What in¯uences maintenance of a statewide school improvement effort? A ®ve-year follow-up study. Paper presented at the meeting of the American Education Research Association, New York. Loup, K., Garland, J., Ellett, C., & Rugutt, J. (1996). Ten years later: Findings from a replication of a study of teacher evaluation practices in our 100 largest school districts. Journal of Personnel Evaluation in Education, 10, 203±226. Mann, D. (1983). Soft and hard strategies of change. New York: Teachers College Press. McCleod, M. A. (1981). The identi®cation of intended learning outcomes by early childhood teachers: An exploratory study. Doctoral dissertation, University of Alberta. McLaughlin, M. W. (1990). Embracing contraries: Implementing and sustaining teacher evaluation. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher evaluation: Assessing elementary and secondary school teachers (pp. 403±415). Newbury Park, CA: Sage.

TEACHER EVALUATION AND SCHOOL IMPROVEMENT

165

McLaughlin, M., & Marsh, D. D. (1978). Staff development and school change. Teachers College Record (September), 69±94. McLaughlin, M., & Pfeiffer, R. S. (1988). Teacher evaluation: Improvement, accountability, and effective learning. New York: Teachers College Press. Millman, J. & Darling-Hammond, L. (Eds.) (1990). The new handbook of teacher evaluation. Newbury Park, CA: Sage. Marzano, R. J., Brandt, R. S., Hughes, C. S., Jones, B. F., Presseisen, B. Z., Rankin, S. C., & Suhor, C. (1988). Dimensions in thinking: A framework for curriculum and instruction. Alexandria, VA: Association for Supervision and Curriculum Development. Mazzoni, T. L. (1991). Analyzing state school policy making: An arena model. Educational Evaluation and Policy Analysis. 13(2), 115±138. Natriello, G. (1990). Intended and unintended consequences: Purposes and effects of teacher evaluation. In J. Millman & L. Darling-Hammond (Eds.), The new handbook of teacher evaluation: Assessing elementary and secondary school teachers (pp. 35±45). Newbury Park, CA: Sage. Popham, J. (1988). The dysfunctional marriage of formative and summative teacher evaluation. Journal of Personnel Evaluation in Education. 1, 269±274. Porter, & Brophy, J. (1988). Synthesis of research on good teaching: Insights from the work of the Institute for Research on Teaching. Educational Leadership. 45, 71±85. Rodriguez, S., & Johnstone, K. (1986). Staff development through a collegial support group model. In Improving Teaching: 1986 ASCD Yearbook (pp. 87±89). Publication of the Association for Supervision and Curriculum Development. Schon, D. A. (1987). Educating the re¯ective practitioner. San Francisco: Jossey-Bass. Scriven, M. (1967). Summative teacher evaluation. In J. Millman (Ed.), Perspectives on curriculum evaluation. Chicago: American Educational Research Association Monograph Series on Evaluation, No. 1. Senge, P. M. (1990). The ®fth discipline: The art and practice of the learning organization. New York: Currency Doubleday. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4±14. Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1±22. Slavin, R. E. (1990). Cooperative learning: Theory, research, and practice. Englewood Cliffs, NJ: Prentice-Hall. Stronge, J. (1997). Evaluating teaching: A guide to current thinking and best practice. Thousand Oaks, CA: Corwin Press. Stuf¯ebeam, D. (1988). The personnel evaluation standards: How to assess systems for evaluating educators. Newbury Park, CA: Sage. Taylor, P., Dawson, V., & Fraser, B. (1995). Classroom learning environments under transformation: A constructivist perspective. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA. Tobin, K. (1993). The practice of constructivism in science. Washington, DC: AAAS Press. Wheatley, M. J. (1992). Leadership and the new science: Learning about organization from an orderly universe. San Francisco: Berrett-Koehler. Yinger, R. J. (1977). A study of teacher planning: Description and theory development using ethnographic and information processing methods. Doctoral dissertation, Michigan State University.