|Observational Assessment of Teaching Practices
Teaching Assessment Initiative Proposal
The Teachers for a New Era project
Karen M. La Paro
Robert C. Pianta
The purpose of this research is to assess classroom quality as demonstrated by teachers’ “clinical practice,” (i.e., teaching processes) in K through 3rd grade classrooms and examine the relation between classroom quality and children’s engagement. Issues being debated within several policy arenas such as accountability and accessibility to high quality educational programming for young children provide support for the need for a more systematic and focused effort to document the quality of teaching processes in the classroom.
This project has considerable significance in addressing major shortcomings in educational services provided to young children. First, although it is widely recognized that direct assessment of young children for accountability purposes is fraught with concerns and that the competencies of children are in part dependent on the quality of their experiences in educational settings, there are few studies that assess classrooms’ contributions to child competence. Second, although it is nearly universally recognized that access to high quality educational programs is a key component of improving child outcomes, there is no way currently of gauging whether children are attending high quality programs or ways of improving program quality that focuses on the actual experiences of children and teachers’ behavior in classrooms. Most ways of measuring and improving program quality rely on very distal quality indicators such as credentialing or class size and are thus disengaged from children’s and teachers’ actual classroom interactions
The Classroom Assessment Scoring System (CLASS) (La Paro & Pianta, 2000) is designed to directly address these questions by providing a common metric, vocabulary, and descriptive base for teaching and classroom processes from K to grade 3, by measuring variation in both instructional and social dimensions of the classroom environment. In so doing, information from the CLASS provides a mechanism by which classroom practices can be gauged and improved systematically. There is ample reason to believe that systematic observations of K to grade 3 classrooms would provide a mechanism for linking important policy and professional development questions to child and teacher outcomes in a way that the effects of policy or professional development initiatives could be systematically understood and evaluated in a formative, ongoing manner capable of generating empirically-based decisions. In this way, observations of classrooms fill a critical niche in making measurable and observable changes in classroom experiences that produce developmental gains for children.
For the past ten years, we have been involved in several large-scale observational research efforts related to schooling and young children. These efforts include observations of large numbers of early elementary (NICHD ECCRN, 2002; Pianta, La Paro, Payne, Cox, & Bradley, 2002) and pre-kindergarten (Bryant et al., 2002) classrooms. This work was conducted in partnership with schools and school systems, involving large numbers of teachers, administrators, families, and children in many states across the country.
For example, we have observed in over 240 pre-kindergarten classrooms in six states (Bryant et al., 2002), 223 kindergarten classrooms in three states (Pianta et al., 2002), over 900 first grade classrooms in 295 school districts in 32 states (NICHD ECCRN, 2002), and over 900 third grade classrooms in more than 35 states (NICHD ECCRN, in preparation). In all, these observations describe almost 2,500 early education and elementary school classrooms. The overwhelming conclusion to be drawn from this work is the exceptional variability in learning experiences offered to children in the early grades (see NICHD ECCRN, 2002; Pianta et al., 2002 for specific results). For example, although for the most part the typical child receives instruction in whole-group experiences, in some classrooms children are never taught in a whole group while in others this is the mode of instruction all day. Similarly, although literacy instruction is the predominant activity offered to children, in a substantial number of classrooms children were offered no literacy activities at all. Further, when rating the richness of literacy instruction in first grade classrooms, over 20% of classrooms were rated “poor” on the level and quality of literacy experience offered to children. This is true whether the classroom is a pre-k, kindergarten, first, or third grade room (Bryant, et al., 2002; NICHD ECCRN, 2002; in preparation) and is independent of the level of materials and the physical environment provided.
It is difficult to characterize the level of variability in these rooms other than to say the entire range of codes was reflected in all the classroom samples we observed. For example, we found no typical kindergarten classroom. Across classrooms, we observed nearly the entire range of scale points available on various global and discrete codes. In some kindergarten or first-grade classrooms, 100% of the activity observed in a morning-long block was organized as teacher-directed instruction. In others, none of the activities observed could be coded as this form of instruction. In some of these classrooms, children were engaged in center activities for the entire observation and in others center activities never occurred. How teachers interacted with a target child was also highly variable. In some classrooms, a typical child might be exposed to interactions with their teacher only as a function of being a member of the entire class during whole group instruction, while in others interaction occurred in a mix of small group and one-to-one formats. On average, there was little emphasis in these classrooms on teaching social conventions, and little evidence of displayed affect by the teacher toward a typical child. In third grade, we find some classrooms in which all instruction occurs with the teacher standing at the front of the room talking to the whole class while in others children spend all their time working alone at their desks (NICHD ECCRN, in preparation). Also in third grade, it is still the case that even observing for an entire day’s academic activities, that many children (almost 15%) are not exposed any form of literacy instruction (NICHD ECCRN, in preparation). These findings confirm those of Meyer et al. (1993) in demonstrating the incredible variability in children’s experiences as a function of the classroom they attend. Importantly, the Meyer et al. (1993) data are based on nine full days of observation in each of two years, for 27 classrooms. Relatedly, Bryant et al. (1991) reported ratings using the entire seven point range on the ECERS for a number of scales, and concluded that although kindergarten classrooms showed, on average, good quality, there was no typical classroom. Our studies revealed very similar findings in terms of variability across classrooms for a much larger sample of classrooms (and school districts), albeit observed for a much shorter period of time than Meyer et al. (1993) and using a different methodology than Bryant et al. (1991).
Despite this exceptional variability, there is a picture that emerges of the typical early elementary classroom: whole-group instruction, a positive social environment, and low levels of productivity and engagement in academic activities. We think of these environments as socially positive but instructionally passive: children listen and watch, much time is spent on routines or management of materials, and children have little direct contact with teachers in instructional interactions. This actually seems to be increasingly the case as children move from first to third grade—the classroom becomes even more passive (NICHD ECCRN, in preparation). Despite being generally well-organized and busy places, classrooms appear low in “intentionality,” a term that refers to directed, designed interactions between children and teachers in which teachers purposefully challenge, scaffold, and extend children’s skills. Another major finding across all these studies and grade-levels is that factors used to regulate classroom quality such as teacher education or class size, as well as the use of any or specific curricula, bear little or no relation to observed quality or to child outcomes (Pianta, 2002).
How are teachers applying “pedagogical content knowledge” in the classroom? Specifically how are teachers providing emotional and instructional support for learning in K -3rd grade classrooms?
How are these teaching practices related to children’s engagement?
How do teacher characteristics relate to observed teaching processes?
How do school characteristics relate to observed teaching processes?
Procedures for collecting information
All teachers in city and county elementary schools will be contacted to participate in this study. Resources will allow for approximately 60 classrooms to be observed, 15 per grade level. Information about teaching behaviors and classroom processes will be collected through systematic observations of K-3rd grade classrooms in the Charlottesville/Albemarle area using the Classroom Assessment Scoring System, CLASS. The CLASS is a standardized system of observational rating scales used to measure attributes of the classroom environment that are predictive of social and academic growth and development. In the CLASS, 13 rating scales are used to observe and measure the early elementary classroom environment across grades – the scales measure the same constructs (e.g., teacher sensitivity, literacy instruction) in grade-appropriate ways, for each grade.
Socioemotional climate and quality of teacher-child interactions. The constructs included under emotional climate on the CLASS are: positive climate, which examines the enthusiasm, enjoyment and emotional connection that teacher has with the children as well as the nature of peer interactions; negative climate, includes evidence of anger, hostility or aggression exhibited by the teacher and/or children in the classroom; and sensitivity, how responsive the teacher is to children academic and emotional needs and the degree to which the teacher can be viewed as a secure base for children.
The CLASS constructs related to management are: over-control, the flexibility that teachers display related to children’s interests and classroom schedules and the degree to which autonomous behavior in children is fostered; behavior management, how well teachers monitor, prevent and redirect behavior; and productivity, how well the classroom runs with respect to routines, how well children understand the routine and the degree to which teachers provide activities and directions so that maximum time can be spent in productive learning activities
Instructional support. The CLASS has 3 scales that focus on instructional support provided in the classroom: concept development, the degree to which teachers promote higher order thinking and problem solving, going beyond fact and recall activities for children; instructional learning formats, how teacher engage children in activities an facilitate activities so that learning opportunities are maximized; and quality of feedback, how teachers extend children’s learning through their responses and participation in activities.
Of note is that the multi-grade CLASS rating system also includes scales that correspond to academic content in 4 areas: Literacy, Mathematics, Social Studies, and Science. These scales were developed to capture the increasing amount of time and instruction devoted to these skill domains as the child moves from K through to grade 3. The literacy and mathematics scales have been field-tested in the NCEDL and NICHD studies and correlate, as expected, with children’s performance in those areas (NICHD ECCRN, in preparation). The scales for Social Studies and Science are viewed as descriptive at the present time, as we do not have validity data for those indicators.
Children’s engagement. Children’s engagement is assessed by observing the degree to which children are focused and participating in the activity presented or facilitated by the teachers.
Importantly, the CLASS constructs and 7-point rating scale approach to coding are the same for all grades K to 3 in the CLASS system, thus providing a common metric and vocabulary for description across these early elementary years. At each grade level the construct is defined or operationalized using descriptions that reflect grade-level appropriateness. Thus for “productivity,” “quality of feedback,” “literacy,” or “science” one would find the same construct in both the K and third grade versions of the CLASS but the actual descriptions and examples would differ by grade level.
In addition, teachers will be asked to complete a demographic questionnaire including questions about their education, professional development, age, specialization, and class characteristics. School information will be gathered from a brief principal questionnaire.
Data collected will be analyzed to provide descriptive information related to classroom and teaching processes as well as the relation of these data to children’s engagement. These descriptive data also will be correlated to teacher characteristics and school characteristics.
Expected end product
The expected end product of the current project will be a summary of teaching behaviors and classroom processes across K through 3rd grade and the relation of classroom quality to children’s engagement. These data can be presented across grades and across schools as well as for individuals. We will have data on teaching practices for individual teachers that can be examined in relation to teacher profiles including education and training as well as school characteristics. These data in addition to providing descriptive information for teachers at a local level can be compared to similar data collected at a national level.
Karen M. La Paro, Ph.D
Dr. Karen La Paro has a Ph. D. in Special Education from the University of New Orleans. She holds a research faculty position at the University of Virginia and has worked with NCEDL since 1996 and the NICHD Study of Early Child Care since 1995.
Robert C. Pianta, Ph.D
Dr. Robert C. Pianta is a Professor in the Curry Programs in Clinical and School Psychology at the University of Virginia’s Curry School of Education.
This person will be trained to use the CLASS and will provide support to the proposed project in a number of ways including observing and videotaping in classrooms, contacting teachers, and mailing and correspondence, and data entry.
Graduate Assistant. The Graduate Assistant (GA) will be a Master’s level or Doctoral student in the Curry School of Education. The GA will be supported for 10 hours per week from September 2003 through March 2004 at a rate of $10/hour. Approximate cost = $3080
Payment to Teachers. Each teacher participating in the study will receive a $25 gift certificate to Barnes and Noble. Approximate cost = $1500 for 60 classrooms.
Materials. Videotapes and editing. $420
Bryant, D., Clifford, D., Early, S., Pianta, R., Ritchie, S., Trammel, C., Hawley, S.,
French, A., Gallagher, C., Denno, D., & Henderson, K. (2002, November). What is pre-k? Paper presented at the meeting of the National Association for the Education of Young Children, Chicago, IL.
Bryant, D., Clifford, R., & Peisner-Feinberg, E. (1991). Best practices for beginners:
Developmental appropriateness in kindergarten. American Educational Research Journal, 28(4), 783-803.
La Paro, K.M., & Pianta, R.C. (2000). CLASS: Classroom Assessment
Scoring System. Charlottesville, VA.
Meyer, L.A., Wardrop. J.L., Hastings, C.N., Linn, R.L. (1993). Effects of ability and on
kindergarteners’ reading performance. Journal of Educational Research, 86, 142-160.
National Institute of Child Health and Human Development, Early Child Care Research
Network (in preparation). The relation of global third-grade classroom environment to structural classroom features and teacher and student behaviors.
National Institute of Child Health and Human Development, Early Child Care Research
Network (2002). The relation of global first-grade classroom environment to structural classroom features and teacher and student behaviors. The Elementary School Journal, 102(5), 367-387.
Pianta, R.C. (2002, October). Experiences in p-3 classrooms: The implications of observational research for re-designing early education. Paper presented at the Foundation for Child Development. New York, NY.
Pianta, R.C., La Paro, K.M., Payne, C., Cox, M., J., & Bradley, R. (2002). The relation
of kindergarten classroom environment to teacher, family, and school
characteristics and child outcomes. The Elementary School Journal, 102(3), 225-