Observational Assessment of Teaching Practices Teaching Assessment Initiative Proposal Submitted to The Teachers for a New Era project

Karen M. La Paro

Problem Statement

The purpose of this research is to assess classroom quality as demonstrated by teachers’ “clinical practice,” (i.e., teaching processes) in K through 3^rd grade classrooms and examine the relation between classroom quality and children’s engagement. Issues being debated within several policy arenas such as accountability and accessibility to high quality educational programming for young children provide support for the need for a more systematic and focused effort to document the quality of teaching processes in the classroom.

This project has considerable significance in addressing major shortcomings in educational services provided to young children. First, although it is widely recognized that direct assessment of young children for accountability purposes is fraught with concerns and that the competencies of children are in part dependent on the quality of their experiences in educational settings, there are few studies that assess classrooms’ contributions to child competence. Second, although it is nearly universally recognized that access to high quality educational programs is a key component of improving child outcomes, there is no way currently of gauging whether children are attending high quality programs or ways of improving program quality that focuses on the actual experiences of children and teachers’ behavior in classrooms. Most ways of measuring and improving program quality rely on very distal quality indicators such as credentialing or class size and are thus disengaged from children’s and teachers’ actual classroom interactions

The Classroom Assessment Scoring System (CLASS) (La Paro & Pianta, 2000) is designed to directly address these questions by providing a common metric, vocabulary, and descriptive base for teaching and classroom processes from K to grade 3, by measuring variation in both instructional and social dimensions of the classroom environment. In so doing, information from the CLASS provides a mechanism by which classroom practices can be gauged and improved systematically. There is ample reason to believe that systematic observations of K to grade 3 classrooms would provide a mechanism for linking important policy and professional development questions to child and teacher outcomes in a way that the effects of policy or professional development initiatives could be systematically understood and evaluated in a formative, ongoing manner capable of generating empirically-based decisions. In this way, observations of classrooms fill a critical niche in making measurable and observable changes in classroom experiences that produce developmental gains for children.
Related Work

For the past ten years, we have been involved in several large-scale observational research efforts related to schooling and young children. These efforts include observations of large numbers of early elementary (NICHD ECCRN, 2002; Pianta, La Paro, Payne, Cox, & Bradley, 2002) and pre-kindergarten (Bryant et al., 2002) classrooms. This work was conducted in partnership with schools and school systems, involving large numbers of teachers, administrators, families, and children in many states across the country.

For example, we have observed in over 240 pre-kindergarten classrooms in six states (Bryant et al., 2002), 223 kindergarten classrooms in three states (Pianta et al., 2002), over 900 first grade classrooms in 295 school districts in 32 states (NICHD ECCRN, 2002), and over 900 third grade classrooms in more than 35 states (NICHD ECCRN, in preparation). In all, these observations describe almost 2,500 early education and elementary school classrooms. The overwhelming conclusion to be drawn from this work is the exceptional variability in learning experiences offered to children in the early grades (see NICHD ECCRN, 2002; Pianta et al., 2002 for specific results). For example, although for the most part the typical child receives instruction in whole-group experiences, in some classrooms children are never taught in a whole group while in others this is the mode of instruction all day. Similarly, although literacy instruction is the predominant activity offered to children, in a substantial number of classrooms children were offered no literacy activities at all. Further, when rating the richness of literacy instruction in first grade classrooms, over 20% of classrooms were rated “poor” on the level and quality of literacy experience offered to children. This is true whether the classroom is a pre-k, kindergarten, first, or third grade room (Bryant, et al., 2002; NICHD ECCRN, 2002; in preparation) and is independent of the level of materials and the physical environment provided.

It is difficult to characterize the level of variability in these rooms other than to say the entire range of codes was reflected in all the classroom samples we observed. For example, we found no typical kindergarten classroom. Across classrooms, we observed nearly the entire range of scale points available on various global and discrete codes. In some kindergarten or first-grade classrooms, 100% of the activity observed in a morning-long block was organized as teacher-directed instruction. In others, none of the activities observed could be coded as this form of instruction. In some of these classrooms, children were engaged in center activities for the entire observation and in others center activities never occurred. How teachers interacted with a target child was also highly variable. In some classrooms, a typical child might be exposed to interactions with their teacher only as a function of being a member of the entire class during whole group instruction, while in others interaction occurred in a mix of small group and one-to-one formats. On average, there was little emphasis in these classrooms on teaching social conventions, and little evidence of displayed affect by the teacher toward a typical child. In third grade, we find some classrooms in which all instruction occurs with the teacher standing at the front of the room talking to the whole class while in others children spend all their time working alone at their desks (NICHD ECCRN, in preparation). Also in third grade, it is still the case that even observing for an entire day’s academic activities, that many children (almost 15%) are not exposed any form of literacy instruction (NICHD ECCRN, in preparation). These findings confirm those of Meyer et al. (1993) in demonstrating the incredible variability in children’s experiences as a function of the classroom they attend. Importantly, the Meyer et al. (1993) data are based on nine full days of observation in each of two years, for 27 classrooms. Relatedly, Bryant et al. (1991) reported ratings using the entire seven point range on the ECERS for a number of scales, and concluded that although kindergarten classrooms showed, on average, good quality, there was no typical classroom. Our studies revealed very similar findings in terms of variability across classrooms for a much larger sample of classrooms (and school districts), albeit observed for a much shorter period of time than Meyer et al. (1993) and using a different methodology than Bryant et al. (1991).

• 
  How are teachers applying “pedagogical content knowledge” in the classroom? Specifically how are teachers providing emotional and instructional support for learning in K -3^rd grade classrooms?
  
• 
  How are these teaching practices related to children’s engagement?
  
• 
  How do teacher characteristics relate to observed teaching processes?
  
• 
  How do school characteristics relate to observed teaching processes?
  

The CLASS constructs related to management are: over-control, the flexibility that teachers display related to children’s interests and classroom schedules and the degree to which autonomous behavior in children is fostered; behavior management, how well teachers monitor, prevent and redirect behavior; and productivity, how well the classroom runs with respect to routines, how well children understand the routine and the degree to which teachers provide activities and directions so that maximum time can be spent in productive learning activities

Of note is that the multi-grade CLASS rating system also includes scales that correspond to academic content in 4 areas: Literacy, Mathematics, Social Studies, and Science. These scales were developed to capture the increasing amount of time and instruction devoted to these skill domains as the child moves from K through to grade 3. The literacy and mathematics scales have been field-tested in the NCEDL and NICHD studies and correlate, as expected, with children’s performance in those areas (NICHD ECCRN, in preparation). The scales for Social Studies and Science are viewed as descriptive at the present time, as we do not have validity data for those indicators.

Importantly, the CLASS constructs and 7-point rating scale approach to coding are the same for all grades K to 3 in the CLASS system, thus providing a common metric and vocabulary for description across these early elementary years. At each grade level the construct is defined or operationalized using descriptions that reflect grade-level appropriateness. Thus for “productivity,” “quality of feedback,” “literacy,” or “science” one would find the same construct in both the K and third grade versions of the CLASS but the actual descriptions and examples would differ by grade level.

In addition, teachers will be asked to complete a demographic questionnaire including questions about their education, professional development, age, specialization, and class characteristics. School information will be gathered from a brief principal questionnaire.
Analyses
Data collected will be analyzed to provide descriptive information related to classroom and teaching processes as well as the relation of these data to children’s engagement. These descriptive data also will be correlated to teacher characteristics and school characteristics.
Expected end product
The expected end product of the current project will be a summary of teaching behaviors and classroom processes across K through 3^rd grade and the relation of classroom quality to children’s engagement. These data can be presented across grades and across schools as well as for individuals. We will have data on teaching practices for individual teachers that can be examined in relation to teacher profiles including education and training as well as school characteristics. These data in addition to providing descriptive information for teachers at a local level can be compared to similar data collected at a national level.


Personnel
Karen M. La Paro, Ph.D

Dr. Karen La Paro has a Ph. D. in Special Education from the University of New Orleans. She holds a research faculty position at the University of Virginia and has worked with NCEDL since 1996 and the NICHD Study of Early Child Care since 1995.

Dr. Robert C. Pianta is a Professor in the Curry Programs in Clinical and School Psychology at the University of Virginia’s Curry School of Education.

This person will be trained to use the CLASS and will provide support to the proposed project in a number of ways including observing and videotaping in classrooms, contacting teachers, and mailing and correspondence, and data entry.

French, A., Gallagher, C., Denno, D., & Henderson, K. (2002, November). What is pre-k? Paper presented at the meeting of the National Association for the Education of Young Children, Chicago, IL.

Bryant, D., Clifford, R., & Peisner-Feinberg, E. (1991). Best practices for beginners:

Developmental appropriateness in kindergarten. American Educational Research Journal, 28(4), 783-803.

La Paro, K.M., & Pianta, R.C. (2000). CLASS: Classroom Assessment

Scoring System. Charlottesville, VA.

Meyer, L.A., Wardrop. J.L., Hastings, C.N., Linn, R.L. (1993). Effects of ability and on

kindergarteners’ reading performance. Journal of Educational Research, 86, 142-160.

National Institute of Child Health and Human Development, Early Child Care Research

Network (in preparation). The relation of global third-grade classroom environment to structural classroom features and teacher and student behaviors.

National Institute of Child Health and Human Development, Early Child Care Research

Network (2002). The relation of global first-grade classroom environment to structural classroom features and teacher and student behaviors. The Elementary School Journal, 102(5), 367-387.

Pianta, R.C. (2002, October). Experiences in p-3 classrooms: The implications of observational research for re-designing early education. Paper presented at the Foundation for Child Development. New York, NY.
Pianta, R.C., La Paro, K.M., Payne, C., Cox, M., J., & Bradley, R. (2002). The relation

of kindergarten classroom environment to teacher, family, and school