Discussion
Using the PWT as an instrument and a process for scaffolding reflective practice helped us as teacher educators to reframe our teaching of language and literacy. We found that a key element in the improvement of our teaching was our collaboration. Collaboration plays a key role in self-studies because it allows for one’s data and findings to be examined and challenged professionally (Hamilton & Pinnegar, 1998; Loughran, 2002; Loughran & Northfield, 1998). We found as Bass, Anderson-Patton, and Allender (2002) state, “Involvement with self-study teaches one that all learning is better when integrated with self, and collaborating with critical friends….” (p. 68). The process of sharing and re-evaluating papers with each other gave us the space to develop and maintain a critical stance toward our own teaching practices, theories, and beliefs. We became more sensitive to issues of power in the classroom.
Our study brought out what McNiff and Whitehead (2006) term our “living contradictions” (p. 46) with respect to issues of power. For example, Patience found that despite her beliefs in critical pedagogy and social justice, she did not emphasize how important it was for her graduate students to teach their students about the politics of language learning in the United States, the spread of languages such as Spanish and French through colonialism, and the impact of colonialism and economics in Spanish-speaking and Francophone countries.
We were pleased when our graduate students grappled with and were sensitive to the issues of social and political power inherent in teaching and learning, and expressed these ideas in their PWT papers. However, discussions with graduate students and ideas in their papers made us more aware of whether we were prompting them to talk and write about issues of power and social justice because of their importance in our own teaching and our department’s conceptual framework. The following excerpt from one of our graduate students provides evidence for this concern.
This is my first semester in this program, but I have noticed a trend in all of my classes. The philosophy behind every class I have taken, and all the authors I have read for them, have been influenced in some way by the ideas of Freire (1987). It seems like the M.Ed. program pushes its students towards developing what Freire calls, “a critical practice and understanding of literacy” (p.37). The critical practice we are supposed to adopt seems to imply that we should develop a practice that is culturally relevant to our students. This culturally relevant pedagogy is liberating to the students…. (TK, 11/20/04)
Our experiences with this particular graduate student indicate that he is personally convinced of the value of culturally relevant pedagogy; however, we are sensitive to his comments (i.e., “It seems like the M.Ed. program pushes its students” and “the critical practice we are supposed to adopt”).
Reframing our teaching by using the PWT as a scaffold for learning helped our graduate students realize how their beliefs and values play a major role in who they are as teachers, how these values influence the decisions they make, and lead towards the creation of their living theories of teaching (Whitehead, 1993; Whitehead & McNiff, 2006). As a model, we found the PWT gave our graduate students three concrete, essential, and intersecting elements to reflect upon to improve their teaching. Furthermore, it helped them understand that this is a process; that is, a growth and evolution of understanding and values over time (Dalmau & Gudonjonsdottir, 2002; Zeichner & Liston, 1996). This is consistent with our own beliefs about the education and professional development of teachers.
Scaffolding reflective practice through the PWT led to better dialog between ourselves, as well as with our graduate students. We developed a common language, and through our collaboration we were able to dialog and provide useful feedback to each other about our own and our graduate students’ progress in teaching and learning about reflective practice respectively. For example, we considered our graduate students’ abilities to explain ethical issues in their papers and in their metaphors, and we evaluated the qualities of classroom discussions about theoretical understandings related to our different courses.
In addition to improving our teaching, using the PWT made us more mindful of the connections between our own professional growth as college teachers and the growth of preservice and inservice teachers with whom we work. We realized that we must consistently unpack our thinking and explicitly model and scaffold for university students in the same ways and for the same reasons that we encourage them to model for their students.
We found that we had higher expectations for our graduate students’ professional discussions and their work when we were able to use PWT as a common language for both our courses. We enlarged the role of peer discussions to allow graduate students to compare their ideas and to articulate their understandings related to different elements of PWT. We found that the best discussions emanated from a shared experience or a focused assignment; such as watching a video together, or writing text-to-world connections to a particular article. We became better at planning discussion prompts and better at listening to our graduate students’ responses when we used PWT as an organizing principle. We found ultimately that the whole process of reframing our teaching and scaffolding reflective practice helped us and our graduate students understand the complexities of teaching and learning.
Conclusion
Our self-study of our collaboration to use the PWT to scaffold reflective practice gave us new understandings of practice, theories and ethics as teacher educators (Dalmau & Gudonjonsdottir, 2002). In this study we sought to reframe our practice to develop graduate students’ reflection about themselves as teachers. The PWT provided us with a framework for discussing our graduate students’ growth and provided them with a model for reflective practice. We developed a common language and became more explicit in our explanations of why we used strategies like scaffolding, classroom discussion and metaphors. However, we also realize that the PWT model is perhaps more powerful for practicing teachers than it is for preservice teachers. We both still struggle with finding ways in which preservice teachers might understand the usefulness of using both the big and small “t’s” in developing and maintaining reflective practice.
The more we use the PWT framework and study ourselves in our work as teacher educators, the more meaningful our practice has become for us. “When seriously adapted, self-study of teaching and teacher education practices supports meaningful learning about practice” (Loughran, 2004, p. 183). Although now we teach different courses at different universities, we have been able to carry on an extended dialog about teaching and learning in our classrooms using the PWT as a focal point. As we discuss elements of our students’ work in different courses we find ourselves using the three elements of PWT in our conversations about goals for different classes, for research, and for workshops. We realize too, for our teaching to become effective, that these conversations must be carried beyond our courses and classrooms and become an integral part of our teacher education programs.
We understand that using the PWT is a continual process to explore connections between our own professional growth as teacher educators and the growth of the K-12 teachers with whom we work.
References
Barnes, D. (1998). Looking forward: The concluding remarks at the castle conference. In M. Hamilton (Ed.), Reconceptualizing teacher education practice: Self-study in teacher education. London: Falmer Press.
Bass, L., Anderson-Patton, V., & Allender, J. (2002). Self-study as a way of teaching and learning: a research collaborative re-analysis of self-study teaching portfolios. In J. Loughran & T. Russell (Eds.), Improving teacher education practices through self-study. London: Routledge Falmer.
Bogden, R. & Bicklen, S. (1992). Qualitative research for education: An introduction to theory and methods. Boston, MA: Allyn and Bacon.
Calkins, L. (2001). The art of teaching reading. New York: Pearson.
Creswell, J.W. (2004). Educational research. Planning, conducting, and evaluating quantitative and qualitative Research. New Jersey: Merrill Prentice, Hall.
Cole, A., & Knowles, G. (1998). The self-study of teacher education practices and the reform of teacher education. In M. Hamilton (Ed.), Reconceptualizing teacher education practice: Self-study in teacher education. London: Falmer Press.
Dalmau, M., Gudonjonsdottir, H. (2002). Framing professional discourse with teachers. Professional working theory. In J. Loughran, & T. Russell (Eds.), Improving teacher education practices through self-study. London: Routledge Falmer.
Dewey, J. (1933). How we think. New York: Heath & Co.
Fitzgerald, L. Farsted, & Deemer (2000). What gets “mythed” in the student evaluations of their teacher education professors? In J. Loughran & T. Russell (Eds.), Exploring Myths and Legends in Teacher Education. Proceedings of the Third International Conference of the Self-Study of Teacher Education Practices. (Vol.1, pp 66-69). Hersmonceaux Castle, East Sussex, England. Kingston, Ontario: Queen’s University.
Freire, P., & Macedo, D. (1987). Literacy: Reading the word and the world. Westport, CT: Bergin & Garvey Publishers
Greene, M. (1997). The passions of pluralism: Multiculturalism and the expanding community. In S. Cahn (Ed.) Classic and contemporary readings in the philosophy of education. New York: McGraw-Hill. p.510-521.
Hall, J. K. (2001). Methods for teaching foreign languages. Creating a community of learners in the classroom. New Jersey: Merrill Prentice Hall.
Hamilton, M. L., & Pinnegar, S. (1998). The value and promise of self-study. In M. Hamilton (Ed.), Reconceptualizing teacher education Practice: Self-study in teacher education. London: Falmer Press.
Handal, G., & Lauvas, P. (1987). Promoting reflective teaching: Supervision in action. London: Society of Research into Higher Education/Open University Press.
Harvey, S., & Goudvis, A. (2000). Strategies that work: Teaching comprehension t o
enhance understanding. Portland, ME: Stenhouse Publishers.
Jacobson, J. M. (1998). Content area reading. Boston, MA: Delmar Publications.
Keene, E. O., & Zimmerman, S. (1997). Mosaic of thought: Teaching Comprehension in a reader’s workshop. Portsmouth, NH, Heinemann
Korthagen, F. A., & Kessels, J. P. (May, 1999). Linking theory and practice: Changing the pedagogy of teacher education. Educational Researcher, 28(4) pp.4-17.
Lighthall, F. (2004). Fundamental features and approaches of the s-step enterprise. International handbook of self-study of teacher education practices. J. Loughran, M.Hamilton, V. LaBoskey & T. Russell (Eds.), International handbook of self-study of teaching and teacher education practices. Dordrecht: Kluwer Academic Publishers. p. 193-246.
Loughran, J. (2002). Understanding self-study of teacher education practices. In J. Loughran, & T. Russell (Eds.), Improving teacher education practices through self-study. London: Routledge Falmer
Loughran, J. (2004). Learning through self-study: The influence of purpose, participants and context. J. Loughran, M. Hamilton, V. LaBoskey, & T. Russell (Eds.), International handbook of self-study of teaching and teacher education practices. Dordrecht: Kluwer Academic Publishers. pp.151-192.
Loughran, J., & Northfield, J. (1998). A framework for the development of self-study practice. In M. Hamilton (Ed.), Reconceptualizing teacher education practice: Self-study in teacher education. London: Falmer Press.
McNiff, J., & Whitehead, J. (2006). All you need to know about action research. London: Sage Publications.
Nieto, S. (2002) Language culture and teaching: Critical perspectives for a new century. New Jersey: Lawrence Erlbaum.
Richards, J.C. (1998). Turning to the artistic: Developing an enlightened eye by creating teaching self-portraits in M.L. Hamilton (Ed.), Reconceptualizing teaching practices: Self- study in teacher education (pp 34-44) London: Falmer Press.
Richards , J. C. & Richards, P.H. (1998). “Metaphorically speaking”: Discovering and enhancing our professional selves through figurative learning. In A.L. Cole & S. Finley (Eds.), Conversations in Community. Proceedings of the Second International Conference of the Self-Study of Teacher Education Practices (pp.199-201). Herstmonceaux Castle, East Sussex, England. Kingston, Ontario: Queen’s University.
Richardson, V. (1996). The role of attitudes and beliefs in learning to teach. In J.Sikula, T.Buttery, & E. Guyton (Eds.), Handbook of research on teacher education, 2nd ed. New York: Simon and Schuster Macmillan.
Russell, T. (2002). Can self-study improve teacher education? In J. Loughran, & T. Russell (Eds), Improving teacher education practices through self-study. London: Routledge Falmer
Schon, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.
Schon, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass.
Shrum, J. L., & Glisan, E. (2005). Teacher’s handbook. Contextualized language instruction. 3rd Edition. Australia: Thompson & Heinle.
Whitehead, J. (1989). “Creating a living educational theory form questions of the kind, “How do I improve my practice?” Cambridge Journal of Education, 19 (1): 137-53. Retrieved 21 July 2006 from http://www.bath.ac.uk/~edsajw/writings/livtheory.html
Whitehead, J. (1993). The growth of educational knowledge: Creating your own living educational theories. Bournemouth: Hyde Publications. Retrieved 21 July 2006 from http://www.bath.ac.uk/~edsajw/bk93/geki.htm.
Whitehead, J., & McNiff, J. (2006). Action research living theory. London: Sage Publications.
Zeichner, K., & Liston, D. (1996). Reflective teaching. An introduction. Mawah, N.J: Lawrence Erlbaum Associates.
Author Biographies
Patience Sowa, Ph.D., is Assistant Professor of Education at Zayed University, Abu Dhabi in the United Arab Emirates. Her research interests include the effective preparation of English Language Learner teachers, literacy, and English language learning and multicultural and bilingual education. Email: patience.sowa@zu.ac.ae.
Cynthia Schmidt, Ph.D., is Assistant Professor of Reading and Literacy at the University of Missouri–Kansas City. Her research interests include using reading and writing workshop approaches with struggling readers and English language learners, and preparing preservice teachers for literacy instruction in urban schools. Email: Schmidtcy@umkc.edu.
APPENDIX
ED 3770
Name:__________________
Professional Working Theory
… the professional understanding that evolves through the constant interplay of professional knowledge, practical experience, reflection, and ethical or moral principles (Dalmau and Guodjonsdottir, 2002).
-
Practice
-
What I do in my daily work of teaching.
-
Theory
-
How I understand and explain what I do
-
Ethics
-
Why I do what I do. What are my personal ethics and values? How are they visible in my work? What are the sources of my ethics and values? What are the societal and cultural impacts on my ethics and values?
To complete the table, think about the various activities we have completed in class (e.g., metaphors of teaching activity, what you do or might do as a teacher, and the four theories that square with your beliefs activity.
Draw a visual of your professional working theory.
Using Teaching Learning Projects to Assess and Impact Candidate Practice and Pupil Learning
Larry Maheady, Michael Jabot, and Janeil Rey
SUNY Fredonia
Abstract
Teacher educators are under increasing pressure to show that their preparation programs make a meaningful impact on preservice candidates’ teaching and that their candidates, in turn, improve pupil learning. This external pressure, while debatable on some fronts, poses a formidable educational challenge for teacher educators. Here, an early field-based course and applied teaching project are described that may serve as useful vehicles for examining candidate practice and student learning. Sixty-two preservice candidates completed an authentic teaching assignment in which they taught two formal lessons, collected pre- and post-teaching data, and responded to information gleaned from their professional reflections. Candidates summarized findings in written Teaching Learning Projects (TLPs) that were derived from the teacher work sample methodology. Findings indicated that candidates provided approximately 2,480 hours of in-class assistance during the Spring Term 2007, taught 124 formal lessons, and made a noticeable impact in pupil learning in about 90% of their lessons. Implications for teacher educators are discussed.
Teacher educators are under increasing political pressure to show that they can positively influence candidate practice and that candidates can, in turn, improve student learning (e.g., Abell Foundation, 2001; Ballou & Podgursky, 2000; Carnine, 2000; Coalition for Evidence-Based Educational Policy, 2002; US Department of Education, 2002). While one may question the motivation for this external criticism, it is important to respond constructively to it. One way to respond constructively is by conducting more well-controlled research studies in teacher education (Brantlinger, Jimenez, Klingner, Pugach, & Richardson, 2005; Gersten, Fuchs, Compton, Coyne, Greenwood, & Innocenti, 2005; Odom, Brantlinger, Gersten, Horner, Thompson, & Harris, 2005; Shulman, 2002). These studies might describe how candidate practice develops, how it is influenced positively or negatively by preparation experiences, and how it is maintained and refined throughout candidates’ careers. More studies are also needed on what impact candidates have on student learning, an understudied area in teacher education research (Pugach, 2005; Wilson, Floden, & Ferrini-Murphy, 2002).
This manuscript describes one university’s use of a required early field experience course and an authentic teaching assignment to assess candidate practice and its impact on pupil learning. The course and field experience are described within the larger context of the Responsive Educator Program (REP), an inclusive preparation program for early childhood, childhood, and adolescence education candidates. Findings from 62 Teaching Learning Projects (TLPs) completed during Spring Term 2007 are reviewed. These projects yielded qualitative and quantitative data on candidates’ ability to design and implement developmentally appropriate lessons and to examine their impact on student learning. Basic research questions included, How do these early professional experiences affect candidates’ teaching practice, in particular, their ability to design, implement, and evaluate two, developmentally appropriate lessons? And, what effects, if any, do these teaching practices have on pupil learning?
Before proceeding, however, three caveats are offered. First, this paper describes how candidates can generate evidence to support their impact on student learning by documenting changes in pupil performance from pre- to post-assessment on an authentic teaching assignment. Obviously, there are many issues, conceptual and procedural, associated with the use of pre-post assessment methods and even more so when the analysis is conducted at the individual lesson level. There is, however, a need to identify some forms of evidence where none had previously existed and to identify ways for capturing such information. Our description should not infer, however, that candidate practice is related functionally (i.e., cause-and effect-relationship) to student learning. Such relationships can only be derived from experimental research methods (i.e., single case or randomized control trials). A second related concern is a lack of inter-rater reliability data on project scoring. Although the authors consulted regularly on scoring procedures, no formal analyses were conducted to ensure that data were collected in a reliable manner across course sections. Finally, a substantial proportion of the literature review comes from special rather than general education. As such, generalizations to general education teacher preparation should be made with an appropriate level of caution.
Method
The Responsive Educator Program
The REP is a preservice program for early childhood, childhood, and adolescence education candidates at a medium-sized (i.e., 5,200 students), comprehensive, regional university in New York. The university consists of three inter-related colleges (arts and humanities, natural and social sciences, and education) across which teacher candidates comprise the largest segments of the undergraduate and graduate enrollments. The College of Education offers Bachelors’ and Master’s degrees and an administrative credential, provides the most extensive community-based services, and secures the most external funding annually.
The REP differs, however, in some important ways. First, the faculty is a unique blend of general and special educators (6 full-time special educators on a 21-member faculty) in a common preparation program. There has never been a separate special education program at the institution. Instead, special and general education faculty members have worked collaboratively to redesign the existing general education programs. Second, the blended faculty adopted and maintained a single curriculum that emphasizes the preparation of childhood educators for teaching all pupils in inclusive educational settings. In this project, for example, one general, one special, and one educational administration faculty member worked together to design, implement, and refine the first early field-based course and experience. Third, the REP has adopted a common assessment methodology, TLPs, to monitor candidate practice and to collect data on student learning. Here, candidates completed the first of five formal teaching experiences and TLPs.
Course Participants and Settings
Sixty-two undergraduate teacher candidates (54 female and 8 male) and the pupils enrolled in their respective field placements participated. Demographic data on candidates and their educational placements can be seen in Table 1.
Table 1. Demographic Characteristics of Teacher Candidates and Placement Types
Instructors
|
Candidates
|
Female
|
Male
|
Early Childhood/
Childhood/
Adolescent Education
|
Percent in High-Need Schools
|
Classrooms’ N
|
A (Ray)
|
11
|
11
|
0
|
11 EC
|
29%
|
6 K
4 Pre-K
1 Gr. 2
|
B (Maheady)
|
28
|
26
|
2
|
27 CH
1 EC
|
96%
|
12 Gr. 3
7 K
2 Gr. 1
2 Gr. 5
2 K-5
1 Gr. 4
1 Gr. K-2
1 Gr. 3-5
|
C (Jabot)
|
23
|
17
|
6
|
14 AD
9 CH
|
87%
|
6 Gr. 6
4 Gr. 8
4 Gr. 3
3 Gr. 1
3 Gr.11-12
2 Gr. 3-5
1 Gr. 7-8
|
Totals
|
62
|
54
|
8
|
36 CH
12 EC
14 AD
|
71%
|
|
Most candidates (87%) were female and childhood education majors (58%). About three-fourths (71%) completed their field experience in high-need schools. The two most prevalent placements were small and mid-sized urban school systems with significant Hispanic and African-American representations and high percentages of free and reduced meals. Two-thirds of childhood education candidates were placed in third grade (46%) and kindergarten (18%) classrooms, while all but one early childhood candidate worked in kindergarten and prekindergarten settings. All adolescence education candidates were placed in discipline-specific classrooms at the secondary level. Fifty candidates (8%) were assigned to their field placements in pairs, while 12 worked individually in their respective settings.
In school, candidates worked with approximately 740 students ranging in age from 4 (prekindergarten) to 18 (grades 11 and 12). This number was derived by multiplying the number of candidate pairs by an average class size of 20 and then adding the number of pupils enrolled in classrooms where candidates worked individually.
EDU 105 and the Instructional Assistants Program
EDU 105, Introduction to Contemporary Education, is a required course for all candidates enrolled in teacher education programs. The course is taken typically in the freshman year (or junior year for transfers) and completed before candidates enroll in other education courses. The field experience component (EDU 106) is a 0-credit, multi-faceted teaching/learning experience that includes an 8-week field placement, participation handbooks, and teaching/learning contracts that delineate candidate roles, responsibilities, and expected professional behaviors. Candidates also teach two formal lessons, assess pupils before and after instruction, and successfully complete TLPs. Candidates are paired for their field experience to promote professional reflection and peer-collaboration. Partner pairings also served a number of important logistical functions. They allowed instructors, for example, to be more selective in field placements, eased barriers associated with transportation, emotional support, and lack of candidate initiative issues, and served as natural vehicles for professional collaboration. Candidates worked collaboratively, for instance, to plan, teach, reflect, and respond to individual lessons, shared other instructional responsibilities assigned by the classroom teacher, and collected, analyzed, and responded to pupil outcome data from individual lessons.
Candidates attended their field placements twice per week for eight weeks and for a minimum of two hours per day. They participated daily in a variety of mutually agreed upon instructional responsibilities outlined in their teaching/learning contracts (i.e., 20-item checklists of instructional and non-instructional duties), completed focused observations, and kept reflective journals. They also attended class on campus where they worked in small, collaborative groups sharing information and concerns about their field placements and discussing broader, course-related issues.
Teaching Learning Projects
TLPs were derived from earlier versions of the Teacher Work Sample methodology (McConney, Schalock, & Schalock, 1998; Renaissance Partnership, 2002). Work samples are authentic, multi-faceted performance assessments that measure candidate ability to demonstrate proficiency on seven important teaching standards (i.e., using context to plan instruction, selecting appropriate learning goals, developing an assessment plan, designing instruction, making instructional decisions, analyzing student learning, and reflecting on the teaching/learning process) (Denner, Norman, Salzman & Pankratz, 2003; Denner, Salzman, & Bangert, 2001). Teacher work samples require candidates to develop, implement, and evaluate comprehensive instructional units during applied teaching experiences. Before teaching, however, candidates must describe relevant context factors, identify important standards-based learning goals, and create an assessment plan that evaluates student performance before (pre-assessment), during (formative assessment), and after (post-assessment) instruction (Renaissance Partnership, 2002).
Candidates use information from the contextual analysis and pre-assessments to inform their lesson design. After teaching, they analyze their impact on student learning through graphic and tabular displays of pupil performance on pre- and post-teaching assessments. These data include analyses of entire class, sub-group (e.g., students with special needs, and English Language Learners), and individual learners’ (e.g., students with special needs and high achievers) performance. Candidates reflect on and evaluate their own teaching in relation to pupil outcomes and describe ways to improve their subsequent teaching practice (Renaissance Partnership, 2002).
The work sample methodology was adapted to make it more appropriate developmentally for candidates completing their first formal teaching experience. The adaptations included (a) changing the design from an instructional unit to two, separate lessons, (b) adapting scoring rubrics to reflect design changes and different developmental expectations, and (c) completing a contextual factors table with information derived from respective New York State School Report Cards. The contextual factors table required candidates to extract the following information on their respective community, district, school, and classroom: (a) demographics (number of students, gender, ethnicity, and percentages of English Language Learners, students with special needs, and pupils receiving free and reduced lunches), (b) achievement (e.g., percentage of students earning 1, 2, 3, and 4 on the Grade 4 New York State English Language Arts assessment), and (c) classroom data on the nature of rules and climate, technological resources, and extensiveness of parental involvement.
Candidates were provided with detailed prompts and scoring rubrics for completing each TLP section (Renaissance Partnership, 2002). Each TLP was submitted three times (1) after summarizing contextual influences on student learning, (2) following the completion of the first lesson, and (3) after the second lesson and integrated professional reflections. The three-stage assessment process allowed instructors to provide ongoing feedback to candidates on their performance and gave them an opportunity to see if and how such feedback was incorporated in subsequent submissions. The downside was that it substantially increased grading time, a factor that may make it less attractive to faculty members. All three parts of the TLP were included in the final submission and the products were used to generate information on candidate practice and student learning.
Data Collection Measures and Procedures
Data were collected and analyzed on six specific TLP factors: (a) contextual factors, (b) learning goals, (c) assessment plan, (d) design for instruction, (e) analysis of student learning, and (f) reflection and response. Each TLP factor consisted of two to five performance indicators. For example, the learning goals factor included these indicators: (a) significance and level of challenge, (b) clarity and measurability, (c) developmental appropriateness, and (d) alignment with state learning standards. The analysis of student learning factor was assessed in terms of (a) clarity and accuracy of data presentation, (b) alignment with learning goals, (c) accuracy of data interpretation, and (d) evidence of impact on student learning.
Each performance indicator was scored independently by faculty members using a detailed, 3-point rubric (1 = indicator not met, 2 = indicator partially met, or 3 = indicator met). Performance indicator scores were first summed within each factor for each candidate. Candidates’ summed scores were then aggregated and converted to means for each TLP factor. In addition, the percentage of candidates scoring 2 and 3 (i.e., partially met or met indicator) was calculated for each factor. These data were aggregated across each lesson and course section. Initial analyses revealed few differences between candidate performance on first and second lessons; therefore, scores were combined for subsequent descriptive analysis. TLPs were scored electronically via ANGEL grade book. Candidates received scores and direct feedback on each performance indicator and data were exported to EXCEL spreadsheets for subsequent analysis.
To examine candidate impact on pupil learning more closely, TLPs were analyzed in terms of changes in pupil performance that occurred from pre- to post-assessments. Most candidates, for example, used short (five-item quizzes) objective quizzes to assess their impact on pupil performance. For each lesson, they calculated the percentage correct for the entire class, sub-groups (e.g., students with special needs versus normally achieving peers) and individual pupils, and conducted individual item analyses. They used results from pre-assessments to design their subsequent lessons and then examined pre-post-assessment scores to determine their impact on pupil learning. To facilitate decision-making, candidates were given two options for interpreting data to determine impact on student learning. First, they could calculate learning gain scores, which are measures of how much pupils gain out of the total possible that could be gained from that pre-post assessment. Learning gain scores are calculated as: post-assessment minus pre-assessment divided by 100% - pre-assessment (Emporia State University, 2002). Second, candidates could simply determine the percentage of students who achieved mastery (i.e., 80%) on stated learning objectives in each lesson. For example, what percentage of the entire class earned post-test scores of 80% or higher on each post-assessment? A noticeable impact on pupil performance was noted when: (a) the average class learning gain scores were 0.30 or higher (30% average learning gain for the entire class), or (b) at least 75% of the entire class earned 80% correct or higher on the lesson post-assessment.
Results
During Spring Term 2007, 62 candidates provided approximately 2,480 hours of in-class assistance to predominantly high-need, regional schools. This is the equivalent of about 382 school days (based on 6.5 hours per day) of instructional assistance. During their early field experiences, candidates taught a minimum of 124 formal lessons and assessed their impact on student performance primarily through the use of short (5-item) objective quizzes. Relevant data on candidate practice and pupil outcomes are presented across course sections in Table 2.
Table 2. Mean TLP Scores and Percentages of Candidates Meeting and Partially Meeting Indicators
Course Sections
|
(01)
N = 11
|
|
(02)
N = 28
|
|
(03)
N = 23
|
|
Totals
N = 62
|
|
TLP Factors
|
Mean
|
% of
2 & 3
|
Mean
|
% of
2 & 3
|
Mean
|
% of
2 & 3
|
Mean
|
% of
2 & 3
|
Context Factors
|
NA
|
NA
|
2.82
|
100%
|
2.82
|
98%
|
2.82
|
99%
|
Learning Goals
|
2.93
|
100%
|
2.74
|
100%
|
2.78
|
96%
|
2.81
|
99%
|
Assessment Plan
|
2.41
|
82%
|
2.38
|
98%
|
2.67
|
96%
|
2.49
|
92%
|
Design for Instruction
|
2.72
|
96%
|
2.55
|
98%
|
2.89
|
97%
|
2.72
|
97%
|
Analysis of Student Learning
|
2.89
|
100%
|
2.50
|
96%
|
2.64
|
94%
|
2.68
|
97%
|
Reflection and Response
|
2.95
|
100%
|
2.49
|
98%
|
2.76
|
91%
|
2.73
|
96%
|
Totals
|
2.78
|
96%
|
2.58
|
98%
|
2.76
|
95%
|
2.70
|
97%
|
Table 2 shows that candidates earned a mean score of 2.7 (out of 3.0) across all six factors on the TLP. Similarly, approximately 97% of all candidates earned scores that either partially met or met individual performance indicators. More detailed analyses revealed very few noticeable differences across either TLP factor or course section. The highest percentage of candidates received 2s and 3s on the learning goals factor (99%), while the lowest percentage received comparable scores on the assessment plan factor (92%). The most commonly noted problem was the alignment of assessment practices with contextual factors and learning goals.
To assess candidate impact on pupil learning, average learning gain scores from 18 lessons (i.e., 9 candidates) were examined first. As seen in Table 3, only 9 candidates calculated learning gain scores for their two lessons. The average class gain score for the 18 lessons was 0.62 with a range of 0.08 to 0.88. These data suggest that, overall, pupils gained approximately 62% of all possible gain from pre- to post-assessments. Moreover, 17 of 18 (94%) class gain scores exceeded the 0.30 criterion used to determine if candidates made a noticeable impact on pupil performance.
Table 3. Average Class Gain Scores for 18 Candidate Lessons
Lesson Number
|
Pre-Assessment Class Average
|
Post-Assessment Class Average
|
Entire Class
Gain Score
|
1
|
45
|
90
|
0.82
|
2
|
55
|
90
|
0.78
|
3
|
45
|
80
|
0.63
|
4
|
20
|
60
|
0.50
|
5
|
75
|
95
|
0.80
|
6
|
60
|
90
|
0.75
|
7
|
40
|
75
|
0.64
|
8
|
45
|
80
|
0.64
|
9
|
25
|
60
|
0.47
|
10
|
40
|
85
|
0.75
|
11
|
60
|
95
|
0.88
|
12
|
50
|
78
|
0.56
|
13
|
40
|
85
|
0.75
|
14
|
65
|
90
|
0.86
|
15
|
20
|
55
|
0.44
|
16
|
20
|
60
|
0.50
|
17
|
40
|
45
|
0.08
|
18
|
45
|
83
|
0.69
|
Total
|
44
|
79
|
0.62
|
Ninety-eight lessons that included only pupils’ post-assessment scores (no learning gain scores) were then examined to determine candidate impact on pupil learning. (Eight lessons could not be scored because candidates failed to provide sufficient post-test data). Eighty-nine of the 98 post-tests (91%) indicated that at least three-fourths of the class earned grades of 80% or higher on post-tests. Overall class post-test means averaged 86.5% with a range of 66% to 100%.
Discussion
It is important that teacher educators generate credible evidence to support program and candidate effectiveness. Aside from silencing critics (hopefully), these data are critical to accreditation (e.g., NCATE, 2002), acceptance within the professional community (e.g., Schwartz, 2003), and perhaps most importantly, improving pupil learning. As Popham (2005) noted the primary mission of accountability programs is to ensure that teachers and schools provide excellent instruction so students learn better. Do our candidates provide excellent instruction? Do their students learn better? Teacher educators are just beginning to address these and many other important educational questions. Initial evidence, however, is promising.
This descriptive study contributes to the ongoing dialogue on assessing candidate practice and student learning, the systematic study of teacher education, and the preparation of highly qualified teachers. To begin, TLPs or teacher work samples appear to be good initial measures of candidate practice and its impact on student learning. Researchers (e.g., Denner, et al., 2001; McConney, et al., 1998; Renaissance Project, 2002) have found, for example, that teacher work samples are positively related (with moderate to large effect sizes) to the classroom performance of first-year teachers. These findings should not be surprising given that the assignments require candidates to carefully consider contextual influences in lesson design and implementation; to develop measurable and socially important learning outcomes; and to assess their students before, during, and after instruction. Similarly, the authentic teaching assignments require candidates to analyze their impact on all pupils’ learning and to make reasonable, data-informed adjustments in practice when indicated. Such professional skills are essential for all teacher candidates. These data must be interpreted cautiously here, however, because they involved individual lessons rather than instructional units and relied primarily upon short objective samples of pupil performance.
Candidates also commented favorably on the impact of TLPs on their professional development. One candidate noted, for example, that "my lesson on the five senses gave me more confidence in what I was doing and allowed me to share the information without being as nervous. The graphic organizer was an excellent tool for organizing information for the students and myself.” Another wrote, "I found that by teaching these two lessons I was more excited to be a teacher. I saw that I could influence my students' learning and that's what I hoped for as a teacher."
The Instructional Assistants Program (IAP) or some facsimile may also offer teacher educators an authentic context for conducting more systematic studies of their preparation efforts. Brownell, Bishop, and colleagues (2004) argued that teacher educators need more studies that describe beginning teachers’ instructional practices and link these practices to student outcomes. The IAP provides a natural context for examining candidates’ practices, and TLPs give teacher educators a mechanism for linking practice with pupil outcomes. These assignments and field placements are replicated across subsequent years of the REP with candidates gradually assuming greater instructional responsibility at each program phase.
Finally, the REP provides an interesting and unique teacher preparation model. By merging special and general education faculty into one unit with a single curriculum, engaging faculty in collaborative teaching and research endeavors, and using common applied teaching projects, the program may serve as a positive exemplar of inclusive teacher preparation (Blanton & Pugach, 2007). The intent of the REP, however, is not to supplant special education teacher preparation but rather to better prepare general educators to work more effectively with the exceptional children who will occupy their classrooms.
Obviously, much more work must be done on the IAP, TLPs, and the general assessment methods described here. For example, more direct and robust measures of candidate practice and student learning are needed. Initial data provided only a brief snapshot of what candidates did in the classroom, and individual lesson outcomes reflected a small proportion of what students learned daily. Similarly, existing data collection and analysis procedures must be tightened up substantially. TLP data, for example, were obtained indirectly through candidate submission, and TLPs were scored independently by course instructors. To ensure that data are collected and reported accurately and reliably, inter-rater reliability and blind reviews of randomly selected projects must be implemented. Similarly, there is a dire need for more experimental studies (i.e., single case or randomized control trials) that link candidate practice functionally to improvements in pupil learning. An additional limitation is the absence of generalization and maintenance data.
In summary, an early field-based course and applied teaching project were described and data on candidate practice and student learning were shared. The IAP provided an enormous amount of direct instructional assistance to teachers primarily in high-need schools and offered candidates direct and early opportunities to teach formal lessons and gather evidence to support their impact on student performance. Teacher educators must continue to improve their knowledge and practice to provide the kinds of unambiguous answers that our professional critics demand and that our students, their families, and the general public deserve.
References
Abell Foundation. (2001, October). Teacher certification reconsidered: Stumbling for quality. Baltimore: Author. Available from http://www.abell.org. Retrieved on March 22, 2005.
Ballou, D., & Podgursky, M. (2000). Reforming teacher preparation and licensing: What is the evidence? Teachers College Record, 102 (1), 5-27.
Blanton, L. P., & Pugach, M. C. (2007, June). Collaborative programs in general and special teacher education: An action guide for higher education and state policymakers. Washington DC: Center for Improving Teacher Quality.
Brantlinger, E. Jimenez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative studies in special education. Exceptional Children, 71, 195-207.
Brownell, M. T., Bishop, A., Penfield, R., Sindelar, P., Klingner, J., Menon, S., Gersten, R. Haager, D., & Diminio, J. (2004, November). Beginning teacher quality study. Paper presented at Annual Convention of the Teacher Education Division of the Council for Exceptional Children. Albuquerque, NM.
Carnine, D. (2000, April). Why education experts resist effective practice: Report of the Thomas B. Fordham Foundation. Washington, DC: The Thomas B. Fordham Foundation.
Coalition for Evidence Based Educational Policy Report (2002). Rigorous evidence: The key to progress in education. Available online at http//www.excelgov.org/displayContent.asp?Keyword=prppcEvidence.
Denner, P. R., Norman, A. D., Salzman, S. A., & Pankratz, R. S. (2003, February). Connecting teaching performance to student achievement: A generalizability and validity study of the Renaissance Teacher Work Sample assessment. A paper presented at the Annual Meeting of the Association of Teacher Educators, Jacksonville, Florida.
Denner, P., Salzman, S., & Bangert, A. (2001). Linking teacher assessment to student performance: A benchmarking, generalizability, and validity study of the use of teacher work samples. Journal of Personnel Evaluation in Teacher Education, 15, 287-307.
Emporia State University (2002). Teacher work sample assignment: Prompt and evaluation rubric. Unpublished document, Emporia State University and The Renaissance Partnership for Improving Teacher Quality Project.
Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C. R., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149-164.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179.
McConney, A. A., Schalock, M. D., & Schalock, H. D. (1998). Focusing improvement and quality assurance: Work samples as authentic performance measures of prospective teachers’ effectiveness. Journal of Personnel Evaluation in Education, 11, 343-363.
National Council for the Accreditation of Teacher Education (NCATE). (2002). Professional standards for accreditation of schools, colleges, and departments of education. Washington, DC: NCATE.
Odom. S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137-148.
Popham, W. J. (2005). All about accountability/instructional quality: Collecting credible evidence. Educational Leadership, 62(6), 80-81.
Pugach, M. C. (2005). Research on preparing general education teachers to work with students with disabilities. In M. Cochran-Smith and K. Zeichner (Eds.), Studying teacher education: The report of the AERA Panel on Research in Teacher Education (pp. 549-590). Mahwah, NJ: Lawrence Erlbaum Associates.
Renaissance Partnership for Improving Teacher Quality. (2002, June). Teacher Work Sample: Performance prompts, teaching process standards, and scoring rubrics.Western Kentucky University. Available from http://fp.uni.edu/itq.
Schwartz, I. S. (2003). Social-validity assessments: Voting on science or acknowledging the roots of applied behavior analysis? In T. Stokes & K. Budd (Eds.), A small matter of proof: The legacy of Donald M. Baer. (pp. 53-62). Reno, NV: Context Press.
Shulman, L. (2002). Truth and consequences? Inquiry and policy in research in teacher education. Journal of Teacher Education, 53(3), 248-253.
US Department of Education. (2002). Meeting the highly qualified teachers challenge: The Secretary's annual report on teacher quality. Washington, DC: U.S. Department of Education, Office of Post-secondary Education, Office of Policy, Planning, and Innovation.
Wilson, S., Floden, R., & Ferrini-Mundy, J. (2002). Teacher preparation research: An insider’s view from the outside. Journal of Teacher Education, 53(3), 190-204.
Author Biographies
Larry Maheady, Ph. D., is Professor and Chair of the Department of Curriculum and Instruction at SUNY Fredonia. His research interests include improving the academic performance of all students in inclusive educational settings, peer-mediated learning strategies, and preparing highly qualified teachers for 21st Century schools. Email: Maheady@fredonia.edu.
Michael Jabot, Ph. D., is Professor in the Department of Curriculum and Instruction at SUNY Fredonia. His primary responsibilities include teaching preservice and in-service teachers, conducting classroom-based research, and consulting with school districts locally and nationally. His research interests include the preparation of high quality science education teachers and applied educational research. Email: Michael.Jabot@fredonia.edu.
Janeil Rey, M. ED., is Visiting Assistant Professor in the Department of Language, Learning, and Leadership at SUNY Fredonia. Her primary research interests include school leadership, educational administration, curriculum design, and supervision and evaluation of instructional practice. Email: Janeil.Rey@fredonia.edu.
Hidden in Plain Sight: The Invisibility of Homophobic Harassment and Girl-on-Girl Bullying in Schools
Barbara Garii
SUNY Oswego
Oswego, New York
Suzanne SooHoo
Chapman University
Orange, California
Abstract
The recognition of the negative effects of bullying is a major concern within K-12 schools. However, not all bullying is recognized as serious or problematic. Educational professionals are in the best position to help change school environments (Kosciw & Diaz, 2006) yet few appreciate the seriousness of girl-on-girl or homophobic bullying. Ninety- seven preservice teachers in two states who had completed anti-bullying training as part of their course work participated in allied studies exploring recognition of and responses to these events. Results suggest that while they were able to articulate and discuss these bullying events, they failed to recognize and acknowledge these events in school settings. The preservice teachers, many of whom were recent high-school graduates, reported high-school experiences that may have clouded their ability to recognize and respond to inappropriate behaviors; their own high-school experiences may have desensitized them to the language and actions of homophobic and girl-on-girl bullying.
The recognition of the short- and long-term negative effects of harassment and bullying is acknowledged as a major concern within K-12 school environments. Although the terms bullying and harassment will be used interchangeably in this article, some researchers use these terms to describe slightly different, although related, events. Student and teacher responses to harassment and, by extension, bullying are addressed in schools both explicitly and implicitly through specific curriculum and policies (American Association of University Women [AAUW], 2001; Crothers, Kolbert, & Barker, 2006; Hallford, Borntrager, & Davis, 2006; Harris, Petrie, & Willoughby, 2002; Saufler, 2004), as well as professional development and preservice teacher education modules, all of which emphasize incorporation of social justice, multicultural acceptance, and community spirit (Nieto, 2006; Shariff, 2004). These formal and informal policies and programs address concerns about racism, sexism, ableism, class, ethnicity, culture and, at times, homophobia. These policies encourage both students and teachers to recognize, acknowledge, and intervene when they observe any type of harassment.
The role of teachers as interveners, who both create and uphold a supportive environment for all students, is increasingly recognized as crucial to the overall school climate. Evidence suggests that, as teachers take a more active stance towards school safety for all, students become better able to recognize and then report these events (Crothers et al., 2006; Hallford et al., 2006). However, not all instances of harassment and bullying are understood as such by teachers or other adults (Hallford et al., 2006; Naylor et al., 2006). For example, although incidents of racism and sexism are easily acknowledged, often by the specific words used by the attacker to frame the event, other incidents, such as homophobic harassment and girl bullying, engender both fewer and lesser responses from responsible adults. This is perhaps due to choice of words used by the attacker or to the perceived “normalcy” of the behaviors. Such bullying becomes essentially invisible to teachers and, often, the students themselves, although all recognize the discomfort associated with these attacks (Kosciw & Diaz, 2006; Osler, 2006).
For the past several years, the authors of this paper have been discussing the role of the responsible adult in the classroom as an independent variable in the response to harassment or bullying. Together and separately, we have considered how adults respond to a phenomenon they intellectually acknowledged as problematic but have addressed infrequently. In this paper, we explore how preservice teachers responded to homophobic harassment in a K-6 practicum placement and consider how their responses mirror teacher reactions to similar invisible and/or unacknowledged girl-on-girl bullying events. Our results suggest that invisible or unacknowledged harassment raises many personal issues for these young adults, which ultimately impedes their ability to successfully respond and intervene when these events occur. We also suggest that it is not enough to raise teachers’ awareness of the precursors and types of harassment and bullying that occurs in schools. Instead, we must help teachers recognize that school behaviors that mirror the events that occurred in their own youth are not necessarily acceptable. Further, we must support teachers and preservice teachers as they come to terms with their own past behaviors and indiscretions. And finally, this essay is a call to action by teachers and other school personnel to recognize and act on the behalf of bullied victims by recognizing the cultural conditions in school that affirm victimization and harassment.
Literature Review
Although all harassment events share a number of characteristics, the specific manifestations and long-term effects of girl bullying and homophobic harassment (i.e., incidents targeted toward alleged and identified lesbian, gay, bisexual, transgendered, and queer [LGBTQ] youth) raise additional issues that are specific to these communities. Thus, while it is important to recognize the similarities among all harassment situations, it is equally necessary to differentiate between them to acknowledge that teachers will have biases regarding specific bullying or harassment events based on their own histories and school conditions.
Share with your friends: |