Request for Tender for The evaluation of Barnardo’s Northern Ireland, ‘Ready to Learn Programme’
Download 161.91 Kb.
|
- Navigate this page:
- 2.4 Evaluation Overview
- Research Design Methodologies
- 3.1 Guidelines for Submission
2. Background2.1 Barnardo’s Northern Ireland All of Barnardo’s work is driven by the vision that the lives of children and young people should be free from poverty, abuse and discrimination. Our purpose is to help the most vulnerable children and young people transform their lives and fulfill their potential. Our basis and values inform all aspects of our work. These are expressed through: Respecting the unique worth of every person Encouraging people to fulfill their potential Working with hope Exercising responsible stewardship To work to these values, achieve our priorities and deliver better outcomes for children and young people we evidence all our work through six building blocks of every child’s right to: A loving and supportive family Belonging in the community Protection from harm A stake in society Opportunities to learn Emotional, physical and mental health Barnardo’s believes that education is pivotal to a child’s life chances, opening opportunities for further education, training and employment. The central premise underpinning the Ready to Learn programme is that many children’s opportunities to achieve positive outcomes are blocked or restricted by personal or social disadvantage. Most schools lack the resources or expertise to address the wider needs facing many of their pupils. Ready to Learn aims to work in collaboration with schools to provide a preventative programme to enable children to engage in the school curriculum and encourage their parents / carers to support their child’s learning. ‘Ready to Learn’ Programme Background There is considerable evidence demonstrating underachievement amongst children from disadvantaged communities in Northern Ireland. The topic has been debated within the Northern Ireland Assembly and a NI Audit Office Report (2006) highlighted that despite considerable investment in literacy initiatives, improvements in children’s achievement were not found. Recently the Department of Education has developed a comprehensive policy ‘Every School a Good School’ to promote greater equality of experience and opportunity across schools in Northern Ireland. A strategy to promote positive outcomes in Literacy & Numeracy is scheduled for publication later this year. In the main, efforts from Government to address the achievement gap have focused on school based strategies and programmes. However, research clearly demonstrates that schools cannot tackle or solve the complex issues leading to underachievement on their own. Indeed, many of the contributory factors are in place before a child reaches school. By the age of three years old children from disadvantaged areas may be a much as one year behind their peers from more affluent backgrounds. Too many children arrive in school not ready or able to learn, presenting schools with a considerable challenge. This challenge continues as children progress through school and beyond, resulting in children from disadvantaged areas being over-represented amongst those failing to achieve key literacy and numeracy skills at the end of Primary School, leaving school early with few or no qualifications and subsequently struggling to access training or employment. In late 2007 Barnardo’s Northern Ireland received a development grant from the Atlantic Philanthropies to explore through research, consultations and policy work what contribution Barnardo’s could make to tackling underachievement. Barnardo’s commissioned research from the Centre for Effective Education, Queens University to look at the following (full document available from Barnardo’s on request) - Provide quality baseline data on the distribution of educational outcomes and indicators of well-being among Year 4 and Year 7 children in areas of high disadvantage and areas of low disadvantage. Determine how these outcomes relate together and what socio-demographic variables might be associated with these. Identify levels of need in relation to the outcomes in order to aid Barnardo’s in deciding where and how best to target the intervention. The subsequent research included a survey of 1081 Year 4 and Year 7 pupils in 28 Primary Schools randomly selected across Northern Ireland. The key results of the survey highlighted the following: 1) Year 4 children from disadvantaged areas reported a greater sense of well-being but achieved a lower level of educational attainment than Year 4 children attending school in areas of low disadvantage. 2) There were few differences in well-being between Year 7 children from different areas, however Year 7 children attending school in areas of high disadvantage had poorer educational attainment than children from areas of low disadvantage. When the relationship between educational attainment and the indicators of well-being for all children it emerged that: 1) Year 4 children’s educational achievement was largely unrelated to well-being with the exception of those children who had better peer relationships and higher aspirations had poorer achievement in English. 2) A clearer picture emerged for Year 7’s which indicated that higher attainment was related to less bullying, more enjoyment of school and a greater sense of physical well-being. When a more sensitive measure of deprivation was used in the analysis a similar pattern emerged. 1) Attainment in both Year 4 and Year 7 was lower in the more deprived areas. 2) Year 4 children who attended school in more deprived areas reported an overall better sense of well-being. Children’s right to have their views given due weight in all matters affecting them is considered to be a cornerstone of the UNCRC. As part of their efforts to reinforce the status of the child as a active participant in the promotion and protection of their rights Barnardo’s also engaged the team from QUB to directly engage with children as part of a service design process. To this end, the team consulted a group of children creating a Children’s Advisory Group (CAG). The children in the CAG provided interesting insight into the survey findings and also their perspectives on the factors affecting children’s ‘readiness to learn’. Conclusion The work carried out during this development stage clearly indicated that any intervention or service targeted at raising achievement should be explicitly focused on educational outcomes rather than indictors of well-being. Given that a significant gap in achievement is already evident by Year 4, the intervention should be targeted at an early stage in their schooling. Reviews of literature and the range of stakeholder voices also advocated strongly for a service extending learning opportunities for children at risk of underachieving beyond the school day. Furthermore it was agreed that an element of the service should be focused on building parental capacity to support their child’s learning, encouraging their child’s natural curiosity and better equipping parents to help children with school work at home. The family and community environment of a child plays a significant role in shaping a child’s capacity and motivation to learn. The parental role in promoting language and cognitive development essential to engage and enjoy in learning is crucial. Where parents are overwhelmed with other pressures such as poverty, mental or physical ill health and lack positive sources of family support, the parental skills to nurture and support a child’s school readiness and development are often missing or inconsistent. It was strongly felt that being a voluntary organization, Barnardo’s are well placed to engage and support parents, particularly those with whom schools may struggle to reach. Taking on board the full range of research and consultation undertaken throughout the development phase, parental engagement in children’s education and increasing opportunities to learn during out-of-school-hours emerged as key issues to be addressed to improve the achievement of children from disadvantaged areas. Following from this the key elements driving the overall service design to significantly increase the educational achievement of children participating in the project are: Enhancing the value placed on education by family Providing educational opportunities outside ‘class time’ Building the capacity of parents to assist their children in learning 2.3 Ready to Learn Programme Description ‘Ready to Learn’ is a Literacy Programme with two distinct but related strands of delivery. The work with parents and children will share a focus on building the key literacy and other skills needed to help children engage in the school curriculum. Both strands will build on the Foundation Stage Curriculum to compliment the work children are doing with teachers during the school day. Family Literacy - Extending Learning Opportunities by Supporting Parents The Family Literacy strand of Ready to Learn will provide a programme of parental engagement, specifically designed for parents / carers with a child in their first year of school. There will be careful attention given to engagement and retention of parents and methods used may include home visiting, individual support and group work activity. The programme’s explicit aim will be to encourage and support parents to take on a meaningful and effective role in engaging with their child’s education. Parents from disadvantaged communities want their children to succeed at school and beyond but often lack the knowledge, skills and resources to make this happen. The Family Literacy strand of Ready to Learn will directly address this issue and provide ongoing support to parents / carers during the first three years of their child’s primary education. After-School Literacy Programme - Extending Learning Opportunities by Supporting Children Research shows that children from disadvantaged communities arrive at school already behind their more affluent peers, a gap that not only continues but expands as they progress through school. Using learning opportunities beyond the school day can offer a chance to start to address this deficit. Ready to Learn will provide an innovative high quality after-school programme to stimulate and nurture the school readiness skills needed to do well at school. Starting with children in their first year of Primary School, an enriching and exciting three year programme of learning opportunities will be provided to focus on key literacy skills. Careful attention will be spent to complement and support the work being done within the curriculum. During the first year, the ‘Talking and Listening’ element of the Foundation Stage curriculum will provide the core structure. Children will be encouraged and supported to develop the verbal, cognitive and behavioral skills needed to develop good early literacy skills. The programme will be delivered three afternoons per week throughout the school year (30 weeks) for one hour. All children in Primary 1 in participating schools are eligible to take part. 2.4 Evaluation OverviewThe programme will start in September 2010 and will run for three consecutive years following the same cohort of children and parents / carers. The programme will be evaluated robustly in order to provide clear evidence of any impact that the programme may make on the educational outcomes of disadvantaged children and young people. The evaluation will use both a qualitative and process evaluation alongside a random control trial experimental methodology. Tenders should specify the sample sizes needed to robustly evaluate the programme, including appropriate power analyses, within the allotted budget (detailed on page 12). This will be overseen by an International Advisory Committee of local and international experts in educational policy, prevention science, children’s literacy and evaluation (see appendix for more information). The evaluation brief will be subject to approval by The Atlantic Philanthropies. 2.5 Methods It has already been agreed that a Randomised Control Trial will be used to evaluate the ‘Ready to Learn’ Programme. A total of 16 schools have already been selected and agreed to engage in the programme and the experimental research design. It is envisaged that approximately half of Primary Schools will recieve the Ready to Learn Programme and the other half will act as a control group. The randomisation process has been commissioned with NWORTH, the Clinical Health Trials Unit at the University of Bangor in Wales. This will be completed in April 2010. The Ready to Learn Team aim to engage between 150 - 200 Primary 1 children in the After-School Programe by October 2010. The engagement of parents in the parenting component will be a more fluid arrangement necessitating a longer lead in period to build relationships and trust between parents and the Programme Team. All children and families will continue to recieve existing services as usual. Tenders should specify the sample sizes needed to robustly evaluate the programme within the alloted budget (detailed on page 12). The evaluation tender should also involve relevant school personnel (for example Principal, Foundation Stage teachers) and Barnardo’s staff in the evaluation process. The range of methods and measures used in evaluating the ‘Ready To Learn’ programme will be determined by the Contractor selected to undertake the work in agreement with Barnardo’s and the Ready To Learn International Advisory Committee. Barnardo’s Ready To Learn staff will work closely with the evaluation team to identify the most critical outcomes to be measured and tracked. Key evaluation questions Outcome Measurement Does the Ready to Learn Programme achieve the outcomes set out for children and parents in relation to increased literacy skills, greater parental involvment in education and higher levels of achievement? Over-arching outcome the educational attainment of the children in each school in the programme will have improved by a significant margin Outcomes for children Short-term Outcomes (Year 1) For the first year of the programme these will reflect the ‘Talking and Listening’ component of the Foundation Stage Curriculum with a focus on children developing the following skills and making progression in the following key outcome areas: understanding concepts and beginnings of print phonological awareness oral language and social use of language vocabulary attention and listening It is also planned to incorporate a social curriculum to promote social, emotional and behavioral regulation skills into the literacy curriculum to facilitate development of literacy skills. The evaluation tender should outline proposed steps to detect any changes in these indirect outcomes also. Medium-term planned outcomes (Years 2 and 3) Specific outcomes for Years 2 and 3 will be developed as the service design process progresses but will retain an explict literacy focus, linked to progression in key literacy areas of the Foundation Stage Curriculum. Outcomes for the work with parents Short-term outcomes (Year 1) significant increase in parents attending parent-teacher meetings (including normal school meetings, programme specific meetings and informal parent-teacher discussions) parents reporting increased confidence in relation to supporting children and communicating with schools children reporting increased participation by parents in their education creation / maintenance of positive home learning environments Medium-term planned outcomes (3 years) parents reporting substantial improvements in relationships and co-operation in children’s learning teachers reporting substantial improvements in relationships and co-operation in children’s learning (e.g. child related perceptions, time spent on managing behavior, sense of progress, stress levels etc) increase in parents volunteering to support the school (and other parental engagement) increase in active participation of parents in schools’ decision-making processes increase in parents actively participating in adult learning children’s education performance improving Implementation In addition to measuring the impact of the programme, the evaluation will examine the implementation in terms of fidelity and quality. This includes measures of: adherence, exposure (dosage), quality of programme delivery, participant responsiveness and programme differentiation. Proposals should indicate how the study will measure if they have been attained. To what extent the manual specifications are applied in practice across the 8 school sites. An exploration of factors explaining the degree of fidelity. Research Design & MethodologiesProposals should consider the best methods and measures for the evaluation and the limits of any one method or data source. In addition, a data analysis plan should follow the presentation of the methods and measures. Data collection and analysis plans should reflect the requisite timeframe for the detection of outcomes. The proposal should also address likely challenges in implementing the evaluation and propose possible solutions. In particular, the proposal should discuss how the evaluator will work with the study participants to collect the necessary data about children and the families as well as other relevant information required for project implementation.
General information. Name, address and contact details of tenderer; confirmation of acceptance by the tenderer of the conditions of the tender described in the proposal; confirmation of acceptance that Barnardo’s may contact any previous client referenced in the proposal; C.V.s of the people who will be working on the evaluation. The C.V.s should state the person’s specific experience and training in this field and the relevant knowledge and skills they bring to the work. Evaluation Framework Design, Methodologies and Analysis Plan. Describe the proposed research/evaluation methodologies to be used to collect data and evaluate the programme implementation and outcomes. The proposed quantitative and qualitative data sources should be clearly described as should the mechanism for integrating both quantitative and qualitative data sources. An analysis plan should be presented which describes the steps taken to apply appropriate statistical tests. Proposed Products and Deliverables. Include a high-level vision about the use of the evaluation data as well as methods for providing feedback to Barnardo’s and the Ready to Learn International Advisory Committee including the products to be delivered such as databases, reports, presentations and briefings. Some consideration should be given to the format of the expected deliverables which should include: 1) A monthly memorandum from the Principal Investigator briefly outlining activities, problems encountered, expenditure, planned activities for next month, overall status of data collection and ongoing table and schedule of deliverables. 2) Quarterly updates on progress regarding various aspects of the study. Bi-annual reports on research findings (most likely January and July each year). Schedule of meetings / presentations to Barnardo’s and the International Advisory Committee High-Level Project Plan. Include schedule of deliverables as well as interim dates by which preliminary conclusions may be drawn. Data Collection & Analyses. Submissions should include a plan for addressing each of the evaluation questions in terms of how the data will be gathered and analysed (e.g., surveys, interviews etc). Qualifications. Describe the Principal Investigators and the team’s qualifications and briefly summarize the background of the firm or organization. Describe previous experience in evaluation generally and specifically in the areas of experimental evaluation design, implementation and analysis, and advocacy and policy changes evaluation. Describe your approach to project management. Staffing Plan (or others you will partner with on this project). Evaluation teams will comprise a range of expertise in order to evaluate project implementation and intervention effectiveness. Identify the project team assigned to the evaluation, including (i) CV’s, (ii) responsibilities for the project, (iii) percentage of time devoted to the project and (iv) collaboration with others, both national and international, where appropriate, to ensure there is sufficient expertise to conduct all elements of proposed evaluation. Tenderers should also identify other national or international studies with which they may develop links. Professional Fees and Proposed Total Costs. Prepare a budget, including incidental costs, to achieve deliverables outlined in Research Design and Methodology. The budget should reflect all evaluation costs. References. Provide references from two clients. Ownership of Data. All intellectual property rights arising from the results of the Evaluation / Research Study, including reports, databases and other materials shall be the property of Barnardo’s Northern Ireland and The Atlantic Philanthropies. The successful tenderer will be required in the contract to assign all intellectual property rights created as a result of the Study to Barnardo’s Northern Ireland and The Atlantic Philanthropies. Barnardo’s Northern Ireland will remain the sole owner of all data (micro level, aggregate and metadata) and end-products, e.g. instrumentation, reports, studies and any other related items or documents, irrespective of whether or not the project is terminated prior to its completion. An exception to this is where it is agreed in advance that such documents / publications / reports are produced jointly with Barnardo’s Northern Ireland, in this case the property rights are held jointly. This exception includes scholarly publications which can be submitted with approval from Barnardo’s Northern Ireland. It is intended, subject to data protection requirements, to make data from the Study widely available for research and / or other purposes (and it is acknowledged that The Atlantic Philanthropies and their agents will have the same licence to us all such data and related intellectual property as the Contractor). 3.2 Costs It is envisaged that that the total budget available for the three year evaluation will be within the range of £325 000.00 - £425 000.00 (inclusive of VAT). The quotation for the tender shall set out as follows: A daily rate for each of the staff to be involved in the project. VAT amounts at the appropriate rates should be shown separately. A total cost for the project with VAT indicated separately must also be provided. The total cost and the costs for each item must be stated in Sterling. The price stated must be the total all-inclusive price for the duration of the tender. Price increases during the term of the contract will not be accepted. The cost of the tender must be the individual’s / organisation’s best and final offer. 3.3 Management and Administration Structure Please provide details of the arrangements and systems which the tenderer will put in place to manage this contract and to liaise and communicate effectively with Barnardo’s Ready to Learn. In the event either an organization or a consortia tendering please indicate who will be the Principal Investigator with responsibility for the contract and his / her position within a specified organization. General Process InformationTerms of Reference Modifications. Barnardo’s Northern Ireland reserves the right to initiate modifications to the Terms of Reference should this be deemed necessary. Application Procedure: Applicants are invited to submit a full proposal for this evaluation. Full proposals are due by noon, 1st June 2010 in both electronic and hard copy formats. Electronic versions should be emailed to Jenny Gribben, jenny.gribben@barnardos.org.uk Selection Criteria: Selection will be based on: Quality, rigor and relevance of the proposal (Project plan, methodologies, analysis etc) Composition and engagement of the project team (Subject areas, level of experience, continuity of staff etc) Evidence of excellence and impact of prior work Cost / value of money Ethical & Other (Informed consent, confidentiality, data protection, ethics procedures, child protection issues etc) Availability of team to commence work immediately after contract is awarded. Barnardo’s is not bound to accept the lowest tender or any tender offered. Review Process: Proposals received will be reviewed by Barnardo’s Northern Ireland and the Ready to Learn International Advisory Committee and any other agency they may consider relevant to the review process. Right of Rejection. Barnardo’s Northern Ireland and the Ready to Learn International Committee reserve the right to reject any or all Proposals submitted. Acceptance of Proposal. The contents of the proposal of the selected consultant will become part of the contractual obligations if agreement is reached. Cost of Preparation of the Response. Costs incurred by any agency in the preparation of its response to this call for proposals are the responsibility of the responding agency and will not be reimbursed by Barnardo’s Northern Ireland. Clarification of Responses. Barnardo’s Northern Ireland and the Ready to Learn International Advisory Committee reserves the right to obtain clarification of any point in a proposal or to obtain additional information necessary to properly evaluate a particular proposal. Failure of a vendor to respond to such a request for additional information or clarification could result in rejection of that response or responses. Interviews. As part of the reviewing process, tenderers may be invited to participate in an interview. APPENDIX 1 Barnardo’s Ready to Learn International Committee Membership as of April 2010. Mark Dynarski Vice President; Director, Center for Improving Research Evidence Mark Dynarski is vice president and director of Mathematica's Center for Improving Research Evidence. He also directs the What Works Clearinghouse for the Institute of Education Sciences of the U.S Department of Education. Dynarski is a nationally recognized expert in evaluation methodology, including the design, implementation, and analysis of evaluations of education programs using random assignment and quasiexperimental designs. His focus is on experiments to study the effectiveness of educational and social programs for children and youth. He has played a leading role in many of Mathematica’s elementary, secondary, postsecondary, and adult education studies, including a congressionally mandated national study of education technology and the largest evaluation ever conducted of dropout prevention programs funded by the federal government. He also directed the firm’s evaluation of the 21st Century Community Learning Centres. Mark Dynarski publishes widely in peer-reviewed journals and presents findings at conferences of researchers, policymakers, practitioners, and journalists. He is a member of the National Research Council committee on Evaluating Effectiveness of the National Board of Professional Teaching Standards certification program. He is also an associate editor of Educational Evaluation and Policy Analysis and Effective Education. He has a Ph.D. in economics from the Johns Hopkins University. Dr Daniel Perkins Daniel F. Perkins (Ph.D., 1995, Michigan State University) is an Associate Professor and Extension Specialist within the Agricultural and Extension Education Program of the Pennsylvania State University. Prior to coming to Penn State in 2000, he worked in the Family and Child Ecology Department at Michigan State and the Family, Youth, and Community Sciences Program of the University of Florida. His research area is family and youth resiliency and policy, with a focus on strengthening the capacity of families, youth, and communities to be partners in building strong families, fostering positive youth development, and creating caring, safe, and productive communities. Daniel provides advice and support to a number of Atlantic Philanthropies projects and recently worked with the Social Research Institute at Dartington where he led on designing the RTCS for Birmingham’s Brighter Future Strategy.
Dr. Shanahan is an internationally recognized reading researcher with extensive experience with children in Head Start, children with special needs, and children in inner-city schools. Download 161.91 Kb. Share with your friends:
|