d) Creation of a Project Logic Model and Elaboration of the Evaluation Framework
(July-August 2006)
The promising practices identified in this will provide the basis from which to review the IPECPCP implementation and evaluation plan. Once key insights from the better practices perspectives have been incorporated into the project plan, a logic model for this initiative will be created. The logic model will provide a schematic overview of the key components of the proposed project plan, including: theoretical assumptions, goals, resources (inputs), participants, outputs, and outcomes (immediate, short-term and long-term). Upon completion of the logic model, it will be submitted to project authorities for their consideration and review. Feedback will then be incorporated and the logic model will be finalized.
The accepted framework will subsequently be used to structure and elaborate the planned implementation and evaluation activities of the proposed project. It is anticipated that the finalized evaluation framework will examine three key areas related to the program’s development, delivery and outcomes. These include evaluation of the stakeholder training and capacity-building phase, the evaluation of the effectiveness of the actual project implementation, as well as evaluation of the outcomes realized as a result of the delivery of the interprofessional educational initiative.
e) Development of Evaluation Instruments (July-August 2006)
During the later part of this phase, work will also be undertaken to design the data- collection instruments that will be used in the various evaluation phases of this initiative. Identified better practices related to the evaluation of interprofessional programs gleaned from the literature scan and key experts will assist in guiding both the format and content of the evaluation instruments and data-collection protocols. Ongoing deliberations with Project Authorities will be undertaken to ensure that approaches to data-collection and inquiry are ethical and sensitive to the needs of students and faculty members who will participate in the implementation of this educational initiative. Evaluation instruments will be developed for the three evaluation components of this initiative: the faculty development evaluation, the process evaluation, and the outcome evaluation. The final deliverable for this phase will entail submission of the various evaluation tools to Project Authorities for their consideration. Feedback from this review will be subsequently incorporated, and the final evaluation instruments and methods will be finalized.
At the close of this phase, University and Regional Health Authority Ethics forms will be completed for the remaining evaluation phases. This will also entail outlining the specific data-collection and ethics procedures required to meet the standards defined by the Tri-Council Policy Statements on Ethical Conduct for Research Involving Humans. Once ethics clearance has been received, then the subsequent evaluation phases will be undertaken.
Evaluation Phase II: Faculty Development Evaluation (September 2006 to August 2007)
From September 2006 to December 2007 faculty members and health educators from participating universities and colleges will attend a series of orientation sessions related to the development and delivery of interprofessional educational programs. These training workshops will be designed not only to acquaint participants with current practise related to interprofessional training approaches, but also to prepare them for engagement in subsequent curriculum planning/development activities to be carried out between January 2007 and August 2007. Overall, the purposes of training and eliciting the participation of faculty participants are to enhance their professional capacity and readiness to implement effectively the proposed interprofessional educational program in the fall of 2007.
It is anticipated that a pre-post survey approach will be employed to evaluate participants’ perspectives regarding the impact of the training and curriculum development sessions on enhancing their readiness to implement the proposed interprofessional educational program. Areas of inquiry will include investigation of knowledge of essential content, skills acquired or strengthened, the nature of professional attitudes and working relationships, and perceived confidence. In addition, participants’ satisfaction with the training and curriculum development process will be examined, including lessons learned as a result of their involvement in this aspect of the initiative.
The outcomes of the pre-post surveys will be coded and entered into the project database. Basic statistical analyses and content methods will be used to investigate potential changes in faculty capacity/readiness (knowledge, skills, attitudes, and working relationships, confidence) to implement the interprofessional educational program. Upon completion of these analyses, a concise summary report of the findings will be submitted to Project Authorities.
Evaluation Phase III: Process Evaluation (September 2007 – December 2007)
This aspect of the evaluation will examine the effectiveness of the implementation of the initiative. Key areas of inquiry might include, but would not be limited to:
Was there consistency between the project’s intent and the activities of the initiative?
Did the initiative effectively reach and engage student and faculty partipcants?
Did participants perceive that the objectives of the project were met?
Were participants satisfied with the implemented activities of the project?
What were participants’ perceptions regarding the overall impact of this initiative?
What specific challenges were encountered in the implementation of the program? How were these addressed and how did this impact program delivery?
What lessons learned were identified as a result of the implementation of this initiative?
What specific developments might be considered to enhance the effectiveness of subsequent applications of this capacity-building model in other jurisdictions?
It is anticipated that an evaluation questionnaire will be administered to participants at the close of each educational session. Questions will include a range of rating scale and open-ended items, designed to examine participants’ perspectives regarding the implementation of the various initiative activities. For this aspect of the evaluation, two process evaluation questionnaires will be developed, one for use with students and the other to be administered to faculty members/health educators. In addition to the session evaluation questionnaires, four focus groups will also be carried out with project participants towards the end of the project implementation period. Two of these sessions will be carried out with students, whereas the remaining ones will involve faculty members/health educators. Areas of inquiry will be similar to those used in the questionnaires; however, these group sessions will focus on gathering perspectives related to the overall/global implementation of the initiative.
Consistent with the previous phase, the results of the post-session questionnaires and focus group sessions will be coded and entered into the project database. Basic statistical analyses and content methods will be undertaken to investigate data related to the implementation of the interprofessional educational program. Upon completion of these analyses, a concise summary report of the findings of this aspect of the evaluation will be submitted to Project Authorities.
Evaluation Phase IV: Outcome Evaluation (September 2007-June 2008)
This aspect of the evaluation will examine the accomplishments and outcomes resulting from the implemented interprofessional education program. Particular attention will be given to documenting potential changes in students’ knowledge of essential content, acquired skills, professional attitudes, working relationships, and confidence related to interprofessioanl team efforts and delivery of patient-centred healthcare. It is envisioned that baseline data for these areas of professional functioning will be gathered at the outset of the program. Subsequent data-collection periods will be undertaken at mid-point in the program, at the close of the implementation phase, and three month following the termination of the program. It is anticipated that a range of data-collection methods will be employed for this evaluation phase, including administration of baseline and follow-up self-report surveys and problem-solving test situations that incorporate scenarios relating to interprofessional health team efforts and delivery of patient-centred health care. The possibility of identifying a suitable comparison group for this aspect of the evaluation will also be explored. If it is deemed feasible to include a comparison group, then the data-collection activities completed with the treatment group will also be carried out with the participants from the comparison group during the same time period.
The results of the self-report surveys and the problem-solving test situations will be coded and entered into the project database. Inferential statistical analyses and content methods will be undertaken to investigate the data associated with the outcomes of the interprofessional educational program. Upon completion of these analyses, a concise summary report of the findings of this aspect of the evaluation will be submitted to Project Authorities.
Throughout the curriculum delivery component of the project, each of the partner organizations will apply their normal evaluation procedures for instruction quality assurance.
9. Knowledge transfer, networking and dissemination
Information about the project and its outcomes will be disseminated through publication in scholarly journals and through national (e.g. CASN, ACCC) and international (e.g. Congress of Health Professions Educators) conferences. Conferences play a particularly important role in dissemination of the results as this provides an opportunity to network with others working on interprofessional education for patient centred care.
In addition the partners will seek to develop curriculum and faculty development materials which may be transferred to another site. Training in the use of these materials will also be developed. It is anticipated that this will occur after the assessment of the pilot project proposed here has been completed.
As previously described, a continuing education programme will form part of this project and this will provide direct knowledge transfer into the sites where undergraduate students receive clinical experience.
10. Sustainability
The components of the program once established will create a sustainable education experience that can be built into our current curriculums. With planning it will be possible to cycle all of our students through the program. It is very difficult to organize team experiences that truly represent the practice environment. Well-designed simulations and workshops with multiple perspectives brought into the discussions will provide valuable learning experiences for students. The simulations and the descriptions of our experience will be available to the many other health care education programs that are in similar settings.
Following assessment of the pilot project we anticipate that the interprofessional education experiences and related preparation will be integrated into the curriculum of the partner programmes in a manner similar to that of the pilot project proposed here. All programmes would require students to take the interprofessional simulations and workshops to complete the interprofessional education component. We also anticipate that the interprofessional education simulations and workshops will become a regular part of the programme for other allied health profession students through the Bachelor of Health Sciences (BHS) programme (UNB). MD undergraduate students and residents will access the programme as part of their interprofessional education programme. AHSC will maintain the programme for in-practice professionals as part of their continuing education programme.
Principal increased costs will be those directly involved with the capstone workshop and will include the costs of including patients (simulated and real) in the simulation and costs of using in-practice professionals with interprofessional education experience as facilitators. As a sustainable alternative, web based simulations and provision of workshop materials will be investigated as to cost effectiveness and pedagogical value. Certain web based components are outlined in the work plans and include simulated interaction with the electronic health record system that supports interprofessional team approaches (see objective 3).
Atlantic Health Science Corporation is committed to the patient centred care model and sees the recruitment of staff with experience in inter-disciplinary health care as a key component in its human resources strategy. Institutional commitment to collaborative practice for patient centred care must be supported by an appropriate education programme for staff who have not received training in this area.
Appendix: Project Summary
Project Title: Interprofessional Education for Collaborative Patient-Centred Chronic Disease Care
Please provide a brief description of the proposed project (maximum 350 words) in the space below or on a separate sheet. (If the latter, order the description according to the following.)
1. Mandate of the primary applicant organization
The University of New Brunswick (UNB) is a national comprehensive university providing high quality and innovative programmes in both education, including health education programmes.
2. List of partners with whom the organization will work on this project and their roles
The educational partners (Dalhousie University Faculty of Medicine, New Brunswick Community College (NBCC) will provide core groups of learners from their health education programmes as well as providing additional expertise in curriculum development and evaluation.
The practice site partner, Atlantic Health Sciences Corporation (AHSC) through the St. Joseph’s Community Health Centre, will provide access to health care professionals working in a Collaborative Practice model. Other areas within AHSC where students receive practice setting experience will be prepared for IPECPCP. In addition, patients with diabetes will be recruited from the diabetes teaching unit for initial consultations on chronic disease health care.
3. Objectives of the project
The overall objective of the project is to develop a model of health care education which will equip students to work in interprofessional teams in patient-centred practice, through simulated care experiences for patients with chronic disease in various stages of the disease.
Specific objectives are (see the following work plans):
Objective # 1: To facilitate and increase the capacity for health educators to deliver the interprofessional education model (IPECPCP).
Objective # 2: To increase the competencies of students and health professionals across disciplines to deliver interprofessional health care.
Objective # 3: To provide opportunities for students and health professionals across disciplines to apply their IPECPCP program on interprofessional team work.
Objective # 4: Evaluation of strategies for an effective IPECPCP programme and identification of better practices
4. Major activities required to achieve these objectives are indicated in the following work plans
5. Expected results of the project
The project is expected to demonstrate the benefits of IPECPCP and to increase the numbers of professionals trained in collaborative patient centred practice, with emphasis on the benefits of collaborative practice along the continuum of care along the trajectory of chronic disease.
For specific indicators of success see the following work plans.
6. Methods that will be used to evaluate both the process and the outcomes of the project
Qualitative and quantitative data will be collected from learners, patients, and educators.
For specific evaluation methods see the following work plan for objective 4
7. List of the project deliverables with timelines
Deliverables include
IPECPCP literature survey (Phase I August 2006)
Faculty Development workshop materials and evaluation framework (Phase II.1 December 2006)
Interdisciplinary practice scenarios (Phase II.2 April 2007)
Curriculum schedule and materials for implementation, and summary of evaluation results for phase II (Phase II.3 August 2007)
Logic model and evaluation materials for IPECPCP implementation (Phase III December 2007)
Professional Development and Curriculum Packages, presentation of findings to stakeholder groups, conference presentations, and final report (Phase IV June 2008)
Timelines and deliverables for the objectives and actions are indicated in the attached work plans.
8. Dissemination plan (including to whom, when and how the information will be disseminated).
The findings from this pilot project will be disseminated through the following:
Presentation of findings to patient, educator and learner groups, and to senior administration of the partner institutions.
National and international Conference presentations and submission of articles to scholarly journals.
Dissemination of the final report, including posting of the report on the websites of the partner institutions.
Share with your friends: |
