Office of Academic Assessment and Program Review, 2014
Purpose
Program review ensures the effectiveness and currency of established degrees. The review process helps strengthen the quality of UVU’s degree programs and ensure that they are cost-effective and address regional work force needs. Program review and possible redesign of degrees is essential in maintaining vibrant and responsive academic programs. According to USHE policy 411, all programs are reviewed every 7 years. See http://higheredutah.org/wp-content/uploads/2013/08/R411.pdf.
Process
Report Preparation General Guidelines
Program reviews are submitted in Insight – go to the planning module and then the program review tab.
Criterion 1, 8, and 9 should be addressed at the department level and reflect all degree programs – bachelor, associate, minors, emphases, etc. Criteria 4-7 should be completed for each degree program within the department.
Write a succinct response for each of the criterion. Responses should not be more than 250-300 words.
Use the guiding questions in the i-buttons for each criterion to direct your response. Be sure to respond specifically to the questions. If there is a reason you are unable to respond to an area within a criterion, please explain why.
Program reviews cover a time period of 5 years.
Note the rubric by which each criterion will be evaluated.
Specific data sources are indicated for some criterion. In other cases, a variety of choices for supporting evidence exist. These data sources are listed by criterion at http://www.uvu.edu/iri/academicprograms/program_review.html
Additional sources can be used to support responses as appropriate. IRI is available to help with additional data.
In cases in which data tables are used, the written response should summarize and analyze the data.
Avoid making claims for which no evidence is provided.
Keep in mind that peers outside of your department will be reviewing this report. As such, be clear and specific, and remember that you are communicating to those outside of your field.
Associate deans and Academic Effectiveness Committee members are available for training and guidance.
Checklist
The checklist below provides specifics for each of the criterion.
General:
Insight – include Criterion 1, 8, 9 at the department level; complete fields for criterion 4-7 for each degree program.
Responses generally concise
Responses summarize and analyze data
Data referenced with links (tables not included in the text of the report)
Criterion 1: History, Development, and Expectations
Reference to program history, original intent; no numerical data included unless briefly to demonstrate program growth or development
Reference to adaptation to change, evolution of program, anticipation of change/future
Criterion 2 – External Demand
2-3 external data sources used/referenced – highlight main/important points
Summary/analysis of evidence - what does it mean? For example – trends for the degree, not just the subskills of the degree
Criterion 3 – Internal Demand
Reference to connection of program to other degrees
Summary/analysis of these connections in terms of service to other areas in the institution
Criterion 4: Degree Profile
Description of department dashboard statistics referred to/relevant points summarized
Student credit hours and FTE
Number of majors by class level
Number of awards granted
Faculty headcount and composition
Faculty teaching load (FTE, SCH, ICHE, sections by full-time, overload/adjunct)
Graduation rates
Analysis/interpretation of what this means – course and program/degree strengths and weaknesses, etc.
Reference to student learning outcomes for department, summarize main points
Reference student placement/alumni data
Analyze the above per findings and resulting actions (acknowledge significance of findings based on number of respondents, etc.)
Reference faculty output/productivity (% of faculty publishing/presenting, etc.), summarize highlights
If areas requiring data are missing or the department is unable to provide, a statement of how this is in process/plan to get should be given
Criterion 6: External Revenue and Resources Generated
Reference to highlights of grants, fundraising, etc., include totals, external relationships providing benefits
Analyze these resources, describing significance to department/school /institution, and resulting actions/plans
Criterion 7: Costs and Other Expenses
Reference to finance dashboard populated by the Budget Office
Summarize/analyze the data given (comment on efficiencies, investments provided and needed, how the program is supported in terms of the resources – students, degrees, labs, faculty, etc.); Refer to DFTE data.
Summation of essentiality of program – impact, benefits, connection to institutional mission, relation to internal factors
Summation of how the degree program should continue, be strengthened/altered
Criterion 9: External Opportunity Analysis
Analyze opportunities for improvement, strengthening, change, etc. – refer to how these could be capitalized on/benefitted from, factors that affect the program, etc.
Criterion Guidelines
The following guidelines are available in the i-buttons in Insight and are included here for your reference.
Criterion 1 – History, Development, and Expectations (Department Level)
Guiding Questions: What was the original intent of the department’s degree programs? How have the degree programs evolved over the years? How have they adapted to meet change?
Criterion 2 – External Demand
Guiding Questions: What external indicators show the need for and attractiveness of the degree program? Consider national and local statistics and trends over time. Consider employer demand for broad educational outcomes.
Criterion 3 – Internal Demand
Guiding Questions: What is the relationship of the degree program to other degrees or curriculum? How does the degree program serve other degrees or institutional needs? How would alterations in the degree program affect other degrees?
Criterion 4 – Degree Profile
Guiding Questions: What are the strengths, weaknesses, efficiencies, and needs of the degree program? How effective is the program in graduating students? What issues might need to be addressed in terms of the viability, health, or size of the programs? What trends are indicated by the data?
Criterion 5 - Quality of Degree Program Outcomes
Guiding Questions: What evidence exists for congruence between intended and actual student learning outcomes? How does placement data provide evidence for program effectiveness? How does faculty research/creative work contribute to the program?
Criterion 6: External Revenue and Other Resources Generated
Guiding Questions: What external sources of data (e.g., research grants, fundraising, equipment grants, etc.) does the program have? Indicate the source, amount, and timeframe for the funding as well as total amounts. What potential revenue sources exist? What external relationships or joint partnerships with other educational institutions, corporations, businesses, and governments exist of might be nurtured?
Criterion 7: Costs and Other Expenses
Guiding Questions: What demonstrable efficiencies are associated with the degree program? What investments are needed to bring the degree programs to a higher level of quality?
Criterion 8: Internal Impact, Justification, and Overall Essentiality (Department Level)
Guiding Questions: What impact have the degree programs had or are likely to have? What are the benefits to the institution? How do the degree programs help achieve the institutional mission? How are these degrees related to the success of other degrees? This is a summative measure of why the degree programs should continue or be strengthened.
Criterion 9: External Opportunity Analysis (Department Level)
Guiding Questions: What opportunities exist for improvement and strengthening the degree programs (cf. curriculum, student achievement, competitiveness, innovation)? How might these be capitalized upon? What external environmental factors affect the degree programs? Would a change in degree program formats be beneficial?
Rubric
Program Prioritization Scoring Rubric Adapted from Robert C. Dickeson The purpose of the scoring rubric is to ensure inter-rater reliability. AEC members should rate degree programs based on the information provided. The scores 1, 3, and 9 are intended to force differentiation among program review results.
Criterion
1 - Minimal/Limited
3 - Moderate
9 - Exceptional/Significant
History, Development, and Expectations
The program meets the original expectations of the
University
The program meets the original expectations of the University, and has demonstrated the ability to adapt to the changing needs of the
The program meets the original expectations of the University, has demonstrated the ability to adapt to the changing needs of the University and its internal and external stakeholders, and demonstrates exceptional ability to anticipate change and build for the future
External Demand
Demand for the program is limited; trends are flat or declining, which raises questions about its efficacy
Demand for the program is moderate
Demand for the program is exceptional; it enjoys a positive trend; it meets a variety of external expectations, and is seen as central to the University’s future
Internal Demand
The program provides minimal or no service to other programs
The program provides moderate service to other programs
The program provides exceptional service to other programs; such
programs could not flourish without the service provided by this program
Enrollments and awards granted are satisfactory compared to dedicated program resources (faculty, sections, class size, etc.)
Enrollments and awards granted are exceptional given the dedicated resources (faculty, sections, class size, etc.)
Quality Outcomes
Evidence of exemplary student performance and results of student learning outcomes assessment and related actions are limited; placement data is weak; faculty productivity is minimal
Evidence of exemplary student performance and results of student learning outcomes are satisfactory; placement data is acceptable; faculty productivity is adequate
Evidence of exemplary student performance and results of student learning outcomes assessment and related actions are exceptional; faculty productivity is commendable
