Methodology for Quintiling Non-Academic Programs
The first step towards non-academic program prioritization by way of the attached questionnaire will be to have each non-academic area identify its respective programs to be reviewed. Once the programs for evaluation are identified and reviewed, the following criteria and weights will be used to rank non-academic programs into quintiles:
-
25% - Cost-Effectiveness (budget vs. actual, productivity, performance)
-
20% - Importance to the institution (mission, vision, strategic plan, core themes, mandates)
-
20% - Demand (internal, external)
-
20% - Quality (input, outcomes, how well delivered)
-
15% - Opportunity (collaboration, resource sharing, savings, improvements)
Each program will be scored from 1 to 5 for each of the above criteria according to the following scale:
-
Does not meet criteria
-
Slightly meets criteria
-
Meets criteria
-
Slightly exceeds criteria
-
Exceeds criteria
The score (1-5) for each criteria will be converted to a weighted score according to the importance of the criteria. Under this methodology, the maximum score a program could receive is 5 and the lowest is 1. For instance, a program receiving a 5 (exceeds criteria) for each of the five criteria at their respective weights, would receive a total weighted score of 5 because the weights total 100% (100% x 5 = 5). Or, a program receiving a 1 (does not meet criteria) for each of the five criteria at their respective weights, would receive a total weighted score of 1 (100% x 1 = 1). To better illustrate this scoring methodology, following is an example of how an intermediate non-academic program could be scored:
Criteria
|
Weight
|
Score
|
Weighted Score
|
Cost-Effectiveness
|
0.25
|
1
|
0.25
|
Institutional Importance
|
0.20
|
4
|
0.80
|
Demand
|
0.20
|
4
|
0.80
|
Quality
|
0.20
|
2
|
0.40
|
Opportunity
|
0.15
|
3
|
0.45
|
Total Weighted Score
|
2.70
|
Each program will sum to a total weighted score that will enable categorization into quintiles with the following rankings:
-
Programs needing enrichment.
-
Programs needing a higher level of continued support.
-
Programs needing a neutral or similar level of continued support.
-
Programs needing a lower level of continued support.
-
Programs requiring further review, assessment, and action plan to determine what options will be taken.
The following scores will determine which quintiles the program will fall into:
Score
|
Quintile
|
4.21 – 5.00
|
1. Programs needing enrichment
|
3.41 – 4.20
|
2. Programs needing a higher level of continued support
|
2.61 – 3.40
|
3. Programs needing a neutral or similar level of continued support
|
1.81 – 2.60
|
4. Programs needing a lower level of continued support
|
1.00 – 1.80
|
5. Programs requiring further review, assessment, and action plan to
determine what options will be taken
|
In the previous example illustrating how an intermediate program could be scored, the example program received a weighted score of 2.70. This score places the program in quintile three, a program needing a neutral or similar level of continued support.
Idaho State University
Administrative (Non-Academic) Program Review
for University Program Prioritization1
Program Name:
Administrator: Department/Unit:
Program Definition: any activity or collection of activities that consumes resources
(i.e. dollars, people, time, space, equipment)
For your responsible area, please identify the major, significant activities that consume resources and complete this questionnaire for each of these programs. For example, in the Controller’s Office there is the general accounting department that has major, significant activities such as accounts payable, collections, financial reporting, travel, and cash management that will qualify as separate programs within the single area of general accounting. In some cases, these activities themselves may be subdivided further into additional programs. The major, significant activities are the programs that will be reviewed through program prioritization by use of this questionnaire. Identify your programs to be reviewed as those that are major activities consuming significant resources. Please keep in mind that on our first pass with this new program prioritization process, areas are encouraged to keep programs broadly defined, so as not to produce more programs that can be reasonably evaluated.
1. Key Goals and Objectives:
- Organizational structure and performance
- Institutional objectives primarily driven by this organization
1.a. Is this program mandated federally?
1.b. Is this program mandated by the state?
1.c. Is this program required because of obligations other than federal or state?
If the program is required by previously signed contracts or agreements, identify the party or parties involved in the agreement, the date and history of the agreement, and the date of expiration of the agreement.
1.d. Is this program essential to the operation of the university?
ISU Mission Statement
The mission of Idaho State University is to advance scholarly and creative endeavors through the creation of new knowledge, cutting-edge research, innovative artistic pursuits and high-quality academic instruction; to use these qualities to enhance technical, undergraduate, graduate, and professional education, health care, and other services provided to the people of Idaho, the Nation, and the World; and to develop citizens who will learn from the past, think critically about the present, and provide leadership to enrich the future in a diverse, global society.
Core Themes
Learning and Discovery, Access and Opportunity, Leadership in the Health Sciences, and Community Engagement and Impact.
1.e. How does this program support the mission, vision, strategic plan, core themes, and institutional objectives of the university?
1.f. How is the program’s performance reviewed? By whom? How often?
2. Key Services Provided to Customers (Internal/External)
2.a. Please provide a brief description of each service/function your program provides, its primary customers, and the nature of demand.
Description of Service/Function
|
Primary
Customers
|
Demand:
1. # of people served/unit time (e.g. 10/day)
2. Increasing/Stable/Decreasing
|
Internal
|
External
|
Example:
i.e. Planning
|
List the primary customers of the service
|
Insert the number of people this function services per unit of time (day, week or month) and if that number is increasing, stable, or decreasing for internal users
|
Insert the number of people this function services per unit of time (day, week or month) and if that number is increasing, stable, or decreasing for external users
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2.b. How many positions were assigned to the program over the last two years?
|
2011 – 2012 (FY12)
|
2012 – 2013 (FY13)
|
Position Type
|
Headcount
|
FTE
|
Headcount
|
FTE
|
|
FT
|
PT
|
Total
|
FT
|
PT
|
Total
|
Professional
|
|
|
|
|
|
|
|
|
Classified
|
|
|
|
|
|
|
|
|
Student
|
|
|
|
|
|
|
|
|
Exempt
|
|
|
|
|
|
|
|
|
Other
|
|
|
|
|
|
|
|
|
Total
|
|
|
|
|
|
|
|
|
2.c. How does the size and scope compare with similar/same programs at peer institutions?
2.d. Are there any current or proposed state, regional, or local mandates, or new policies or laws that may impact external demand for the program’s services?
Yes
|
No
|
Provide Details (describe the expected impacts)
|
|
|
|
|
|
2.e. Are there any current or proposed state or regional mandates or new policies that may impact internal demand for the program’s services?
Yes
|
No
|
Provide Details (describe the expected impacts)
|
|
|
|
|
|
3. Key Processes or Implementation Elements:
- Review to reduce bureaucracy and/or streamline operations
- Sunset clauses (where applicable)
- Cost-effective technological improvements
3.a. What are the key processes or implementation elements for this program? How many personnel are assigned to the key processes or implementation elements? Can any of these key processes or implementation elements be streamlined or eliminated?
1. Key Implementation Element Identified
|
Personnel Assigned to Key Element
|
|
|
Can This Element Be Streamlined or Eliminated?
|
Yes
|
No
|
Explain how this implementation element can be streamlined or eliminated, or why this implementation element cannot be streamlined or eliminated.
|
|
|
|
|
|
2. Key Implementation Element Identified
|
Personnel Assigned to Key Element
|
|
|
Can This Element Be Streamlined or Eliminated?
|
Yes
|
No
|
Explain how this implementation element can be streamlined or eliminated, or why this implementation element cannot be streamlined or eliminated.
|
|
|
|
|
|
3. Key Implementation Element Identified
|
Personnel Assigned to Key Element
|
|
|
Can This Element Be Streamlined or Eliminated?
|
Yes
|
No
|
Explain how this implementation element can be streamlined or eliminated, or why this implementation element cannot be streamlined or eliminated.
|
|
|
|
|
|
4. Key Implementation Element Identified
|
Personnel Assigned to Key Element
|
|
|
Can This Element Be Streamlined or Eliminated?
|
Yes
|
No
|
Explain how this implementation element can be streamlined or eliminated, or why this implementation element cannot be streamlined or eliminated.
|
|
|
|
|
|
5. Key Implementation Element Identified
|
Personnel Assigned to Key Element
|
|
|
Can This Element Be Streamlined or Eliminated?
|
Yes
|
No
|
Explain how this implementation element can be streamlined or eliminated, or why this implementation element cannot be streamlined or eliminated.
|
|
|
|
|
|
3.b. Does the program have unmet equipment/software needs?
If yes, complete the chart below.
Description of What is Needed
|
Role of the Needed Item in Fulfilling Program Mission
|
Approximate Cost
|
Describe equipment/software needs
|
How will the equipment/software enable your office to work effectively and efficiently
|
Estimate the cost
|
|
|
|
|
|
|
|
|
|
|
|
|
3.c. What technologies are available to the program? Are there technological improvements that could be made to save on labor, or to improve the product/service offered? How does the program get technological support?
Technologies Available
|
Improvements Could Be Made For Savings
|
Savings Expected on Labor/Products/Service Currently Offered
|
Technological Support Currently Receiving
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4. Organization Review
- Detailed organization chart for program area
• Position analysis
• Span of control
- Opportunities for:
• Collaboration and restructuring
• Sharing skill sets and resources
• Cross-training
4.a. Display, or attach, the program organization chart with the number of positions in each program area. Name the key positions identified in span of control.
4.b. Designate the key positions in each program area. Which individuals are cross-trained and in what areas?
Position/Title
|
Cross-Trained
Yes / No
|
If Yes, List Areas
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4.c. Are there other campus programs providing similar services? If yes, how are this program’s services different than those of other programs?
4.d. Are there any external or outsourced programs/services providing similar services? If yes, how do this program’s services differ from theirs?
4.e. Does this program have any external collaborations? If yes, how do these external collaborations benefit the university? Examples include, but are not limited to, cooperative agreements, articulation agreements, outreach efforts, corporate partnerships, economic relationships, etc.
5. Budgeting/Planning
- Organization structure and performance metrics comparisons to:
• Similar institutions
• National standards
• “Best of Breed”
5.a. Does the program have any operations that generate revenue?
If yes, provide a list of financial resources generated by the program.
Revenue Source
|
Amount $
|
List the source of any revenue collected, i.e. sales, consultancy, etc.
|
List the amount per program/year
|
|
|
|
|
|
|
|
|
|
|
5.b. What were the budgeted costs of the program, actual expenditures, and difference between the two for fiscal year 2012?
Program Expenditures
|
Budget
|
Expenditures
|
Difference
|
Salaries
|
|
|
|
Benefits
|
|
|
|
Services
|
|
|
|
Supplies
|
|
|
|
Travel
|
|
|
|
Equipment
|
|
|
|
Professional Development
|
|
|
|
Other
|
|
|
|
TOTAL
|
|
|
|
5.c. What metrics are used to evaluate the program’s performance? How does this program’s performance metric compare to those from peer institutions, national standards, and industry benchmarks? Click on Peer Institutions to see a list of Institutional Research identified peers.
Name/Description of Performance Metric
|
Program
Metric Value
“N/A” if not applicable or metric does not exist
|
Peer Institution Metric Value
“N/A” if not applicable or metric does not exist
|
Industry Benchmark Metric Value
“N/A” if not applicable or metric does not exist
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6. Opportunities for Savings or Additional Investments
6.a. Have you identified any opportunities for savings or additional investments? If yes, please describe.
Program Review Signature and Date
Program Director: Date:
Vice President: Date:
Share with your friends: |