Assessment of Institutional Effectiveness and Co-Curricular Programs

At the institutional level, a similar assessment process is used to monitor the quality of overall institutional effectiveness. For example, student surveys are conducted by the Office of Planning and Assessment through the CIRP Survey for new and transfer students, a climate survey for new and continuing students, and graduating senior surveys for students completing their degree programs. The University has also exerted great efforts to improve student learning outcomes and the learning environment through the establishment of learning communities, professional tutoring, and peer tutoring sessions for students.

The University evaluates the effectiveness of its educational programs through monitoring course completion, retention and graduation rates. Details of the processes used to determine retention, course completion and graduation rates are provided later under Core Component 4.D.

 4. B.2. The institution assesses achievement of the learning outcomes that it claims for its curricular and co-curricular programs.

Faculty and the CSU staff assess and evaluate the effectiveness of programs and services in order to improve the programs and services or determine the return on investments made by the University. Assessment of learning outcomes in academic programs are conducted at the general education and discipline-specific program levels. Assessment of co-curricular activities and services is conducted at the institutional level by the Office of Assessment and Institutional Research. Institutional assessments are conducted on an ongoing basis throughout the year through the use of surveys at specific times during students’ programs of study.    

Assessment in Curricular Programs

Assessment for student learning is conducted in the general education and specific discipline area programs. Both levels of assessment are included in the Student Success Plan in 4.B.1.

This work, led by the Senate Committee on Academic Policy and the Senate Committee on Teaching and Learning has continued through the initiation of the Self-Study in Spring 2011, through the writing of the report in Spring 2012. The Senate Committee on Assessment of Teaching and Learning continues to work on the alignment of the institutional learning outcomes with the general education and program learning outcomes. The April 2012 alignment of general education Student Learning Outcomes with institutional outcomes is provided in the evidence file.  

 Assessment of Institutional Outcomes

The general education curriculum is liberal arts-based and consists of courses in English composition, mathematics, humanities and fine arts, social and behavioral sciences, natural and physical sciences, computer science, and health and physical education. Placement tests are given to new first-time and transfer students with fewer than 30 semester hours, and are used to determine the appropriate freshmen-level courses for each student. The general education curriculum has historically been assessed by the English Proficiency Exam (EPE). In the Fall of 2012, the University began using the Proficiency Profile Exam (PPE) developed by the ETS to assess students' performance levels in communication, critical thinking, and competency in the field of study.

The English Proficiency Exam (EPE) has historically been used at CSU as a graduation requirement. The EPE is administered by the Office of Assessment and Institutional Research and is used to assess competency in writing, critical thinking, and other general knowledge once courses have been completed in general education and respective disciplines. Students complete the EPE after successful completion of ENG 1102, Writing and Research. Students have 85 minutes to write a four- or five-paragraph essay based on argumentative topics of current interest supplied and vetted by full-time faculty in the English Department and implemented by staff in the Office of Assessment.

EPE essay topics are changed at every testing session by staff in the assessment office. Full-time English faculty members grade students’ essays using a six-level rubric. Passing scores are between eight and twelve with twelve as the maximum score students can earn on the EPE. Students who have not passed the EPE must retake the exam or enroll in ENG 3000, an advanced composition course which requires demonstration of reading, writing and critical thinking skills. Students must pass the course with a grade of C or higher. EPE Testing Dates for Fall 2012 and Spring 2013 and scoring rubrics are provided in the evidence file.

An analysis of the EPE results during the Self-Study revealed that single administration of the EPE as a graduation requirement did not provide a complete picture of the progress students have made during their various academic milestones. The EPE provides a snapshot of student's competency in communication and did not demonstrate students’ progress in the other competencies. It was recommended was that faculty is encouraged to use the AAC&U VALUE rubrics to reinforce institutional competencies across the curriculum.

Indirect Assessments of Learning Outcomes through Surveys

In addition to assessments completed by faculty members in the classrooms in internships, capstone courses and through research-based courses, indirect assessments of student learning outcomes are also conducted through the Office of Assessment and Institutional Research using surveys.

Data from the last five years (2007-2011), from surveys administered to graduating seniors during an event called “The Senior Salute,” show that a majority (80% - 93%) of the graduates responded favorably to the statement that CSU adequately prepared them to gain effective communication skills. However, the remaining (7% - 20%) graduating students were not sure or responded unfavorably, indicating that opportunities still exist for improvement of CSU’s programs in which communication skills, particularly in verbal and written communication competencies, can be improved. Results of graduating seniors’ responses are provided in Figure 4.B.2 below.

Similarly, when graduating seniors were asked the question, “Did your experience at CSU equip you with skills to think critically,” a majority (80% - 90%) responded favorably to the question, indicating that CSU prepared them to adequately gain critical thinking skills. However, the remaining (10% - 20%) students were not sure or responded unfavorably, indicating that opportunities exist for improvement of CSU’s programs in which critical thinking skills are the intended student learning outcomes. Results of graduating seniors’ responses are provided in Figure 4.B.2 below.

When students were asked the question, "Did your experience at CSU help you increase your awareness of the roles cultural and social factors play in human achievement,” results showed that a majority (80% - 89%) of the graduates responded favorably, but the remaining (11% - 20%) students were not sure or responded unfavorably, indicating that there is plenty of opportunity for improvement of CSU’s programs in which cultural competency and social skills are intended student learning outcomes. Results of graduating seniors’ responses are provided in Figure 4.B.2 below.

When students were asked the question, "Did your experience at CSU provide you with an understanding of the scientific process,” results showed an even lower number (64%-75%) of the graduates responded favorably. However, the remaining 25% - 37% indicate that a lot of improvement is needed to reinforce this competency, particularly considering the job market in the 21^st century, which requires graduates who can readily apply problem-solving skills similar to those learned through the application of the scientific process.

When graduating students were asked the question, “Did CSU prepares you well in your selected field of study,” 82%-90% of the graduating seniors responded favorably; and the rest responded unfavorably indicating a need for improvement. Results of the surveys are summarized in Figure 4.B.2., which provides a summary of results of indirect assessments of institutional learning outcomes.

To improve the reliability of these results, it is recommended that these indirect assessments be complemented with direct assessments that independently determine if the student learning outcomes are being achieved.

Figure: 4.B.2: A Summary of Indirect Assessment of Institutional Learning Outcomes

Assessment of the Effectiveness of Student Support Services 

The following table describes the offices providing student support services, assessment tools used by the Assessment and Institutional Research Office to assess the services, and the timelines for the assessments. Detail descriptions were provided in Criteria 3.D.1. 

Students’ responses on library services: About 71% of the CSU graduates in 2007 indicated that CSU provided good library services. The percentage declined to 51% in 2008 but has been rising steadily since library improvements were made in the 2010-2011 academic year. About 63% of the 2012 CSU graduates indicated that library services had a positive impact on their educational experiences and success in today’s knowledge-based economy. The decline from 2007 in positive responses indicates that the University needs to continue to improve library service and resources in order to stay competitive with other Ohio public colleges and universities.

Students’ responses on computer facilities: About 83% of the 2007 CSU graduates agreed that computer facilities had a great impact on their academic performance. This number dropped over the last five years to 70%. The decline in positive responses also indicates that the University needs to improve availability and student access to computer facilities in order to stay competitive with other Ohio public colleges and universities.

Students’ responses on advising: About 82% of the 2012 graduates stated that faculty advising had a positive impact on their academic performance, up from the last three years when only about 75% stated that academic advising had a positive impact on their academic success. With the implementation of more intentional academic advising begun in Fall 2012 and assignment of every CSU student to an academic advisor, the percentage of students positively affected by academic advising should increase in coming years.

Figure 4.B.1: Assessment of Student Support Services

Residence Life: Only about a half (53%) of the graduating seniors stated that residence life had a positive impact on their academic performance and success. Even though there had been an increase from previous years, results from the 2012 graduates indicated a 5% decline from the previous year (58%).

Other sources of data used for improvement of programs and students’ experiences are the Climate Survey and Graduating Senior Surveys. For example, according to the Fall 2010 Climate Survey, 84% of students enrolled at CSU attest to experiencing intellectual growth in their classes.  Fall 2010 Climate Survey results show that 89 % of the CSU programs support student achievement. Also, 89.4 % of the students surveyed stated that they made a good choice to attend CSU. An even higher number of students (93%) indicated that their needs were readily met through the learning assistance, tutoring services and writing labs on the CSU campus. 

Another source of data is the annual survey of graduating seniors.  A Spring 2011 graduating senior survey evaluation revealed that 82.4% of CSU graduates were mildly/strongly satisfied with the education they received at CSU while 44.9% were strongly satisfied with the education they had received at CSU.  This instrument and its administration and subsequent evaluation suggest that CSU conforms to best practices in eliciting information from its constituencies and analyzing this as a basis for continuous improvement.

About 76.4% of the students surveyed indicated they had access to technology (Internet, My CSU, Printing Services) while 29% stated that they did not, The high numbers of students who indicated no access suggests opportunities for improvement.  Discussions of these survey results have led to suggestions of ways to improve student access to computer technology, including consideration of laptops/notebooks/netbooks complete with appropriate software provided as part of students’ tuition. However, financial aid realities and challenges and associated risk factors have limited further discussions on these issues.

At the end of every semester, faculty members complete the assessment process by reflecting and documenting improvements made in each course as a result of improvements in teaching and learning activities. CSU demonstrates that it had made progress in improvement of student learning based on the assessment findings. For example, there was increase in the number of graduating seniors who stated that CSU improved their communication skills, critical thinking skills, and knowledge and skills in their selected fields of study.

Due to the high number of underprepared students who were admitted to CSU in recent years, retention and course completion are crucial metrics for monitoring student success. Retention and course completion data compiled by the Office of Assessment and Institutional Research are used for improvement of student support services. As stated earlier, the Ohio Board of Regents recently changed the funding formula for state universities from a subsidy based on full-time enrollment (FTE) to a more nuanced model that includes course completion rates. These funding formula presented challenges for CSU and other state institutions as funding became more closely tied to course completion and graduation rates. Table 4.C.2 below provides CSU retention rates.

Table 4: C. 2. CSU Student Retention and Persistence Rate from 2006-2007 to 2010-2011 academic year.

Student Retention Rates - Entering Full-time Freshmen Fall Term
Year (Fall)	N	Still Enrolled	Non-Persisters	Retention Rate
2006	536	290	246	54.1%
2007	562	284	278	50.5%
2008	646	371	275	57.4%
2009	703	324	379	46.1%
2010	504	277	227	55.0%

CSU’s student retention based on the freshmen cohorts from 2008 shows that retention rates for new first time students was at its highest (57.4% ) in the Fall 2008 and then dropped in subsequent years (2009 and 2010) even though the population of students served by CSU remained the same. Figure 4.2 provides a graphical representation of CSU’s retention rates in the last five years.

Figure 4.C.1: Graph of CSU Retention Rate from 2006-2010

Graduation Rates for CSU students in the Last Five Years

4. C.2. The institution collects and analyzes information on student retention, persistence, and completion of its programs.

 Staff in the Office of Assessment collect data on student retention, course and degree completion rates, persistence from year to year, graduation rates, number of students in each major, ethnicity and gender of student body and faculty, and numerous other kinds of student data. These data is increasingly more fundamental to the understanding of the health of the University because of a new funding formula for all Ohio public universities, mandated by the Board of Regents, which for the first time includes course and degree completion rates as part of the funding formula for each institution. Many examples of the kind of data generated by the Office of Assessment can be seen in the CSU Fact Book

The table below, with data from the CSU Fact book and the University’s Strategic Academic and Enrollment Management (SAEM) Initiative (which had two goals (1) To improve academic quality and (2) To increase student enrollment) presents the inverse relationship between the projected graduation rate and the actual graduation rates as the University realized that retention rates began to decline as headcounts increased. This trend has led to the establishment of CSU’s University College for new first time (NFT) freshmen students.

With proposed changes in admission criteria as CSU begins to implement a targeted enrollment plan, retention rates and graduation rates are expected to increase regardless of the state’s funding formula.

Table: 4.C.1 CSU’s Projected and Actual Graduation Rates (2001-2013)

Similarly, staff in the Assessment and Institutional Research Office compute course completion rates using data extracted from the data warehouse, which contains integrated sets of data resulting from integration of data in CSU’s academic management system, called Banner, following data entry by various departments on the campus, including data from the Registrar, Student Affairs and Enrollment Management (Admissions) and Financial Aid Offices. Data collected by the Assessment Office through the data warehouse is then aggregated and analyzed for persistence, retention, completion and graduation rates by freshmen cohort.

 The following chart shows CSU graduation rates for the last five years. Data revealed that freshman cohorts graduating within four years was at 19.4% during the 2001 academic year and declined to 9.3% in the 2005 academic year.  Freshmen graduating within five years remained constant at 20.0% between 2001 and 2005.  Data indicated a slight increase in the percentage rate of students graduating within six years.  During the 2001 academic year 21.0% of students graduated within six years, 24.2% in 2005.

Table: 4.C.3.  Six-Year Graduation Rates for CSU students in the Last Five Years

As stated earlier in this Self-Study report, CSU is also in the process of making other transformative institutional changes as it implements the strategic plan, Honoring Ohio’s Historically Black Public University: A Plan for Advancing Progress at Central State University, developed in 2011 in collaboration with the Ohio Board of Regents. That plan calls for the following institutional changes to improve student retention, persistence, and course and program completion:

• 
  Conduct a more selective recruitment process by identifying and recruiting “rising achievers,” and leveraging STEM programs to attract such students;
  
• 
  Promote greater accountability of students during their first year on campus by enhancing and intensifying the University College retention effort, hiring or appointing a Director of University Retention to coordinate first-year strategies, and establishing “Preferred Pathway” pilots with community colleges;
  
• 
  Assess the viability of developing a “Success Roadmap” to degree completion that might include awarding certificates for completion of 30 credit hours and associate degrees for completion of 60 credit hours;
  
• 
  Improve the University’s physical environment.
  

• 
  Redesign and repurpose the University College;
  
• 
  Assign a Retention Specialist to each academic department to assist all students;
  
• 
  Implement and utilize an Early Alert System to inform campus-wide interventions.
  

• 
  Develop Community College Articulation Agreements;
  
• 
  Coordinate regular academic advising in each academic area;
  
• 
  Assign a Retention Specialist to each academic department to assist upperclassmen;
  
• 
  Align Community College coursework;
  
• 
  Expand summer school programming and course offerings.
  

4.C.4. The institution’s processes and methodologies for collecting and analyzing information on student retention, persistence, and completion of programs reflect good practice. (Institutions are not required to use IPEDS definitions in their determination of persistence or completion rates. Institutions are encouraged to choose measures that are suitable to their student populations, but institutions are accountable for the validity of their measures.)

The Office of Planning and Assessment in the Division of Academic Affairs has oversight over retention, persistence, course completion and other institutional data which are collected and analyzed by staff in the Assessment and Institutional Research. Staff in the Assessment and Institutional Research use Integrated Postsecondary Education Data System (IPEDS) definitions developed by the National Center for Education Statistics (NCES) to determine student enrollment, retention, and graduation.

For instance, undergraduate student enrollment is reported based on class levels (freshmen to seniors), and by degree seeking status (showing totals broken down by gender, race/ethnicity per IPEDs report). Enrollment is also reported by the number of undergraduate and graduate students (also showing totals broken down by gender, race/ethnicity per IPEDs report), and by age range of undergraduate students (showing those who are 24 years old and under or 25 and older) to provide an indication of the proportion of traditional and non-traditional students served by the University. Last but not least, the number of graduate and undergraduate students is reported by residency status of credit-seeking and non-credit seeking to provide information on the proportion of students who are in-state and out-of-state residents. The number of non-U.S. residents is also included in the enrollment report.

Based on IPEDS definitions, retention is calculated based on the total number of returning new first time (NFT) freshmen to the institution the following fall semester, and unduplicated 12-month head counts of full time NFT students (this number excludes transfer students, stop- outs etc., who return the following fall semester). Similarly, graduation (usually six-year graduation) rate is computed based on the total entering class each fall (by freshmen cohort). CSU’s retention rate has fluctuated in the last three years between 50.5 - 57.4%. CSU’s six-year graduation rates increased in the last three years from 19.3 -24.2% among new first time students in the 2003 and 2005 freshmen cohorts, respectively.