Submitted to the college of education and health sciences benedictine university lisle, illinois

Student persistence is directly related to student engagement, and student engagement is related to increased academic performance. The three benchmarks are connected in models prevalent in the literature (Astin, 1977, 1985; Pascarella & Terenzini, 1979; Tinto, 1975; 1987; 1993). The models have shown different approaches to retaining students at institutions, none of them particularly disagreeing with the others, but rather allowing for an understanding of student success from different perspectives. The literature has shown that the League for Innovation in the Community College and Achieving the Dream utilize the recommendation of increased use of technology to enhance student success (AtD, 2012).

Valencia Community College in Orlando, Florida won the Aspen Institute award as the top-performing community college in the nation in 2011. Through the Achieving the Dream initiative, the college proposed expanding the learning communities program called Learning in Community (LinC), in efforts to close performance gaps in student achievement (Brighton & Phelps, 2012). Valencia had a unified digital presence at the college and learning communities were enhanced. Similar to the college in this study, Valencia also used data extracted from to identify at-risk students so that professors could engage in academic interventions to assist them. Further, as part of the tenure process at Valencia, full-time faculty had to develop three-year action research projects on "teaching techniques involving training courses, advisors, and peer-review panels" (Gonzalez, 2012, para. 29).

The use of technology for gathering data, also called academic analytics (Campbell & Oblinger, 2007), is crucial for the use of data to track student retention, graduation rates, and even for the use of predictive modeling to assist faculty and academic advisors in determining which students need help and allow academic intervention on their behalf. Student retention and graduation rates may be improved through the use of data gathered through IT systems on campus (Campbell & Oblinger, 2007). Retention of students can save the institution money in recruiting costs, which amount to approximately $74.00 per student on average, at a two-year institution. If the institution has 25,000 students, this would result in an estimated $1.85 million in savings. Studies from the University of Alabama also determined other costs associated with the loss of students, ranging from tuition, books and food to be over $1 million at their institution (Campbell & Oblinger, 2007). The economic impact of the loss of students is heavy on all institutions, in particular community colleges with lower retention rates than four-year institutions with a comparable student population.

Course Management and Student Response systems relating to student retention and academic performance have the potential to improve learning and student success (Campbell & Oblinger, 2007). The institutional software can select and organize data according to:

• 
  Student Demographics;
  
• 
  Academic Ability;
  
• 
  Academic Performance;
  
• 
  Academic History;
  
• 
  Financial Aid;
  
• 
  Student Participation Activities;
  
• 
  Academic Effort (Use of library and labs); and
  
• 
  Institutional Information (Campbell & Oblinger, 2007).
  

The models and theories identified in the literature (Braxton, 2000; Chickering, 1969, 1993; Tinto, 1975, 1987, 1993) will be the framework for this study. The AtD participant data retrieved from the college will be correlated with the CCSSE participant data to discover possible relationships and whether AtD has an impact on student engagement, student persistence, and academic performance.

This study will document the data prior to AtD implementation in 2005, document its data as an AtD participating college from 2005 to 2008, and then document its data as an AtD Leader college from 2009-2012. It will then analyze the data on each longitudinal period and correlate results to the Community College Survey on Student Engagement (CCSSE) data, for relationships between Achieving the Dream and student success.

Chapter 3: Methodology

Introduction

The purpose of this study is to determine the relationship between Achieving the Dream (AtD) implementation and student success among first-year, full-time degree-seeking students at College X. Student success will be evaluated in terms of: student engagement, student persistence, and academic performance. Student success will be examined before the implementation of AtD (from 2002 through 2004), and after its implementation by the institution (from 2005 to 2011).

This study will use extant data on student success. Student success will be evaluated in terms of three variables: student engagement, student persistence, and academic performance. Student engagement will be measured through the Community College Survey on Student Engagement (CCSSE) administered between 2002 and 2011. Student persistence will be measured using first-year student persistence rates at the institution of interest from 2002 to 2011, restricted to the years the CCSSE instrument was used. (AtD status between 2005 and 2011). Academic performance will be measured using G.P.A. from 2002 to 2011 restricted to the years the CCSSE instrument was used. (AtD status between 2005 and 2011). Additionally, the data also include demographics such as gender, and race/ethnicity. Data from 2002 to 2004 reflect pre-AtD implementation, while data from 2005 to 2011 reflect post-AtD implementation.

The study employs a non-experimental exploratory research design, even as it seeks to describe the nature of the relationship between AtD and student success variables. Because relationships are studied, the study can be described as having a correlational design (Creswell, 2012). The data will be drawn from instruments utilized by Achieving the Dream (AtD) and the Community College Survey on Student Engagement (CCSSE) at the college where the research is conducted. The instruments used by AtD include surveys of established student participants, student focus groups, and collection of data on persistence and grade point average. For this study, the data on persistence and grade point average will be extracted. The CCSSE uses a student survey instrument for student engagement. The study will use the college's AtD longitudinal tracking analyses for persistence and grade point average to determine a relationship between student engagement and student success from data compiled by the CCSSE.

The student participants were selected by the college's AtD executive team for the AtD initiative, targeting students from low-income socio-economic families and at-risk minority students for longitudinal tracking for all the years of the grant. AtD tracking of student participants began in 2005 and has continued through 2011. Student participants consisted of entering freshmen; data were collected from these student participants on academic performance (grade-point average) and student persistence to the next year. The data are extracted from ATD information compiled at the institution for the years 2005, 2006, 2007, 2008, 2009, 2010, and 2011. Data on these student participants were collected by the college's Institutional Research office. The CCSSE did not provide separate student engagement benchmarks for these participants, however, they are inclusive to the population of first-year, full-time, degree-seeking students which were benchmarked.

The CCSSE survey contains approximately 38 questions. CCSSE data are utilized to determine changes in student engagement during all the longitudinal periods of the study. The CCSSE data are available from the academic years of 2002-2003, 2003-2004, 2005-2006, 2007-2008, and 2010-2011, or for five years of the study. CCSSE had five benchmarks of effective educational best practices as indicators of student engagement.

Five facets form the basis of student engagement as defined by the NSSE:

• 
  active and collaborative learning;
  
• 
  participation in challenging academic activities;
  
• 
  formative communication with academic staff;
  
• 
  involvement in enriching educational experiences;
  
• 
  feeling legitimated and supported by university learning communities (NSSE, 2009).
  

The Community College Student Report (CCSR), which is the data obtained from the CCSSE indicate that the instrument has a high internal consistency, measured by Cronbach’s Alpha for all constructs examined: Active Collaboration= 0.66; Student Effort =0.56; Academic Challenge = 0.80; Student-faculty Interaction =0.67; and Support for Learners =0.76. In addition, the CCSSE also is reported to have a high level of test-retest reliability, with test-retest correlations exceeding 0.70 for all constructs (Marti, 2008). In addition, reliability of the instrument for the current sample will be determined using Cronbach’s alpha, once the data are analyzed.

CCSSE is in collaboration with the National Survey of Student Engagement (NSSE); the two instruments have a significant overlap in questions on student engagement. The NSSE which began in 1998 focuses on four-year colleges and institutions while the CCSSE, created in 2001, focuses on two-year technical institutions and community colleges. The two organizations and survey instruments differ in terms of mission, student populations and resources. There is significant overlap in the survey instruments, particularly in dealing with institutional improvement and student perceptions of institutional and academic experiences (CCSSE, 2013). Chapin indicates in his 2008 study that due to significant overlap between the two instruments, CCSSE data is a validated indicator (by reason of concurrent validity) of areas of student engagement that can be used or referenced to improve student performance, learning, and persistence (Chapin, 2008; Marti, 2005).

Sample and Population

The AtD student participants represent a convenience sample and were drawn from students with low income socio-economic status and students of racial or ethnic backgrounds. CCSSE participants also reflect a convenience sample and were those students who filled out the CCSSE surveys distributed to the entire student population. The respondents to the CCSSE surveys were students drawn from the entire population of first-year, full-time degree-seeking students retrieved from the Institutional Research office. The AtD and CCSSE student participant data include those students who began full-time in the Fall and dropped to part-time by Spring. These students will not be included in the study as full-time students who persisted to the second semester as full-time.

1. 
   What are the trends in student engagement before and after the implementation of AtD at the College X?
   
2. 
   What are the trends in student persistence before and after the implementation of AtD at the College X?
   
3. 
   What are the trends in student academic achievement, measured by GPA, before and after the implementation of AtD at College X?
   

The study tests the following hypotheses

1. Null Hypothesis (H0): There are no statistically significant differences in student engagement levels: 1) before implementation of Achieving the Dream, 2) after implementation of Achieving the Dream among first-year, full-time, degree-seeking students.

Alternate Hypothesis (H_a): There are statistically significant differences in student engagement levels: 1) before implementation of Achieving the Dream, 2) after implementation of Achieving the Dream among first-year, full-time, degree-seeking students.

2. H0: There are no statistically significant differences in student persistence rates: 1) before implementation of Achieving the Dream, 2) after implementation of Achieving the Dream among first-year, full-time, degree-seeking students.

H_a: There are statistically significant differences in student persistence rates: 1) before implementation of Achieving the Dream, 2) after implementation of Achieving the Dream among first-year, full-time, degree-seeking students.

3. H0: There are no statistically significant differences in student academic performance, as measured by GPA: 1) before implementation of Achieving the Dream, 2) after implementation of Achieving the Dream among first-year, full-time, degree-seeking students.

H_a: There are statistically significant differences in student academic performance, as measured by GPA: 1) before implementation of Achieving the Dream, 2) after implementation of Achieving the Dream among first-year, full-time, degree-seeking students.

4. H0: There is no statistically significant relationship between student engagement and student persistence among first-year, full-time, degree-seeking students following the implementation of the Achieving the Dream initiative.

H_a: There is a statistically significant relationship between student engagement and student persistence among first-year, full-time, degree-seeking students following the implementation of the Achieving the Dream initiative.

5. H0: There is no statistically significant relationship between student engagement and academic performance among first-year, full-time, degree-seeking students following the implementation of the Achieving the Dream initiative.

H_a: There is a statistically significant relationship between student engagement and academic performance among first-year, full-time, degree-seeking students following the implementation of the Achieving the Dream initiative.

6. H0: There is no statistically significant relationship between student persistence and academic performance among first-year, full-time, degree-seeking students following the implementation of the Achieving the Dream initiative.

H_a: There is a statistically significant relationship between student persistence and academic performance among first-year, full-time, degree-seeking students following the implementation of the Achieving the Dream initiative.

Correlational analyses will be conducted to explore the relationships between student engagement, student persistence and academic performance (GPA). This analysis will be done for the longitudinal periods of 2001-2004 (pre-AtD) and 2005-2012 (post AtD). Differences in student engagement, retention and academic performance based on AtD will be explored using independent samples t-tests and Analyses of Variance (ANOVA). Statistical significance will be set at α = .05.

This study will examine institutional data on student GPA and student retention, along with CCSSE data pertaining to student engagement, to determine the relationship between Achieving the Dream and student success. The data were collected between 2001 to 2012 first-year, full-time degree-seeking students at a state community college. Trends in the data and significant relationships between the variables will be explored using descriptive and inferential statistics.

The purpose of this study was to explore the relationship of Achieving the Dream implementation and student success among first-year, full-time degree-seeking students at College X. The data released from College X corresponded with the pre-AtD years of 2002 through 2005 and the post-AtD years of 2005 through 2012 and explored the variables of grade point average (GPA), student persistence and student engagement, as measured by the CCSSE. The CCSSE data were not available in certain years because CCSSE surveys were not administered every academic year. Therefore, the data for GPA and student persistence were included in this study only for the years the CCSSE surveys were administered to students. These years were: 2003, 2004, 2006, 2008, and 2011. The years 2003 and 2004 represent pre-AtD implementation, while the years 2006, 2008 and 2011 represent AtD implementation.

Student persistence data are on first-year, full-time, degree-seeking students from the Fall to Spring semesters only. The CCSSE data released were the learning and engagement benchmarks for the years 2003, 2004, 2006, 2008, and 2011. The data for GPA and student persistence are cumulative for the Fall and Spring semesters of the first year of college enrollment. The CCSSE results are taken from the spring of an academic year and represent the results of student surveys from the Fall to Spring semesters.

The AtD students were separated from the data of the aggregate or general student population as separate data files. The exceptions to this are the CCSSE student engagement benchmark data. College X administered the student engagement surveys only on a sample of the general student population. All data are broken down by race and gender except for the aggregate AtD data.

A total of 2139 first-year, full-time degree-seeking students responded to the CCSSE survey at College X. Of these, 1121 were African-American, 97 were Asian/Pacific Islander, 757 were Caucasian, 72 were Hispanic, and 92 identified themselves as belonging to the category of “Other” (non-specified). Among the CCSSE respondents, 582 were male and 1000 were female students, with 557 students missing data in response to this question. For the variables GPA and Student Persistence, a total of 8296 students were reported by the institution as first-year, full-time degree-seeking students. Of these, 2869 were African-American, 154 were Asian/Pacific Islander, 4493 were Caucasian, 381 were Hispanic, and 399 identified themselves as belonging to the category of “Other”. A total of 3695 males and 4601 females were reported in the data from the institution.

What are the trends in student engagement before and after the implementation of AtD at the College X?

Table 1

Aggregate Student Engagement, Persistence, and GPA Data for Total Years of Study

Year	GPA	Persistence	CCSSE BMK 1 Collaboration	CCSSE BMK 2 Effort	CCSSE BMK 3 Challenge	CCSSE BMK 4 Interaction	CCSSE BMK 5 Support
2002-2003*	2.48	75.1%	0.37	0.43	0.60	0.36	0.49
2003-2004*	2.39	72.0%	0.34	0.45	0.56	0.35	0.41
2005-2006	2.41	69.3%	0.34	0.51	0.58	0.36	0.46
2007-2008	2.26	67.7%	0.35	0.48	0.58	0.38	0.46
2010-2011	2.37	71.6%	0.36	0.50	0.61	0.42	0.47

* Pre-AtD Years. Source: College X Office of Institutional Research for GPA and Student Persistence and the Community College Survey of Student Engagement, The University of Texas at Austin (2014).

Aggregate grade point average (GPA) and student persistence were decreasing in pre-AtD years and the trend continued through 2008. This trend reversed in 2010-2011. For student engagement benchmarks, Active Collaboration decreased from the pre-AtD years but began to increase in the years of 2008 and 2011. Student Effort increased in the pre-AtD years (0.51) in 2006, decreasing in 2008 (0.48) and increasing again in the year 2011 (0.50). Academic Challenge saw a decrease in pre-AtD years (0.60 to 0.56) and increased (0.61) by 2011. Student-Faculty Interaction decreased in Pre-AtD years (0.36 to 0.35), and began increasing in AtD years (0.36 in 2006, 0.38 in 2008, and .0.42 in 2011). Support for Learners saw a decrease in Pre-AtD years (0.49 in 2003 to 0.41 in 2004) but an increase again (0.46 in 2006, 0.46 in 2008, and 0.47 in 2011). The following Figures (1-5) display the trends and fluctuations in the data: