By Yasuko Kanamori, Tracey A. Glaessgen & Jeffrey H. D. Cornelius-White | Similar to the public-welfare aim of many universities, Missouri State University (MSU) was granted a specific statewide public affairs mission in 1995 comprising three pillars: community engagement, cultural competence, and ethical leadership. Since the implementation of this mission, the university has engaged in various efforts to promote and foster public-affairs awareness among students, including through its first-year seminar (FYS). This article details a study conducted to evaluate the effectiveness of the FYS in enhancing students’ public-affairs awareness. The researchers solicited input from students in the first and last weeks of their first semester at MSU using the Public Affairs Scale–Short Survey (PAS-SS) as well as other questions. The study sample consisted of 540 students who completed both the pre- and post-surveys. The researchers found that students’ public-affairs awareness in the cultural competence domain increased during the FYS program, but not in community engagement or ethical leadership. Additionally, there were significant differences in public-affairs awareness over time between first-generation students enrolled in specialized sections and those who were not. No significant differences were found in public-affairs awareness between faculty- and staff-taught sections or between sections with a peer leader and those without. The article concludes with a brief discussion of the study findings and a consideration of implications for future practice.

Authors Note

Yasuko Kanamori, Department of Counseling, Educational Psychology and Research, University of Memphis; Tracey A. Glaessgen, Center for Academic Success and Transition, Missouri State University; Jeffrey H. D. Cornelius-White, Department of Counseling Leadership and Special Education, Missouri State University.

Correspondence regarding this article should be addressed to Tracey A. Glaessgen, Associate Director, Center for Academic Success and Transition, Missouri State University, 205 University Hall, 901 South National Avenue, Springfield, MO 65897. Phone: (417) 836-8343. E-mail: TraceyGlaessgen@MissouriState.edu

---

The first-year seminar (FYS) has become a popular means to help students transition to college and succeed academically. However, many higher education institutions have expressed concerns about the effectiveness of their FYS programs, prompting researchers to study their overall success. For instance, Permzadian and Crede (2016) reviewed 195 studies involving over 150,000 students and found that first-year seminars tend to have small average effects on grades (δ = .02) and retention (δ = .11); yet, these effects are meaningful in that even small changes in college success and retention can have significant financial impacts on both institutions and the communities they serve. Universities also often imbue their first-year seminars with local nuances to reflect the mission and “flavor” of their respective campus environments. More recently, higher education institutions have placed greater emphasis on civic engagement—including the importance of appreciating diversity, developing a sense of social responsibility, and leading by example—as a core value to be cultivated in students (Jacoby, 2009).

Missouri State University’s Public-Affairs Mission

Founded in 1905, Missouri State University (MSU) is a Carnegie-classified public, four-year, comprehensive, residential, selective-admissions institution; in 2010, MSU also received the Carnegie Foundation’s elective Classification for Community Engagement. The university is located in Springfield, the state’s third largest city, and enrolls approximately 24,000 students. Expanding upon its mission “to develop educated persons,” MSU received a public-affairs designation in 1995, making a commitment to prepare students to heighten their cultural competence, engage with their communities, and become ethical leaders. (While some higher education institutions may use the term civic engagement, MSU envelops civic engagement within the concept of public affairs.) Each year, the university decides upon a public-affairs theme, which is then promoted during annual signature events, including its public-affairs conference, and through the selection of a common reader for the first-year seminar.

The university’s public-affairs mission comprises three pillars: ethical leadership, cultural competence, and community engagement. The goal of ethical leadership is for students to articulate their values, act ethically in a democracy, and engage in principled leadership. The goal of cultural competence is for students to recognize and respect multiple perspectives and cultures. Finally, the goal of community engagement is for students to recognize the importance of their contribution and that of scientific principles to sound public policy and to society more broadly (MSU, 2018a). The university strives to fulfill its public-affairs mission through various means, among the most important of which is the first-year seminar.

First-Year Seminar and Public Affairs

Previous research has provided only limited evidence of the association between first-year seminars and increases in students’ civic learning, suggesting a gap in the literature. For example, in their mixed-methods study, Schamber and Mahoney’s (2008) found that when a first-year seminar included a short-term service-learning component (as occurs at MSU), students showed significant gains in their political awareness (i.e., knowledge of local and national politics) and social-justice attitudes (i.e., awareness of institutions’ role in determining individuals’ fates). However, the effects were small (d = .11, d = .33, respectively), which was further supported by qualitative evidence. The study also explored students’ motivation for civic action (i.e., plans for future community involvement), but the findings were inconsistent, resulting in an unclear picture of the extent to which student motivation was affected by the intervention. Similarly, Troxel and Cutright (2008), in their case studies of Georgia State University, Millersville University, and Texas A&M-Corpus Christi, reported on how these three institutions have attempted to integrate civic learning into their first-year seminars, but they provided little quantitative evidence related to program evaluation.

At Missouri State University, the FYS is required of all incoming, first-time students who have fewer than 24 hours of postsecondary experience or who are not enrolled in Honors College (Honors College students take a topic-specific, two-credit-hour course, which has different goals and objectives). Civic engagement is central to the FYS; as part of the general education curriculum, this two-credit-hour, semester-long course has three objectives: (a) to heighten students’ academic skills; (b) to introduce students to MSU’s public-affairs mission, with equal emphasis given to each of its three pillars; and (c) to foster students’ awareness of campus resources and establish connections with its community members. To help promote public-affairs awareness, all sections of the FYS are required to utilize the annually selected common reader. With input from faculty, student affairs, community library, and student representatives, the common reader is selected via a campus-wide and communitywide nomination process. The common reader selection committee reviews nominations and selects the book that encompasses all aspects of the university’s public-affairs mission. For the 2017-2018 academic year (during which this study took place), the common reader was Will Allen’s (2013) The Good Food Revolution, tied to MSU’s theme of “Sustainability in Practice: Consensus and Consequences.” This book served not only as a scholarly introduction to the theme for the academic community, but also allowed students to conduct a deeper critical analysis of the public-affairs mission and to make connections among the reader, the mission, interdisciplinary coursework, students’ academic major, and their communities (MSU, 2018b). The common reader curriculum committee develops discussion questions, quizzes, and activities that are then shared with the instructors to utilize in the FYS. While no specific assessment is conducted across all sections of the course, instructors of every section are required to assign the entire book and to assess student learning related to it.

The instructional requirements for teaching the first-year seminar include a master’s degree and current affiliation with the university, giving faculty and staff from various divisions an opportunity to teach the course. All instructors participate in an annual professional development workshop. Additionally, some FYS sections are assigned a peer leader who serves as a resource for the partnering instructor and for students. Peer leaders are sophomores, juniors, and seniors who are selected from a competitive application process and have a desire to help first-year students as they transition to university life. Partnering with an instructor, peer leaders attend every FYS session and take a peer-leadership class (MSU, 2018b). The participation of peer leaders in the FYS program is supported by the literature, which has highlighted the benefits of peer-leader programs in facilitating students’ personal development, academic success, satisfaction, and persistence (e.g., Shook & Keup, 2012). What remains to be explored in future research, however, is whether—and to what extent—peer leaders facilitate civic engagement and learning outcomes.

Since Missouri State University’s population of first-generation students is slightly higher than the national average (34% compared to 30%), and since the retention rate of first-generation students is 8% to 10% lower than that of continuing-generation students (MSU, 2018c), faculty and staff decided in fall 2014 to implement some specialized FYS sections as a way to better support this population. These specialized sections were offered to first-generation students grouped by college (referred to as “college-dedicated”) but also included sections for undeclared, first-generation students. The literature supports the implementation of programs designed to address unique first-generation student needs toward improving academic outcomes (e.g., Pike & Kuh, 2005; Vaughan, Parra, & Lalonde, 2014; Woosely & Shepler, 2011). In the case of peer-leader programs, however, it is not clear whether and to what extent specialized programming would also improve civic engagement among first-generation students.

Study Purpose

Program evaluations provide an opportunity to assess the effectiveness of a program (McDavid, Huse, & Hawthorn, 2013). The results of an evaluation may generate suggestions for improving a program, thereby helping to ensure that it meets its intended outcomes (McDavid et al., 2013). The primary aim of this study was to conduct a program evaluation of the first-year seminar at Missouri State University, specifically as it relates to the development of public-affairs awareness. A secondary aim of this study was to assess the efficacy of the components of the FYS program (i.e., specialized sections, peer-leader sections, faculty/staff-taught sections) in facilitating public-affairs awareness among students.

The study was guided by the following research questions (RQs):

• RQ1: Does the FYS program at MSU produce gains in public-affairs awareness, conceptualized in terms of ethical leadership, cultural competence, and community engagement?
• RQ2: Are there differences in public-affairs awareness gains between first-generation students in specialized sections and those in non-specialized sections?
• RQ3: Are there differences in public-affairs awareness gains between peer-leader sections and non-peer leader sections?
• RQ4: Are there differences in public-affairs awareness gains between staff-taught sections and faculty-taught sections?

Method

Procedures

After obtaining approval from the university’s institutional review board, participants were recruited through instructors of FYS courses at Missouri State University. Recruitment emails were sent to 83 instructors (nine of whom taught multiple sections) on August 8, 2017, prior to the start of the fall semester, asking for permission to survey students at the beginning and end of the semester. The link to the pre-survey was sent on August 14 to the 74 instructors who agreed to participate. On the first day of class (August 21), reminder emails were sent to the 74 instructors who agreed to participate. A link to the post-survey was sent on November 17 to the 73 instructors whose students participated in the study; a reminder was sent on December 4.

The survey for the study was built and administered using Qualtrics online survey software. The survey included the items from the Public Affairs Scale–Short Survey (PAS-SS) (Levesque-Bristol & Richards, 2014) as well as questions pertaining to ethical leadership, university involvement, peer-leader interactions, and advisor roles along with a demographics questionnaire. After participants gave their informed consent, they were directed to the 42-item online survey. The “request response” function in Qualtrics minimized inadvertent item nonresponse, and three attention-check items were also included to identify inattentive and random response patterns.

Participants

G*Power was used to conduct an a priori power analysis, specifying an effect size of f = 0.25, an alpha level of 0.05, and a power of 0.95 to determine the needed sample size (Faul, Erdfeld, Lang, & Buchner, 2007). The analysis showed that a sample size of 251 would be required to reveal a statistically significant effect for the study. After data screening 687 potential participants with matched pre-post survey scores (Tabachnick & Fidell, 2007), a sample of 540 individuals was included in the study. The sample consisted of 29.3% men, 70.4% women, and 0.4% “other.” The age of participants ranged from 18 to 53 years, with a mean age of 18.6 (SD = 2.40). The majority of the participants reported being White (89.1%). Fifty-eight percent of the participants indicated that their parents’ combined annual income was greater than $70,000, and 31.5% reported being employed. Approximately 40.4% identified as a first-generation college student, defined as a student whose parents did not graduate from college (see Table 1 for complete demographic characteristics). The sample overrepresented female and first-generation students.

Table 1

Demographic Characteristics

Characteristic	N = 540
	n	%
Gender
Man	158	29.3
Woman	380	70.4
Other	2	0.4
Ethnicity/Race
White	481	89.1
Black	26	4.8
American Indian/Alaska Native	4	0.7
Asian	9	1.7
Other	20	3.7
First Generation College Student Status
First-Generation College Student	218	40.4
Non-First Generation College Student	322	59.6
Employment Status
Unemployed	370	68.5
Work 1-10 hours a week	75	13.9
Work 11-20 hours a week	60	11.1
Work 21-34 hours a week	26	4.8
Work more than 35 hours a week	9	1.7
Parental Annual Income
$0	10	1.9
Less than $10,000	15	2.8
$10,000 – $19,999	23	4.3
$20,000 – $29,999	22	4.1
$30,000 – $39,999	31	5.7
$40,000 – $49,999	34	6.3
50,000 – $59,999	49	9.1
$60,000 – $69,999	42	7.8
$70,000 – $79,999	47	8.7
$80,000 – $89,999	45	8.3
$90,000 – $99,999	59	10.9
$100,000 – $149,999	109	20.2
More than $150,000	54	10.0

Measures

The Public Affairs Scale–Short Survey is a modified version of the original Public Affairs Scale (PAS) (Levesque-Bristol & Cornelius-White, 2012). The PAS-SS is a self-report measure consisting of 15 items assessing civic learning and engagement within three dimensions: community engagement (five items), cultural competence (five items), and ethical leadership (five items). The community engagement subscale assesses students’ ability to recognize and meet community needs (e.g., “I am confident that I will participate in community service activities in the future”). The cultural competence subscale measures students’ cultural self-awareness, knowledge, and skills for engaging with individuals from other cultures (e.g., “I am able to communicate effectively with people from different cultures”). The ethical leadership subscale assesses the development of students’ moral and ethical awareness as well as excellence and integrity as they lead and serve others (e.g., “When working in groups, I assure everyone’s voice is heard before a decision is reached”). Items in this measure are rated on a 5-point scale, ranging from 1 (“strongly disagree”) to 5 (“strongly agree”), with higher scores indicating greater levels of the three dimensions of civic learning and engagement. Internal consistency of PAS-SS scores was supported by Levesque-Bristol and Richards (2014) in a sample of largely college seniors (community engagement α = 0.84; cultural competence α = 0.76; ethical leadership α = 0.81) as well as in the current sample (community engagement α = 0.87; cultural competence α = 0.65; ethical leadership α = 0.68). The scale revision study reported evidence of construct validity in the forms of factor analysis and correlations in the expected direction between the PAS-SS and two measures of similar constructs (see Levesque-Bristol & Richards, 2014). The participants also completed a demographics questionnaire.

Results

SPSS (version 22.0) was used for data screening, an assumptions check, and the main analyses. Following standard procedures for data screening for inferential statistics (Tabachnick & Fidell, 2007), the researchers checked assumptions for MANOVA. The assumptions check revealed univariate outliers for items on the PAS-SS, which were excluded from the analyses. The data also violated normality; however, given that MANOVA is robust against violations of assumptions when there is an adequate sample size (N = 540), it was determined that this analysis was appropriate for this study.

Descriptive Statistics and Preliminary Analyses

An examination of the descriptive statistics for the three factors of the PAS-SS—community engagement, cultural competence, and ethnical leadership—indicated that, overall, participants rated all three factors high at well above the midpoint of the factor score of 15 (see Table 2). Additionally, ethical leadership was consistently rated highest of the three factors in all subgroups, suggesting that participants held a high subjective view of themselves as ethical leaders. Notably, while cultural competence and ethical leadership scores increased over time in all groups, community engagement scores decreased over time in some groups. Table 2 provides the pre and post PAS-SS factor score means and standard deviations for the overall sample as well as for the subgroups used in the study.

Table 2

Pre/Post PAS-SS Factor Score Means and SD by Group

	Pre-Test		Post-Test		n
	Mean	SD	Mean	SD
Overall					540
Community Engagement	21.79	2.74	21.74	2.79
Cultural Competence	20.19	2.69	20.81	2.54
Ethical Leadership	22.49	1.74	22.56	1.80
Peer Leader Sections					209
Community Engagement	21.59	2.80	21.50	2.77
Cultural Competence	20.28	2.61	20.96	2.40
Ethical Leadership	22.44	1.76	22.58	1.72
No Peer Leader Sections					229
Community Engagement	21.93	2.72	22.05	2.80
Cultural Competence	20.07	2.71	20.84	2.55
Ethical Leadership	22.50	1.77	22.57	1.84
Specialized Sections					84
Community Engagement	21.87	2.67	21.68	2.82
Cultural Competence	19.54	2.46	20.89	2.50
Ethical Leadership	22.30	1.81	22.43	1.90
Non-Specialized Sections					101
Community Engagement	21.85	2.80	21.97	2.67
Cultural Competence	20.92	2.49	21.25	2.48
Ethical Leadership	22.71	1.82	22.74	1.72
Faculty-Taught Sections					187
Community Engagement	21.74	2.65	21.59	2.78
Cultural Competence	19.99	2.77	20.84	2.41
Ethical Leadership	22.56	1.69	22.66	1.72
Staff-Taught Sections					251
Community Engagement	21.78	2.85	21.94	2.80
Cultural Competence	20.30	2.58	20.93	2.53
Ethical Leadership	22.40	1.82	22.51	1.82

Evaluation of FYS in Relation to Public-Affairs Awareness

A repeated measures MANOVA was conducted to answer RQ1 (“Does the FYS program at MSU produce gains in public-affairs awareness, conceptualized in terms of ethical leadership, cultural competence, and community engagement?”). The analysis was conducted on the entire sample (N = 540) with the three PAS-SS factors (community engagement, cultural competence, and ethical leadership) as the dependent variables in the analysis. The results showed that there was a significant difference in public-affairs awareness over time (F(3, 537) = 13.10, p < .001, partial η²= .068). Univariate tests further indicated that the FYS course had a significant effect on cultural competence (F(1, 539) = 37.09, p < .001, partial η²= .064) but not on community engagement (F(1, 539) = .16, p = .687, partial η²= .0003) or ethical leadership (F(1, 539) = .86, p = .353, partial η²= .002). Overall, the mean cultural competence scores increased from 20.19 (SD = 2.69) before to 20.81 (SD = 2.54) after taking the FYS course.

Evaluation of Specialized Sections

The program evaluation included students in FYS from both 42 general and 16 specialized sections that cater to students within a particular college and who self-identify as first-generation. For this subgroup analysis (as well as the following two), students in the nine sections taught by graduate teaching assistants were removed since the latter are not faculty or staff, are not involved with specialized first-generation sections or college-specific sections, and do not use peer leaders. A two-way mixed-design MANOVA was conducted to answer RQ2 (“Are there differences in public-affairs awareness gains between first-generation students in specialized sections and those in non-specialized sections?”). In this analysis, first-generation students in specialized sections (n = 84) were compared to first-generation students in non-specialized sections (n = 101) in relation to the three PAS-SS factors. Results showed a significant interaction effect, indicating a difference between the specialized and non-specialized sections in public-affairs awareness over time (F(3, 181) = 3.16, p = .026, partial η²= .05). Univariate tests indicated specifically that the specialized sections had a significant effect on cultural competence over time (F(1, 183) = 8.29, p = .004, partial η²= .043) but not on community engagement (F(1, 183) = .68, p = .410, partial η²= .004) or ethical leadership (F(1, 183) = .12, p = .725, partial η²= .001). The mean cultural competence scores increased from 19.54 (SD = 2.46) to 20.89 (SD = 2.50) in the specialized sections and from 20.92 (SD = 2.49) to 21.25 (SD = 2.48) in the non-specialized sections. There were also significant main effects of time and specialized sections (see Table 3 for all results of multivariate tests and Table 4 for all results of univariate tests).

Table 3

Multivariate Tests for First-Generation Specialized vs. Non-Specialized Sections (N = 185)

Effect	Λ	F	df1	df2	p	Partial η2
Specialized Section	0.95	3.12	3	181	0.027	0.049
Time	0.89	7.45	3	181	< 0.001	0.110
Section*Time	0.95	3.16	3	181	0.026	0.050

Table 4

Univariate Tests First-Generation Specialized vs. Non-Specialized Sections (N = 185)

Effect	Measure	MS	F	df1	df2	p	Partial η2
Specialized Section	Community Engagement	1.72	0.15	1	183	0.703	0.001
	Cultural Competence	69.40	7.39	1	183	0.007	0.039
	Ethical Leadership	12.19		1	183	0.107	0.014
Time	Community Engagement	0.12	0.04	1	183	0.849	0.000
	Cultural Competence	65.02	22.13	1	183	<0.001	0.108
	Ethical Leadership	0.59	0.31	1	183	0.577	0.002
Section*Time	Community Engagement	2.19	0.68	1	183	0.41	0.004
	Cultural Competence	24.35	8.29	1	183	0.004	0.043
	Ethical Leadership	0.24	0.12	1	183	0.725	0.001

Evaluations of Sections with a Peer Leader

The evaluation of the FYS included 41 sections with a peer leader and 55 sections without a peer leader. Instructors are invited to indicate whether they would prefer to teach the course with or without a peer leader, and most instructors who prefer a peer leader are matched with one. The researchers used a two-way mixed-design MANOVA to answer RQ3 (“Are there differences in public-affairs awareness gains between peer-leader sections and non-peer leader sections?”). In this analysis, participants in sections with a peer leader (n = 209) were compared to participants in sections without a peer leader (n = 229) relative to community engagement, cultural competence, and ethical leadership. The results revealed no significant differences between sections with a peer leader and sections without a peer leader relative to public-affairs awareness over time (F(3, 434) = 0.43, p =.733, partial η²= .003). Results indicated a main effect of time on cultural competence (F(1, 436) = 43.77, p < .001, partial η²= .091) but not on community engagement (F(1, 436) = .03, p = .871, partial η² < .001) or ethical leadership (F(1, 436) = 1.49, p = .223, partial η²= .003), with the mean cultural competence scores increasing from 20.17 (SD = 2.67) before to 20.89 (SD = 2.48) after taking the FYS.

Evaluations of Sections taught by Faculty versus Staff

The FYS comprised 38 sections taught by faculty and 49 sections taught by university staff. The researchers used a two-way mixed-design MANOVA to answer RQ4 (“Are there differences in public-affairs awareness gains between staff-taught sections and faculty-taught sections?”). In this analysis, participants in sections taught by staff (n = 251) were compared to participants in sections taught by faculty (n = 187) in relation to the three PAS-SS factors. The results showed no significant differences between staff- and faculty-taught sections regarding public-affairs awareness over time, (F(3, 434) = 0.98, p =.401, partial η²= .007). As with previous analyses, results indicated a main effect of time on cultural competence (F(1, 436) = 45.21, p < .001, partial η²= .094) but not on community engagement (F(1, 436) = .001, p = .982, partial η² < .001) or ethical leadership (F(1, 436) = 1.39, p = .240, partial η² = .003), with the mean cultural competence scores increasing from 20.17 (SD = 2.67) to 20.89 (SD = 2.48) before and after taking the course.

Discussion

Overall, the first-year seminar program showed a moderate positive effect for public-affairs awareness overall and in cultural competence specifically (equivalent to d = .54, d = .52, respectively), suggesting that FYS at Missouri State University is effective in fostering public-affairs awareness in students. This effect is larger than the typical effects of FYS on grades and retention, as shown in Permzadian and Crede’s (2016) meta-analysis. At the same time, the results of the present study suggest that there are no significant differences in public-affairs awareness gains over time between sections with or without a peer leader and sections taught by faculty or staff. Although the findings concerning peer leaders were somewhat unexpected given the benefits of peer-leader programs highlighted in the literature (e.g., Shook & Keup, 2012), the positive effects of peer leaders may be limited to areas of academic achievement, student satisfaction and adaptation to college, and retention, and do not extend to the development of public-affairs awareness. Also, sections without a peer leader might be no less effective because the instructor has chosen intentionally not to have a peer leader, suggesting that the program’s policy to only provide peer leaders when preferred is sensible. Moreover, the lack of evidence for differences between faculty-taught and staff-taught sections regarding public-affairs development supports the continued recruitment of a diverse set of FYS instructors from both staff and faculty to teach the course since findings indicated that public-affairs awareness is fostered as a result of taking the course and does not depend upon who teaches it.

Notably, there was a moderate effect over time on public-affairs awareness overall and on cultural competence specifically for first-generation students taught in specialized sections compared to those in general sections. In other words, the specialized sections were effective in producing gains in public-affairs awareness overall and in cultural competence in particular (but not in ethical leadership and community engagement) for first-generation students. This finding suggests that first-generation students do benefit from specialized sections that provide college-specific instructors and additional support and resources tailored to their needs. Given these findings, further expansion of the specialized sections seems warranted.

By contrast, changes in the community engagement and ethical leadership aspects of public-affairs awareness were not significant at all levels of analyses. The lack of evidence that the FYS fosters the development of community engagement is consistent with the inconclusive findings from Schamber and Mahoney’s (2008) research concerning students’ future plans for community involvement (described earlier). The PAS-SS may not be sensitive enough to assess change in community engagement or ethical leadership or that these two qualities are perceived by students differently at the beginning and end of the course.

Limitations

While the participant pool of 540 students from sections taught by 73 instructors is large, it still only represented a sample of the 2,995 students taught by a total of 83 instructors across all sections of the program. It is possible that the students taught by these instructors or those who self-selected to participate in the study were inherently different from the group as a whole. Likewise, while some sections had high participation rates, others did not, which suggests that bias was introduced at the student level.

Moreover, the PAS-SS is limited in its reliability and validity because it assesses public-affairs awareness as conceptualized by the university’s public-affairs mission, which is somewhat different from the exact conceptualization used in the current FYS course goals and objectives. In other words, while the PAS-SS was designed to measure Missouri State University’s public-affairs mission, it does not measure the precise conceptualization of the mission utilized in the course. Additionally, the reliability of PAS-SS scores in the current sample was generally lower than in previous studies, though still marginally adequate, suggesting that errors in instrumentation may account for the lack of evidence of larger effects.

Furthermore, it is possible that student participants did not fully understand the concepts asked about in the PAS-SS, especially the concept of ethical leadership, anecdotal reports of which support the findings in this study. Specifically, instructors have reported for years that, at the start of the course, students think they are strong leaders, but upon learning more about what ethical leadership encompasses during the course, they come to appreciate their shortcomings, and therefore their self-perceptions as leaders do not necessarily increase. In this way, it is possible that their awareness of ethical leadership actually improves due to a more accurate and informed understanding of the concept, but that the PAS-SS simply lacks sensitivity to capture this nuanced change over time. Another limitation relates to the lack of open-ended questions in the study, which, if included, may have helped yield unexpected results.

Conclusion and Implications for Practice and Research

Based on the current evaluation study, the FYS program at Missouri State University appears successful in producing gains in public-affairs awareness among students, particularly in relation to the cultural competence component. Likewise, the specialized sections for first-generation students in the various colleges appear to help first-generation students develop elements of civic learning at a rate greater than when they are integrated in non-specialized sections. Thus, continuing to route first-generation students into an expanding number of specialized sections seems warranted. As no differences were found in public-affairs development between sections taught by faculty and staff or in sections with or without a peer leader, there is no finding that supports altering these practices and variations according to availability and choice. As research on FYS programs’ small but important effects on academic achievement and retention is robust (e.g., Permzadian & Crede, 2016), but nascent for public-affairs outcomes, further research is encouraged, as also indicated by the results of this study.

References

• Allen, W. (2013). The good food revolution. New York: Penguin Random House.
• Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. doi:10.3758/BF03193146
• Jacoby, B., & Associates. (2009). Civic engagement in higher education: Concepts and practices. San Francisco: Jossey-Bass.
• Levesque-Bristol, C., & Cornelius-White, J. (2012). The Public Affairs Scale: Measuring the public goods mission of higher education. Journal of Public Affairs Education, 18, 695-716.
• Levesque-Bristol, C., & Richards, K. A. R. (2014). Evaluating civic learning in service-learning programs: Creation and validation of the Public Affairs Scale–Short Survey (PAS-SS). Journal of Public Affairs Education, 20, 413-428.
• McDavid, J. C., Huse, I., & Hawthorn, L. R. L. (2013). Program evaluation and performance measurement: An introduction to practice. Thousand Oaks, CA: Sage.
• Missouri State University. (2018a). Pillars of public affairs. Retrieved from https://publicaffairs.missouristate.edu/pillars.htm
• Missouri State University. (2018b). First-year programs. Retrieved from http://www.missouristate.edu/firstyearprograms
• Missouri State University. (2018c). Bear intelligence: Retention. Retrieved from http://www.missouristate.edu/data/105220.htm
• Permzadian, V., & Crede, M. (2016). Do first-year seminars improve college grades and retention? A quantitative review of their overall effectiveness and an examination of moderators of effectiveness. Review of Educational Research, 86, 277-316. doi:10.3102/0034654315584955
• Pike, G. R., & Kuh, G. D. (2005). First- and second-generation college students: A comparison of their engagement and intellectual development. Journal of Higher Education, 76, 276-300. doi:10.1353/jhe.2005.0021
• Schamber, J. F., & Mahoney, S. L. (2008). The development of political awareness and social justice citizenship through community-based learning in a first-year general education seminar. JGE: The Journal of General Education, 57, 75-99.
• Shook, J. L., & Keup, J. R. (2012). The benefits of peer leader programs: An overview from the literature. New Directions in Higher Education, 2012(157), 5-16. doi:10.1002/he.20002
• Tabachnick, B. G., & Fidell, L.S. (2007). Using multivariate statistics (5^th ed.). Boston: Allyn & Bacon.
• Troxel, W. G., & Cutright, M. (2008). Exploring the evidence: Initiatives in the first college year. The first-year experience monograph series no. 49. Columbia, SC: National Resource Center for the First-Year Experience and Students in Transition.
• Vaughan, A., Parra, J., & Lalonde, Trent. (2014). First-generation college student achievement and the first-year seminar: A quasi-experimental design. Journal of the First-Year Experience and Students in Transition, 26, 51-67.
• Woosely, S. A., & Shepler, D. K. (2011). Understanding the early integration experiences of first-generation college students. College Student Journal 45, 700-714.

Authors

 Yasuko Kanamori is currently a doctoral student in the Educational Psychology and Research Ph.D. program at the University of Memphis in the Department of Counseling, Educational Psychology, and Research. She holds a degree in Clinical Mental Health Counseling (M.S.) from Missouri State University in Springfield, MO and a theology degree (M.T.S.) from the Reformed Presbyterian Theological Seminary in Pittsburgh, PA. Yasuko has previously worked in community mental health settings with a diverse clientele. Her current research interests include social, educational, and political issues related to transgender, factors associated with college student success, and quantitative research methods. Her research has been published in several journals, including Archives of Sexual Behavior, Psychology of Sexual Orientation and Gender Diversity, and International Journal of Transgenderism. 

 

Dr. Tracey Glaessgen is the Associate Director, Center for Academic Success and Transition, which is part of a larger unit designed primarily to increase student success and retention initiatives, at Missouri State University.  Previously, she worked in First-Year Programs as an assistant director and in the Academic Advisement Center as an academic advisor for undeclared students at Missouri State University.  Additionally, she has taught writing courses, both at Missouri State University and Ozarks Technical Community College, as well as served in the university’s writing center.  She has presented at local, state, and national conferences and serves on university wide committees.  Tracey earned a Bachelor of Arts in English, Master of Arts in English with Literature emphasis, Master of Arts in Writing with Rhetoric and Composition emphasis from Missouri State University, and Doctorate of Education in Educational Leadership from University of Missouri.  Her dissertation focused upon first-generation college students with undeclared majors, and she recently had an article published on the topic in the peer-reviewed NACADA Journal.  She is currently completing a graduate certificate in Conflict and Dispute Resolution.  Her research interests include first-generation students, academic advising, first-year seminars, and organizational analysis and conflict in higher education.  

Jeffrey H. D. Cornelius-White, PsyD, LPC is Professor of Counseling at Missouri State University and doctoral faculty at University of Missouri-Columbia in educational leadership and policy analysis. He has broad research interests in education and the social sciences. Jef has published over 100 works, often concerned with person-centered counseling, learner-centered instruction, multicultural issues in related fields, or higher education domains such as student retention or shared governance. His works include Person-Centered Approaches (Sage, 2016), Interdisciplinary Handbook of the Person-Centered Approach (Springer, 2013), and Learner-Centered Instruction (Sage, 2010). He is co-author of developmental and validity studies on multiple scales, including the Public Affairs Scale and the Transgender Attitudes and Behavior Scale. He aims to mentor graduate students and colleagues to improve and publish their work and is currently co-editor of Person-Centered and Experiential Psychotherapies and a former editor, associate editor, or editorial reviewer for over a dozen other journals, including the Journal of Border Educational Research. Jef enjoys spending time with his family, cycling and volleyball.