Blog: Idea Exchange
Assessing a First-Year Experience: From Buddha to the Beatles
Two years ago, our faculty took on the task of revising our core curriculum. One of the first steps in doing this was to create a first year experience for our incoming freshman students (FYE). After a year in planning, the learning outcomes for the FYE were established: demonstrate effective academic communication through writing, engage in critical thinking, recognize the mission statement of the university, and demonstrate intellectual curiosity. Proposals were solicited from the faculty for a 4 credit pilot project to begin fall 2014. Faculty were encouraged to think creatively and propose subjects in their specific fields that would lend themselves well to this process. Three credits were to be devoted to an academic seminar of the professor’s choice and one credit would be developed as a transition-to-college unit led by a student affairs staff. Each course would also be assigned a student course assistant and the sections would be limited to 15 students in each section. The pilot consisted of ten sections. Some examples of these courses were:
• The Long and Winding Road: The Legacy of the Beatles
• Codes and Codebreaking: Secure Communications in War and Peace
• Public Health within an Apocalypse
• This Course is So Gay: Debating the Current Pride and Pitfalls of LCBT Culture
• The Question for Immortality across Time and Space
• Barbie, Mad Men, and Shopping Malls: Reading Popular Culture
• Biomimicry: Innovation Inspired by Nature
A rigorous assessment plan was mapped out so that we could gauge the effectiveness of the pilot process. This assessment included using a process writing rubric, pre- and post-test surveys, a mid-term student satisfaction survey, interviews with faculty and staff, and focus groups with the students. This article will discuss the results of the process writing rubric.
For the writing component of the FYE, all sections had to assign at least five pages of writing and assess this writing using a rubric developed for process writing. The rubric had categories for drafting, revising, context/purpose, content development, and syntax. The rubric was available to the faculty in Taskstream through their Blackboard course. In the pilot FYE, analysis of the rubric included 145 authors with an average score of 2.84/4 (median of 3.00 and standard deviation of 0.71). We were also able to evaluate the scores on the rubric based on race, sex, high school GPA, and first generation status.
This analysis was used solely for assessment purposes. The students were graded with a rubric that was developed by each professor as an assignment grade. However, for overall measurement of effective academic communication, the professors were asked to evaluate the student artifacts with the writing process rubric. This helped the faculty understand what areas were most effective and what areas needed more focus in the course.
Full implementation of the FYE began in fall 2015 when all incoming first year, first time freshmen were enrolled into a section of FYE. During registration, they were given an opportunity to pick their top three choices from the list of FYE topics and most students were able to be registered into either their first or second choice. Based on feedback from the interviews with faculty, the course credit hour was changed to two credits for the seminar and one credit for the transition. In Fall 2016, we had a very full freshman class which resulted in 48 sections (this included 6 Honors sections and increasing the class size to 17 or 18 seats). Many of the pilot titles were repeated with addition of several other titles such as:
- The Science and Culture of Body Modification
- Human Augmentation: Cyborgs & More
- Play with the Past: Athens and the Threshold of Democracy
- The Buddha and Bruce Springsteen
- Sex and the Scriptures
We again used the writing process rubric in Taskstream to analyze the writing of the students.
While there were observable differences in scores for all these categories, the biggest difference came between males and females, very similar to the pilot process. There was a .52 increase in scores of females over males in the category of drafting. And female students scored higher than male students in all categories. By examining the rubric in this manner, we can see what demographic characteristics had the most influence on the scores of the rubric.
The rubric in Taskstream allowed us to assess the first student learning outcome of the new FYE. We are now into the second full year of implementation with a greater emphasis on the drafting and revising process. It became clear through the two iterations of using the rubric that this was an area of weakness with our freshmen students. We are again using the rubric to analyze the writing done during the courses and will track changes in scores as our faculty work to improve these techniques in their students’ writing.
Baldwin Wallace University is a 4-year private, liberal arts college in Berea, Ohio. They are using Taskstream’s Learning Achievement Tools (LAT) and Accountability Management System (AMS) to manage institution and program assessment planning, general education assessment, to prepare for HLC accreditation, and to engage students in the assessment of learning outcomes across programs such as honors, education, health care, public policy, and the first year experience.