Blog: Idea Exchange

The 6th Survey of College and University Chief Academic Officers is a corker (that’s “outstanding” to you)!

Matthew Gulliford February 1, 2017

Last week, Inside Higher Ed (IHE) and Gallup released their sixth annual Survey of College and University Chief Academic Officers (CAOs), supported, in part, by Taskstream.  In brief, the report offers valuable information on how CAOs perceive and manage the challenges confronting higher education institutions in the U.S.  Some of the issues the report focuses on include: increasing minority faculty hiring, the state of liberal arts education, faculty tenure and its future, the value of student evaluations of teaching, and cost-cutting practices taking place on campuses.  Of course, my main interest in the report was what impact have assessment practices had on teaching and learning at institutions across the nation?

This year’s survey set a different tone, compared to previous years, for learning outcomes assessment; illustrating the current trends, growing importance, and perceived value of assessment initiatives, policy, and practice across the U.S.  For example, from 2012-2016, the annual Survey of College and University Chief Academic Officers (CAOs) focused on three questions related to assessment: 1) whether institutions were measuring student learning via standardized tests (figure 1);  2) what tests or assessments of student outcomes institutions were using, such as the National Survey of Student Engagement (NSSE) or the ETS Major Field Test (figure 2); and 3) whether institutions were making effective use of the data they collected from such tests (figure 3).

 

To measure student learning does your institution use at least one standardized test or assessment?

Figure 1

As many of us agree, standardized tests are fine (for certain things) and can always be used to complement or support other assessment initiatives, but debates about the merit and effectiveness of such tests for exclusively assessing student learning outcomes are well known. Creating assignments that focus on specific learning outcomes and then collecting student work for evaluation with faculty designed rubrics is considered to be one of the most meaningful approaches to assessing student learning.  If we look at figure 2, particularly the category of “another assessment” (highlighted), we can see a clear increase in this field over the years; in fact, more than double from when the survey was first introduced in 2012, to 2016.  Although the survey does not provide specific examples of what “another assessment” is, we can assume it includes rubric-based learning outcomes assessment.  This can be corroborated with a similar survey by AAC&U for CAOs in 2016, which found 87% of respondents reporting that they use institutionally created rubrics applied to samples of student work to assess learning outcomes in general education.

 

Which of the following tests or assessments of student outcomes does your institution use?
(Select all that apply)

Figure 2

While the 2012-2016 survey questions shed light on assessment activity, they were somewhat limiting in providing data for determining the impact, value, and effectiveness of assessment practices across institutions.  However, as a sign of the times in this most recent 2017 survey, the questions moved away from a focus on standardized testing and a more pertinent set of assessment questions for CAOs took avail.  For example, this year’s survey asked CAOs whether the growth of assessment systems has improved the quality of teaching and learning at their college; 51 % strongly agree or agree with this statement, with only 20% who disagree or strongly disagree.  59% of CAOs agree or strongly agree that their institution regularly makes changes in curriculum, teaching practices or student services based on what they find from their assessment results (compared to 15% who disagree or strongly disagree). Similarly in 2013, NILOA’s survey of provosts reported that institutions were using learning outcomes findings to a greater extent, more so than their 2009 study, making changes across the board from strategic planning to curriculum modification and resource allocation – although in this survey, accreditation expectations topped the list of how institutions use assessment evidence.  Likewise, NILOA’s survey of provosts in 2013, as with their 2009 study of college leaders, concluded that the prime driver of assessment remains accreditation standards/requirements.  Yet, the 2017 survey illustrated that 48% of CAOs disagree or strongly disagree that assessment efforts are designed more to keep accreditors and politicians happy than to improve teaching and learning (compared to 27% who agree or strongly agree) – I believe this is the first national study to report such findings that moves the assessment needle away from accreditation/compliance and more towards the improvement of teaching and learning (correct me if I am wrong!)

 

Does your institution make effective use of data used to measure student outcomes?

Figure 3

There were some great new questions from this year’s survey which readers may find helpful on their own campuses, including how CAOs believe faculty value assessment efforts on their campus and whether faculty perceive assessment as requiring a lot of work on their parts.  35% of CAOs agree or strongly agree that faculty members are more likely to value than not value assessment efforts, compared to 25% who disagree or strongly disagree.  81% of CAOs also agree or strongly agree that faculty members view assessment efforts as requiring a lot of work, while only 6% disagree or strongly disagree with this statement.  Even though there was a 10% difference between CAOs agreeing that faculty are more likely to value (35%) than not value (25%) assessment efforts, there is clearly room for improving how assessment can be valued more on campus. This could include a greater emphasis towards sharing assessment results/making them easily available to faculty and highlighting the changes/improvements that have been made to teaching and learning as a result of assessment initiatives (in many cases such information is only highlighted in accreditation self-studies).

A high percentage of CAOs agree that faculty most likely view assessment efforts as requiring a lot of work (81%), which undoubtedly affects their likelihood to engage in assessment practices.  Ideas to consider to help alleviate this may include trying to build assessment into the ongoing work of teaching and learning, reframing the work of assessment as scholarship, or investing more in resources to help support faculty with assessment efforts – such as creating a dedicated office of assessment (if you don’t already have one), considering reducing the workload of faculty who participate or coordinate assessment projects, or investing in assessment technology to simplify and manage the process (for more information see Pat Hutchings’ NILOA paper – Opening Doors to Faculty Involvement in Assessment).

Overall, I was very excited by this year’s IHE and Gallup Survey of College of University Chief Academic Officers, particularly the inclusion of more relevant and meaningful assessment questions and I look forward to seeing the 2018 edition! Please share your thoughts on the survey’s assessment results by emailing me at mgulliford@taskstream.com.


* Please note the results presented in this blog are the total percentages for all types of institutions, readers can drill down to institutions by sector (public, private non-profit) and by academic level (doctoral, master’s, baccalaureate, and associate) by viewing the surveys online.

Association of American Colleges and Universities.  (2016).  Trends in learning outcomes assessment. 

Banta, W., Griffin, M., Flateby, T., Kahn, S.  (2009).  Three promising alternatives for assessing college students’ knowledge and skills.  National Institute for Learning Outcomes Assessment.

Benjamin, R.  (2012).  The seven red herrings about standardized assessments in higher education.  National Institute for Learning Outcomes Assessment.

Hutchings, P.  (2010).  Opening doors to faculty involvement in assessment. National Institute for Learning Outcomes Assessment.

Inside Higher Ed., Gallup.  (2017).  Survey of college and university chief academic officers. 

Kinzie, J.  (2010).  Perspectives from campus leaders on the current state of student learning outcomes assessment.  NILOA.