By Steve Haberlin (Guest Author)
Recently, we shared findings from a first-ever qualitative study of Assurance of Learning (AoL) practices among AACSB accredited schools at AACSB’s Assessment and Impact Conference in Baltimore, Maryland. The study relied on anonymized excerpts from Continuous Improvement Review (CIR) reports submitted by schools around the world as part of the accreditation process. The aim of the project was to apply a qualitative research methodology to uncover some of the patterns and themes that exist in schools’ approaches to defining learning goals.
Rather than begin with a hypothesis or pre-conceived notions, qualitative researchers begin with questions or wonderments and engage in an inductive, often-iterative process of discovery, to see what the data reveals. Furthermore, AACSB BEI (Business Education Intelligence) staff recommended a qualitative researcher with limited or (in my case) no prior knowledge of AACSB or the accreditation process, thus greatly reducing the presence of bias and pre-conceived notions regarding the data. They wanted to bring a “fresh eye” to what was already being done and what was known or presumed.
The directions were simple: let’s start with the learning goals (the competencies of knowledge and skills the students should learn in the various business programs) and see what’s there.
We began with a sample of 40 business colleges that successfully extended their accreditation through reviews taking place between 2015-16 and 2016-17. The sample was stratified by region and degree programs to reflect the overall population. The focus of the analysis was on the school’s program learning goals (which specify the intellectual and behavioral competencies a program is intended to instill) and the mission statements supplied by those schools in the annual AACSB Business School Questionnaire.
Following an initial review of the data (reading and rereading the report excerpts), data analysis involved several steps, including performing a content analysis of learning goals (in both undergraduate and graduate programs), a comparative analysis of undergraduate to graduate level programs, a comparative analysis of college’s mission statements to learning goals, and a content analysis of Ph.D. program learning goals.
Analysis also included peer review, that is several discussions of the findings with AACSB staff involved with research and the AoL process as well as with Dr. Karen Tarnoff in the College of Business and Technology at East Tennessee State University.
The following findings were reported:
- The majority of learning goals in undergraduate programs were aligned with the general knowledge and skill areas identified within Standard 8 and were highly similar in nature. For instance, 65 percent of undergraduate programs featured a “communication goal” and 57 percent had a “ethics” goal, and 57 percent listed a “critical analysis” goal.
- Graduate learning goals in many cases are highly similar to undergraduate goals in the categories of knowledge and skills addressed (though objectives differed, reflecting a different expected level of mastery). In many cases, the language of the learning goals in undergraduate programs were the same or only slightly different from the school’s graduate program learning goals. In some cases, an additional goal (e.g. leadership) was added to the graduate program goal menu.
- In the majority of cases, the language in schools’ mission statements partially or slightly mirrored the language and concepts reflected in their program learning goals.
- More than half of Ph.D. programs did not feature a specific teaching goal.
So what does this mean? We see this as a first step towards an evidence-based illumination of the approaches that schools are taking and a prompt for further discussion and inquiry, both within schools and among the global network of AACSB accredited business schools. While worth considering that the schools included in this review all successfully passed their individual mission-driven peer review for accreditation, it is reserved for later phases of research and the expert perspectives within the business education community to explore whether what is happening is aligned with what should be. And there are many other dimensions of schools’ assurance of learning approaches to explore in this manner, including the current state of measures, metrics, and processes. We look forward to sharing the phase of this journey and welcome your comments, questions, and ideas
Steve Haberlin is a Ph.D. student and graduate assistant in the College of Education at the University of South Florida in Tampa, Florida. He has a background in qualitative research (which includes publications in The Qualitative Report, International Journal for Arts & Education, Action in Teacher Education, and other journals.) He also works as a research assistant for AACSB.
Comments