At Art & Science Group, we know that the scientific interpretation of data includes artistic pursuit. We spend hours seeking to better understand the multifaceted stories data reveals about best practices for institutional strategy. As an homage to the nuances and complexities of data analysis, we’ve created this thought leadership series offering market insights guided by our work engaging with institutions across the country and reflecting on how that information can inform strategic thinking for leaders in education. As you read, we ask only that you take a moment to Look at it This Way…
ARE ROBOTS WRITING COLLEGE ESSAYS?
Craig Goebel, Principal | October 17, 2023
Only a little bit at the moment.
As a father working in the higher education industry, the looming specter of the college application process for my high school-aged son occupies a prominent place in my thoughts these days. Having spent nearly three decades working closely with numerous higher education institutions, I believe I can offer valuable insights to my son as he embarks on his college selection journey. However, I am also acutely aware that today's students have a wide array of technological resources at their disposal, a stark contrast to my own college application experience, which occurred in the pre-internet era (or, as my son humorously puts it, before the invention of the wheel).
Among the technological advancements that have captured not only my interest but also that of the entire education sector are generative AI technologies, such as ChatGPT—the robots. I've often wondered how my son and his peers are utilizing these tools both in their academic pursuits and, more specifically, as they navigate the daunting and pressure-filled college application process. Being deeply entrenched in the higher education industry, I can't help but ponder a broader question it raises for the sector: Is ChatGPT a valuable resource for those with fewer advantages in the college application process, or does it further privilege those who already possess the substantial resources necessary to navigate the winding road to college?
The question is timely as autumn and the college application season descend upon us and, with them, the ever-dreaded task of crafting the college essay. The process is fraught with anxiety and consumes countless hours of the lives of many high school seniors (often at their parents' constant nudging!). With the number of apps seniors are submitting increasing, coupled with the rising selectivity of many colleges, it seems logical to consider if prospective college students are eagerly embracing the assistance of AI platforms like ChatGPT to save time and bolster their chances of admission into their dream institutions. If recent reporting on the subject is any indication, we should most likely anticipate an uptick in students deploying these digital tools to tackle school assignments and even, perhaps, to produce the perfect college essay.
In an era marked by seismic shifts in higher education, the college essay is gaining newfound importance. With SAT requirements on the decline in the wake of the pandemic and recent changes to affirmative action, the essay's role in the application process is on the rise. Colleges may devote more time to scrutinizing these literary works (increasingly through the use of AI), potentially leading to more students turning to tools like ChatGPT for essay-writing assistance in their quest to enhance their applications.
However, let's not jump to conclusions just yet. Are students truly harnessing the power of these robots to draft their college application essays? To find out, we conducted a survey among college-bound high school seniors this past spring, posing three simple questions about ChatGPT:
How much have you heard about ChatGPT?
Have you ever used ChatGPT?
What have you used ChatGPT for?
The results were illuminating. Most students had at least a passing familiarity with ChatGPT, with a substantial two-thirds indicating some degree of awareness. Digging deeper, we inquired whether students had actually used ChatGPT, and nearly two-fifths confirmed that they had.
Delving further still, we asked those students who indicated they had used ChatGPT to detail how they employed this tool. The data revealed a diverse range of functions among students who had engaged with this AI. Interestingly, nearly three-fifths stated they have been using ChatGPT for brainstorming ideas or simply "fooling around." A smaller but still noteworthy contingent, roughly a quarter, reported using it for more academic purposes, including writing assignments or essays (26% and 23%, respectively). Surprisingly, a mere 9% of those students who used ChatGPT claimed to have used it for writing college essays, representing only 3% of the total population of students surveyed.
Does this revelation suggest that the fears of AI detractors, who have made headlines expressing concerns about its impact on education, are perhaps overblown? Does AI spell the demise of critical thinking, necessitating a reevaluation of academic standards for students? As of this past spring, while some concerns may be justified, especially with nearly a quarter of the students who indicated using ChatGPT are using it for school assignments, the domain of college essays appears relatively unaffected—for now.
However, it's essential to recognize that not all ChatGPT users are alike. While many are using ChatGPT for some purpose, it’s important to understand who is using it the most. In other words, who potentially gains advantages through the use of ChatGPT? Our survey data reveals intriguing differences among certain demographics, with some more likely to be not only very familiar with ChatGPT but also to have used it. When only looking at those who indicated they are very familiar with ChatGPT and used it, the standout populations include males, students from private high schools, non-first-generation students, and those residing in urban or suburban areas.
More broadly, males, non-first-generation, White, and Asian students are more likely to be at least somewhat familiar with and have used ChatGPT than their peers. Additionally, we found that while middle-income students were more likely to indicate they were somewhat familiar with ChatGPT than their upper- or lower-income peers; interestingly, we found no differences in ChatGPT usage by income groups.
What do these distinctions imply? Could resource disparities play a role in students' access to and familiarity with ChatGPT? Are the same inequities that have historically limited women's access to STEM fields manifesting themselves in access to this technological tool? Is ChatGPT now among the myriad additional resources to which White, Asian, and non-first-generation students have more access to facilitate a leg up in the admissions process while Black, Hispanic, Mixed Race, and first-generation individuals don’t? If these assumptions hold true, we need to find ways to factor in these advantages (and disadvantages) when evaluating the quality of essays submitted to admissions committees. More importantly, in considering the broader impact of ChatGPT on higher education, if certain students have less access to resources that could assist them not only with their schoolwork but also with their college applications, could prohibiting the use of AI in education potentially further disadvantage already marginalized groups? And, if colleges are using it to assess students’ applications, why shouldn’t students use it as well?
In reality, much of the road ahead remains uncertain. The influence of AI tools like ChatGPT on higher education is a puzzle waiting to be solved, and it seems that each day brings new perspectives on its potential impact. However, as colleges strive to enroll a more representative student body while fostering inclusive environments, understanding the influence of these digital tools becomes increasingly crucial.
As we navigate the unpredictable seas of higher education's future, let us remember that, in the grand tapestry of academia, ChatGPT is, but one colorful thread, and its ultimate significance is still unfolding. Will these robots reshape the academic landscape, starting with the college admissions process? Only time will reveal the answer. In the meantime, the best I can do is support my son and ensure that he understands the implications of using a tool like ChatGPT and that he knows I was born well after the invention of the wheel.
So you know…
ChatGPT helped write this piece. To be honest, it didn’t do a great job in the first draft. There was little in the way of thoughtful analysis, and if this had been a college assignment, I would not have turned it in as is. After a draft or two of my own, I did ask ChatGPT to make my writing more “readable”. It made some changes that I liked, but I still went back and “edited the edited edits”, to align with what I felt was appropriate. Perhaps I’m not yet an adept ChatGPT user, and my prompts were inadequate. Ultimately, I’d consider ChatGPT a lazy co-author or uninspired copy editor to this piece at best.
Study Methodology
The findings in this issue of Look at it This Way are based on survey research fielded in May-July 2023. Surveys were completed by 2,087 domestic high school seniors, including 802 who intended at that point to attend a four-year institution as a full-time student this fall.
Respondents were 62% female and 60% white. The average reported household income was around $94,000. Responses are weighted by income, race, region, and gender so that findings represent the larger domestic college-going population. The margin of error for this population of students is plus or minus 3.5%.
The study was designed to provide a broad perspective on the market challenges facing higher education. Findings and conclusions, therefore, do not reflect the circumstances, challenges, and opportunities of any individual institution, which tend to be highly idiosyncratic. Rather, they are intended to contribute to the national conversation around access to higher education, inform policymakers, and prompt each individual institution to consider what its own particular situation might be with regard to this topic and how it can determine what its own best solution might be.
For the purpose of assessing differences across student populations, our analysis divided students into the following racial/ethnic groups. Students self-identified as: “Asian”, which included Asian or Asian American and Indian, Indian American, Indo-American, or Asian Indian; “Black” included Black, African American, African, or Caribbean; “Hispanic” included Hispanic, Latino/Latina/Latinx, or Mexican American; “White” included White, Caucasian, or European; and “Multi-racial/Other” included American Indian or Alaskan Native, Middle Eastern, Arab American, North African, Native Hawaiian or Pacific Islander, and those who identified with multiple groups.
