This is Brad DeLong's Grasping Reality—my attempt to make myself, and all of you out there in SubStackLand, smarter by writing where I have Value Above Replacement and shutting up where I do not… I think Paul Musgrave has it correct here…This is, I think, 100% correct for those professors who want to get real feedback with respect to what their students are learning, or who simply want the grades they give to not be grossly unfair and to do something to motivate students to study:
Then of the four options for who to examine face-to-face he offers, I vastly prefer his option 4:
These students are going to have modern advanced machine learning models where their descendants at their elbows for their entire lives going forward. They need to get good at using them productively. Teaching students to perform intellectual exercises in a way that assumes these things away is not, in the long run, going to do much to help anybody. So: write the essay outside of the classroom, submit it, and then come in to talk about it. Paul goes on:
This means, at least as I see it, half-hour blocks: ten minutes reminding yourself what the student’s outside-the-class work was about, a three-minute opening presentation, your first question and their answer, ten minutes of back-and-forth, a thank-you and five minutes making your notes, and then on to the next student. Give them the questions beforehand, then pick one or two; then follow-up: Did the student answer the question, respond to the query, engage with the course material, reason with the material and arguments, appear well-prepared? And then comments… “this isn’t grading a presentation, it’s getting into the student’s head…” 32 99+ Academia & MAMLMs: The Seven Labors of the Academic East-African Plains ApeHigher education, “AI”, and the eternal return of learning in the context of 5,000 years of pedagogical history from cuneiform to chatbots: why every new technology—from clay tablets to machine...Jul 01, 2025 Higher education, “AI”, and the eternal return of learning in the context of 5,000 years of pedagogical history from cuneiform to chatbots: why every new technology—from clay tablets to machine learning—fails to dethrone the real work of education, and why that provides a very strong case for optimism (and oral exams) in an age of AI panic and academic hand-wringing…
If you think MAMLMs spell doom for universities, think again. Start this AM with the wise Chad Orzel of Union College:
And I endorse this 100%. And now let me offer some strong pushback against the “Technological Change Crisis in Academia!!!!” branch of the “Crisis in Academia!!!!’ industry: Start with something I consider very obvious: For the past 5000 years, ever since the invention of writing, higher education (and lower education too) have really had one overwhelming purpose: to equip people to be front-end nodes to the East African Plains Ape Natural Anthology Super-Intelligence—the EAPANASI. As nodes in that and able to draw on that anthology super-intelligence, trained white-collar workers have, for 5000 years now, drawn on its knowledge and wisdom, remix it as they apply it to their own situation, do some information processing of their own, and then upload their conclusions and insights to add to the store on the one hand and use their conclusions and insights to inform others and act in the world on the other hand. Training people to be such front-end nodes has always involved training them to do seven things:
As in -3000, when a key part of being a scribe was learning how to mix the clay to the proper smoothness and consistency so that it would take the imprint of the cuneiform stylus, so today. That is the center-of-gravity of formal education—lower, and higher—aimed at literacy and numeracy, at letters and numbers as data codes. It was, is, and shall be. As a professor, this is what you do as you teach your subject. You start with the survey of the subject, and you go on from there. You teach the answers to the interesting key questions along with enough context that your students can grasp why these are key questions and why the answers are interesting. You also model the process. And then you try to drag them, kicking and screaming, into practicing the process as well. Now, the technologies of literacy and numerous do undergo many and substantial changes: things are not at all the same in terms of what materials come before your eyes and what you then do with them in order to be an effective front-end node. Noting, browsing, skimming, reading, attending to, delving into, and wrestling with individual texts are all modes of engagement that have their place always, but the balance shifts. As does how far you can chase down reference rabbit-holes. And there are a similar bunch of different modes of interaction for dealing with collections of numbers. And now we have MAMLMs to deal with. MAMLMs—modern advanced machine-learning models—are software program devices that run on doped silicon hardware devices to engage in very big-data, very high-dimension, very flexible-function classification, regression, and prediction analyses at a scale that was previously unimaginable. One very prominent of the use cases of these brand-new devices is as natural-language front-ends to structured and unstructured data stores. A second very prominent of the use cases is as a prose (and code) slop machine: take a ChatBot version of MAMLMs, feed it a piece of a conversation, and it will then output its interpolated stochastic guess about what a typical internet s***poster would say or write to continue that conversation. How can it do this? Because the weights that have been fitted to its neural network are those that get it to be, well, as good as possible at taking an internet conversation and matching what a typical internet s***poster did say to continue the conversation. (Plus there is RLHF, and other things making the actual picture a little more subtle.) But how has all this come to be seen as a mortal, existential threat to higher education as we have known it, rather than simply another shift, like the coming of papyrus, or the scroll, or the codex, or Gutenberg, or the pamphlet, or mass media, or—indeed—< http://arxiv.org > and the internet? How is it seen to be threatening to end it as an intellectual enterprise? How is it seen to be threatening to end it as an economic sector of production networks that provide many, many stakeholders with very good livings on the one hand and skills useful for enriching them and enriching their lives on the other? Now come before the bar Sean Illing of The Grey Area podcast <
> and James Walker of New York Magazine’s “Intelligencer” section <https://nymag.com/intelligencer/article/openai-chatgpt-ai-cheating-education-college-students-school.html> to wring their hands. They are assisted in this task by Walker’s sources, both students and professors:
Yes, technologies of literacy and numeracy have undergone many near-complete transformations over the past 5000 years. Yes, this might be another one—although the coming of video and MOOCs and so forth, and indeed the fact that the lecture survived Gutenberg, impresses me with the stability of the university as a system. And yet the fundamental tasks we academics train people for—(1) through (7) above—have remained visibly the same even though the literate and numerate procedures to accomplish them have changed. Thus, in my view at least, the right way to handle “AI” in higher education is to spend the first day of a class surveying all the changes in the idea of the university starting from the days of Héloïse d’Argenteuil and Peter Abelard, to tell this story:
Start there. And end the course with each student taking a ten-minute live oral one-on-one final exam. If it is an oral exam, I will take their final project and ask them to outline how they went through steps (1) through (7) and whether they found they found the information technology they used helpful or not. And, no, I will have those oral exams administered over SMS by SubTuringBradBot. Ten minutes is enough to gauge how much a student understands the final project that they have submitted. Thus we can still use the grade goad to get them to do the work. And then all of the problems leading to the Illing and Walsh handwringing seem to me to melt completely away. That, I think, is so, as long as we are willing to invest those ten minutes per student per semester per course—figure, for a class of 60, what with scheduling overhead, preparation, and such, an extra 20 hours of professor time devoted to these tasks. Now if you are a professor spending ten hours a week on your particular course for sixteen weeks a semester, this is a work-speedup of 1/8. It is a real change. And it is an added cognitive load. First, it is such in terms of engaging with students one-on-one. Second, it is such in terms of figuring out what the right assignments are to get students to practice doing (1) through (7) in the most effective way. That, especially, will be hard. You will also have to figure out how to do (1) through (7) in the MAMLM new technological world, before you can teach your students how to do it, and before you can design the right assignments. There is, however, one thing that gives me great pause. It seems to me very clear what the coming of MAMLMs to academia brings in terms of opportunities, challenges, and required adaptive action to maximize the potential win. Yet that is not a reaction I see around me and outside in the broader culture. What I see is, indeed, the cheating panic and the hand-wringing over it which are, these days, more than merely a constant background hum. To recap: two things that are stable and constant in academia are (a) technological panics at least since the days of Platon’s Phaidros and (b) a constant essential purpose in training knowledge workers so that they can enrich their lives and be useful to others. The patterns of workings that constitute humanity as an anthology super-intelligence have changed profoundly and repeatedly. Thus the way that individuals become effective front-end nodes of and to that anthology super-intelligence have changed as well. Yet it is still, at bottom, the familiar seven academic labors: surveying, questioning, researching, analyzing, storing, and persuading. And academia exists to (a) teach students the context and the answers to key interesting questions, (b) modeling the process of being an effective front-end node, and (c) goading them to practice becoming effective front-end nodes. From this perspective, while the coming of MAMLMs does require a substantial pedagogical pivot, that pivot is easily accomplished because it is still possible via one-on-one q-&-a sessions to quickly ascertain to what degree the document presented as evidence that the student has practiced the seven academic labors are in some real sense “their own work”. The question is thus whether academia is too ossified and blinkered to perform what I see as a simple and straightforward pedagogical pivot with respect to assessment and its usefulness as a goad to getting students to do the work. MAMLMs are an existential threat only if it is. There is another question, however, if we want to maximize the potential win from MAMLMs. It is this: What exercises should be assigned in this new age for students to practice their seven academic labors? Figuring out the answer to that question will require a lot of experimentation and evaluation, and considerable thought and insight. But when I look at these challenges and opportunities, my reaction is one that I think would please Chad Orzel. This is my reaction: This is going to be fun! References:
If reading this gets you Value Above Replacement, then become a free subscriber to this newsletter. And forward it! And if your VAR from this newsletter is in the three digits or more each year, please become a paid subscriber! I am trying to make you readers—and myself—smarter. Please tell me if I succeed, or how I fail… #academia-&-mamlms |
| ||||||||||
No comments:
Post a Comment