Let me start with this: if you use AI, I do not think lesser of you as a person. I understand the desire for a fast, easy, frictionless solution. Heck, I still think the technology itself is super cool. That said, it is my firm belief that if you want to actually learn something, your far better off avoiding AI as much as possible. When I set out to learn about software engineering, my goal was to avoid using it entirely. I was thus more than a little put off when I was told on the first day of the class that I was expected to use the dang thing for class. Given my experience at the end of the class working on a group project with my peers, (see our project here) I feel vindicated in my perception.
There were a few times were I was forced to use AI or did so out of desperation. In these events, I ended up using Google Gemini, because Google keeps shoving it down people’s throats, and by that I mean integrating it into their other products (in this case Chrome) in every single way they can conceive of. I could not tell you in any more detail what model number or what settings I ended up using, as I never fiddled with those, so just assume the default settings, with the exception that on the final project I did use the “understand this error” button on Chrome’s web developer kit to fix bugs (again out of sheer desperation).
My choice in poison
Ironically, the assignment I used AI the most on in the class was the assignment intended to teach students not to trust the outputs. The set up of the assignment was simple: get AI to write code to check if a given string is a valid email address. Write tests to check how well the AI’s code work. Track your results. Over the course of four prompts, I ended up convincing the AI mostly successfully to implement the standards set in RFC 5322 which dictates what is a valid email. However I would argue I put more work into the assignment than the AI did, and honestly more than the professor who wrote the tests we were supposed to compare our end results with. As it turns out, there are a lot of rules surrounding edge cases for what is a valid email address. Admittedly the solution the AI ended with was incomplete, as RFC 5322 allows email servers that wish to include any UTF-8 character, which means you can have an email address in languages like Chinese and Japanese not supported by ASCII character encoding, but also using silly symbols that aren’t found on most keyboard. While I avoided worrying about covering all those bases, I did ensure the AI implemented some of the other obscure rules, such as allowing otherwise illegal characters in the local component of an address if that local component begins and ends with a double quote. I noticed that the AI was decently responsive in this scenario, where I knew exactly what I was asking for, and knew what to tell the AI it got wrong. For example, I knew that the AI had omitted character limits on the address’s length, though to be fair, the professor who wrote the assignment also appears to have forgotten that. I also am not sure the professor who designed the assignment considered, since it appears part of the point of the assignment was to convince students to be more careful in their AI usage, that such a goal would have much more effectively got that point across if it had not been at the tail end of the semester, after the class’s design, often intentionally, encouraged off putting work onto AI.
Technically the first assignment I used AI for chronologically was an assignment to create five navbars in bootstrap. While I have gotten better over the course of the semester, I was no good at bootstrap when first learning it. I suspect the reason why is because I could not get myself to sit down and watch the two hour tutorial on the dang thing, but in my defense, the tutorial we were provided with was poorly paced, and did not, as far as I could tell, give me what I needed for the assignments that were being assigned at the time. I will note this was also at a point in the semester where my mental state was at its worst, with non-stop stress, and Spring Break too many weeks away to save me, I was suffering from an abundance of melancholy at best. Whether you would call it procrastination or depress, I ended up saving this one assignment until two hours before the deadline… only to realize it was a two-and-a-half hour assignment. Fearing getting a zero on the assignment, I tried to get Gemini to bail me out by doing the work for me. This did not work. As I soon learned, Gemini, if given the choice, will try to do all the CSS itself. Not only did that not really do what the assignment set out, it didn’t even get a lot of the key details right. Thus I decided that this would be the only “experience WOD” I would let myself get a 0 on in the course. It also was the only “experience WOD” that I ever used AI in where AI usage wasn’t a mandatory part of the assignment.
The third and final assignment for the entire class I used AI in was the group project. Like the navbars, this was a shameful moment for me, but having searched the web for StackOverflow answers to my problems, I again turned to Google Gemini. I can’t recall exactly all the bugs I got it to look at, as far as I can tell the “understand this error” button has no way of tracking usage, but I will give a few examples. The first bug I used Gemini for was an error with the secret variable for passwords in the database. This was one of only two bugs I tried to get Gemini to fix. None of the code it suggested fixed the issue. To be fair to Gemini, this one wasn’t actually a code issue as I eventually figured out. The issue was that I needed to put the secret variable in the actual database environment. If you just facepalmed reading that, I don’t blame you, I feel silly for not realizing it sooner. Worse, one of my group partners, Ahron, put a decent bit of work getting this bug fixed in the test environment, but because I was the one whose vercel account we were using to host the database and website, I was the one who had to notice my basic screw up that was preventing this solution from working on the public facing website.
Another bug I asked Gemini to fix was the other major issue that still haunts me from the project, getting the “All Events” page to dynamically render likes and dislikes despite Prisma throwing a fit whenever the database is called in an asynchronous way. Again I tried the code solutions Gemini threw at me, and again they did not work. This time however, it was not because the solution was something Gemini is incapable of. While it’s true maybe Gemini would have done better had I given it all my code, I’m too cautious of a person to be willing to risk the effects that would have in the event of a data breach. (I would worry about Google steeling my code, but I’m not stupid enough to believe they would bother when their web developers are better at web developing than I am.) What I did find Gemini useful for in this case, and most of the other times I used it, was explaining what the error messages actually meant, as many of the errors Prisma, React, and Next.JS throw are not useful in figuring out what actually is going wrong.
While I would rather have you actually read what I had to say, I understand that this essay is long, and tracking what I did or did not say could be tedious, so have an ordered list.
From my experience interacting with AI and more often students using AI for work, AI usage does not help build the skills necessary to succeed, it appears to make students more reliant. While there have been studies claiming that strategic AI usage builds higher order skill function, the key word there is strategic. The problem with applying that to the collegiate level is that this is the level where students are expected to manage themselves. What it would take to ensure that AI usage would actually be helpful to students’ education would require a level of micromanaging I (and likely most students) would be comfortable with and an increase in labor from professors that I am almost certain most would not be willing to put in. The simpler solution is to instead regulate how students are allowed to use the technology. Admittedly there are flaws in detecting AI usage, but that has always been true of other forms of cheating. A workaround for poor AI detection software I have heard of from a professor (and an expert in pedagogy no less) was to make students with responses that look AI generated explain their own argument; students who did their own work would be able to say something about their own work while students who abuse AI tend to have no idea what their own arguments are.
As much as I have used this essay to vent my own frustrations on the technology, I do not think it is useless. There are real world practical uses of AI that are positive. For example, AI has helped make breakthroughs in the field of protein unfolding. Similarly, the technology has been used to predict if patients have skin cancer. Admittedly, early versions of the cancer prediction AI hit a snag when a flaw was discovered in the training data: the AI started determining that the way to detect cancer was to look for a ruler indicating why even good uses of the technology need to be monitored for unintentional bias.
However I would be remiss, on the idea of bias, to point out the inherent contradiction of bias that makes me uncomfortable with letting machines dictate decisions that affect peoples’ lives. I must give credit for introducing me to the issue here to Dr. Adam Groce of Reed College, who taught I took a class titled Private and Fair Data Analysis a few years ago. As I understand it, mathematicians and computer scientists en masse started considering the question of what it means for an algorithm to be fair because of a public spat between the journalists Pro Publica and the company Northpointe over the later’s recidivism tool Compas. A quick note: COMPAS is a tool using linear regression, not a neural network, so it does not fall under what many, myself included, define as AI, but the argument around algorithmic fairness still applies to AI. It still applies even if you don’t consider neural networks to be algorithmic because of issues of determinism. Pro Publica’s original article pointed to racial bias in the justice system that COMPAS amplified. They backed this up with statistical analysis pointing to the higher rate of false positives among African Americans and higher rate of false negatives among white people as an indicator of bias. Northpointe responded with their own research, pointing to COMPAS’s equal accuracy across race, and discrediting the usage of false positives and false negatives as irrelevant as those are statical analysis tools instead of probabilistic tools. Mathematicians started noticing because of this article some unfortunate consequences for algorithmic decision making: most ways of defining fairness that an algorithm are mathematically impossible to hold when one or two others are enforced without requiring a lack in differences in data or an output based on randomness. While I can’t recall which exactly was the first mathematical proof to come because of this, if you are interested in reading more, here is one such article from 2017. To clarify, my point in making this argument is that even good faith application should be cautiously approached as there is no way of eliminating some bias in an AI without opening it up to some new form of bias.
A problem I would like to point out with allowing students to rely on AI in classrooms is that even the studies mentioned above admit that lower concept skills are not improved when using AI. What bothers me having skimmed these studies (forgive me for not having read in full multiple academic articles for a single paragraph of a single essay worth only a small fraction of a grade) is that often what gets labeled as lower order function include some of the most important skills for being able to function in many fields. Skills like being able to read a text and pull out it’s key arguments are skills I would argue that college students are already not good enough at, being reliant on AI for this would make them even worse. While I understand that some might want to encourage offloading these skills off on AI, I personally find that suggestion horrible, after all from my own experience this semester I have seen Gemini fail to notice important details and just be totally incapable of doing work with smaller details at all. If students were to only summaries of arguments presented to them, important nuances or details specific to their situation might be totally forgotten. This of course is all working under the assumption that these tools will continue to be developed in good faith and not poisoned with bias. For an example of why good faith cannot be assumed, I point to Grok.
If I were to play my best at pretending to be the CEO of a big tech company, and present an “ideal” future with more AI in it than is already being shoved upon the public, it would look like the following: image how much money the university could save if it stopped relying on human professors to teach. After all, if you are comfortable with AI giving you everything to learn, why bother having those pesky humans who can only do a few office hours a week at all? A chat bot could be open to student questions at any time. Even better, with AI teachers, students will get answers to their questions even faster as the AI will need less time to think or to fact check their response. Don’t worry about accuracy, the technology already has 90% accuracy and if we assume this accuracy will just keep going up, the AI will never be wrong at some point. This way professors can spent all their time working on their research. Heck, we could get AI to do the grant generating research for us and maybe make some brand new discoveries, like P and $\neg$ P. Heck we could increase efficiency in education by replacing the students with AI too. Image how much learning will happen if entire semesters could be run over the course of a few minutes. We could achieve years of learning in a matter of days!
Comparing traditional teaching methods to AI based education is like comparing the food of traditional restaurant to fast food. Certainly there are traditional restaurants that are bad just like there are teachers who are uninterested in teaching. Fast food will save you from starving and it is fast, but it lacks depth. Similarly, my experience this semester has shown that students reliant on AI were surviving in the class, but end up lacking in the skills the class purportedly intended to teach. It is true that students who used AI were, in fact, faster at achieving small tasks, however a student who I shall not name admitted to me near the end of the final project that they still had no idea how to handle Prisma, the tool we used for database management. It is my understanding that AI usage also harshly impacts students’ knowledge retention, as it is easy to discard information when you perceive it as a short search away. The fast food analogy works again when considering student engagement, fast food is addictive, and similarly AI appears to be very good at getting people to keep using them as much as possible. This is not always good thing. On the flip side, like the varying food quality of traditional restaurants, some professors are good at engaging students while others struggle. Though, like traditional restaurants facing challenges with consumers going for the easy quick fast food, even skilled professors have been struggling recently to engage students who through social media and AI usage have reduced attention spans.
If I were to tell my professor what I would recommend for the future, is to put regulations on how students are allowed to use AI in the class, or at least put more guiding instruction on how to use it more effectively. I can’t claim to know what guidelines would work to make AI usage more effective, as my usage was so limited, and the outputs so disappointing I have no inclination towards using it in the future.
In case you’ve not noticed, or more likely skipped to my conclusion first, I do not think AI belongs in the classroom. My experiences this semester have confirmed my previously held beliefs that AI is neither a suitable substitute for traditional teaching nor a replacement for old school coding and bug fixing methods. While the tool may be useful for those that already know what they’re doing, it is a poorly fit tool for those who are trying learn new things. While the technology has the potential to help people in some fields, its deployment should be carefully considered instead of shoved down the throats of consumers to make it look profitable as fast as possible before investors scramble (something that appears likely to happen soon). I would advise educators to either ban it in their classrooms, or at least to put better guidelines on how it can be used to prevent students from using it to offload all their thinking and lose all opportunities for gaining skills. However I would especially advise ensuring that any anti-AI features in the class be probably evaluated for effectiveness not by seeing if they hamper students with AI, but in seeing how they impact those students compared to those who do not use them. No, putting code snippets as images is not an effective anti AI precaution and no, putting a single assignment demonstrating AI’s unreliability is not sufficient when that assignment has been placed well after assignments worth large portions of the class grades have been due. More generally, I would advise professors to avoid thinking that they can offload their own work by having AI do the teaching, as that will result in students using it as a crutch.