(Go stick your head in a pig!)
Come to think of it, “share and enjoy” is exactly the way I would expect an AI-generated YouTube video to end.
(Go stick your head in a pig!)
Come to think of it, “share and enjoy” is exactly the way I would expect an AI-generated YouTube video to end.
You’re paying tens of thousands of dollars to learn how to think critically and do hard shit with your brain. Instead of actually putting in the work, you’re letting an AI do the hard parts. Which defeats the purpose of going/paying in the first place.
I’m sure I would have done the same back then too. But it is short sighted.
I asked my math teacher when I would ever use [whatever we were learning at the time] and his answer stuck with me.
“It’s not about learning how to [thing], it’s about learning to solve a problem in a new way”
I regret not taking it to heart in school but I try to remind myself of that when I “waste” a night working on something “useless” - you learn a lot more than the solution while solving a problem
It hadn’t struck me until now that some think that all you’re trying to learn in school is the solutions to arbitrary problems, but people thinking that makes a lot of sense and helps clarify why they have the posture they do towards education.
Some aspects of education overemphasize memorization so maybe a lot of people think that applies to all of education when it does not.
Yup. Unlike a calculator which just makes doing the things you already understand the steps of doing easier, AI just gives answers whether they are right or not.
I remember a teacher telling me the same thing about calculators way back when. “If you’re not able to do the calculation yourself you won’t know if the calculator’s answer is right or not”
The difference being that if you put in the right equation the calculator will give you the correct result.
AI might give you the right result.
Almost everyone who goes to college is paying tens of thousands of dollars to get a piece of paper so they can get a better paying job.
And considering how many people leave college completely inept I’d say they’re not doing the hard work of learning.
AI is ultimately just a tool, and whether it’s beneficial or detrimental depends on how you use it.
I’ve seen some of my classmates use it to just generate an answer which they copy and paste into their work, and yeah, it does suck.
I use it to summarize texts that I know I won’t have time to read until the next class, create revision questions based on my notes, to check my grammar or rephrase things I wrote, and sometimes I use Perplexity to quickly search for some information without having to rely on Google, or having to click through several pages.
Truly it isn’t much different from what we used to do around 2000-2015, which was to just Google things and mainly use Wikipedia as a source. You can just copy and paste the first results you find, or whatever information is on Wikipedia without absorbing it, or you can use them to truly research and understand something. Lazy students have always been around and will continue to be around.
It’s bad at that, because effective summarization requires an understanding of the whole, which AI doesn’t have.
The difference between what you’re doing and what people were doing 10 years ago is that what they were doing was referencing text written by people with an understanding of the subject, who made specific choices on what information was important to convey, while AI is just glorified text prediction.
LLMs can learn skills beyond what’s expected, but of course that depends on the exact model, training data and training time (See concepts like ‘emergence’ and ‘grokking’).
Currently the models tested in the study you mention (Llama-2 & Mistral) are already pretty outdated compared to other LLMs that lead the rankings. Indeed, research looking at the summarization capabilities of other models suggests that human evaluators rate them equal to or even better than human summarizers.
Well that’s a different argument from the first commenter, but to answer your point: The key here is trust.
When I use an AI to summarize text, reword something or write code, I trust that it’ll do a decent job at that - which is indeed not always the case. There were times when I didn’t like how it wrote something, so I just did it myself, and I don’t use AI when researching or writing something that is more meaningful or important to me. This is why I don’t use AI in the same way as some of my classmates, and the same is true for how I use Wikipedia.
When using Wikipedia we trust that the contributors who wrote the information on the page didn’t just nitpick their sources and are accurately summarizing and representing said sources, which sometimes is just not the case. Even when not being infiltrated by bad actors, humans are just flawed and biased so the information on Wikipedia can be slanted or out of date - and this is not even getting into how the sources themselves are often also biased.
It’s completely fair to say that AI can’t always be trusted - again, I’m certainly not always satisfied with it myself - but the same has always been true of other types of technology and humans themselves. This is why I think that even in their current, arguably still developmental stage, LLMs aren’t more harmful than technological changes in information we’ve seen in the past.
The second study link you gave didn’t find that it was ‘better’ than human writers, it concluded that if you do a lot of fine tuning then it can summarize news stories in a way that six people (marginally better than n=1 anecdote in the first link, I guess?) rated on par with Amazon mturk freelance writers. And they also noted that this preference for how the LLM summarized was individual, as in blind tests some of them still just disliked it. There are leagues and leagues of room between that and “summarizes better than humans.”
You and I both know that 99.9% of people are not fine tuning LLMs that way when they ask for a summary, which means almost nobody is going to be getting that ‘kind of as good as a person’s summary maybe if you like that style of summarizing.’ They’re getting the predictive text slop. Like, good for you if you aren’t, but maybe you should be a bit more upfront about how little you trust it and how much work you have to do to get it to give you an accurate (maybe?) summary?
My problem with LLMs is that it is fundamentally magic-brained to trust something without the power to reason to evaluate whether or not it’s feeding you absolute horseshit. With a human being editing Wikipedia, you trust the community of other volunteers who are knowledgeable in their field to notice if someone puts something insanely wrong in a Wikipedia article. An LLM will tell you anything and phrase it with enough confidence that someone with no expertise on a subject won’t know the difference.
There are several problems with these arguments.
It concluded quite the opposite actually. From the “Instruction Tuned Models Have Strong Summarization Ability.” section:
You might be confusing instruction-tuning with fine-tuning for text summarization. Instruction tuning involves rewarding a model based on the helpfulness of its responses in a user-assistant setting, and it’s the industry standard ever since the first ChatGPT showed its effectiveness.
Also they actually recruited thirty evaluators from MTurk, and six writers from Upwork (See “Human Evaluation Protocol” and “Writer Recruitment”).
Their conclusions are also consistent with the study you linked to, since which they fine-tuned the Mistral and Llama models in an attempt to generate better summaries but the evaluators still rated them lower to the human summaries. Though I’m not sure that you will be convinced by this study either since, as they state in the “PHASE 3 – FINAL ASSESSMENT” section:
Even putting all of this aside, you can actually use custom ChatGPTs that have been fine-tuned specifically to write summaries and test them for yourself if you want:
The exact same thing can be said about the lower scores in the study you linked to, so what is the exact threshold? Would you only trust an AI to summarize things if 100% of humans liked it? Besides, even if you think the best model in the study was still not good enough, there are other, even better models that have been published since then, like the ones at the top of the aforementioned leaderboards, and others like GPT-4o, OpenAI o1 and OpenAI o3.
That’s why I linked to the first article where they specifically asked an actual lawyer to evaluate summaries of legal texts written by LLMs and interns - and as we can see he thought the AI was better.
Whenever I’ve gotten into debates about the philosophy of AI and its relation to things like art, reason and consciousness, the arguments I’ve seen always end up being rather inconsistent and condescending, so I’m not even going to get into that. However I will point out that if we take the general definition of reasoning to mean “drawing logical conclusions through inference and extrapolations based on evidence”, the Wikipedia pages on LLMs, OpenAI o3 and “commonsense reasoning” explicitly describe AIs as reasoning. You’re welcome to disagree with this assessment, but if you do I hope we can then agree that, as I stated previously, Wikipedia contributors and their sources aren’t always reliable.
But sure, let’s put that aside and assume that reasoning is a magical aspect of the human brain that inherently excludes AI, so LLMs simply can’t reason… So what?
AlphaFold can’t reason, but it still can predict the structure of proteins better than humans, so it would be naive to not use it simply because it doesn’t reason. In the same vein, even if you want to conclude that LLMs can’t reason this doesn’t change the fact that they are useful tools, and perform either equal to or even better than humans in many tasks, including summarizing text.
LLMs are, for all intents and purposes, just really complicated functions that model some data distribution we give to it. Language obviously has a predictable distribution since we don’t speak/write randomly, so given proper data and training there’s no reason to believe that an AI can’t model that even better than humans. Hell, we don’t even need to get so conceptual and broad with these arguments, we can just look at the quantitative results of these models, and assess their usefulness ourselves by simply using them.
Again, I don’t trust everything these AIs generate, there are things for which I don’t use them, and even when I do sometimes I just don’t like their answers. But I see no reason to believe that they are inherently more harmful than humans when it comes to the information they generate, or that even in their current state that they’re dangerously inaccurate. If nothing else I can just ask it to summarize a Wikipedia page for me and be confident that it’ll be accurate in doing so - though as the links I mentioned demonstrate, and as you may have come to believe after considering Wikipedia’s assessment of AI reasoning, the Wikipedia contributors and their sources aren’t 100% reliable.
We both know that humans fall for and say absolute horseshit. Heck, your comment is a good example of this, where you moved the goalpost again, failed to address or outright ignored many of my arguments, and didn’t properly engage with any of the sources cited, even your own.
If you just dislike AI on principle because this technology inherently bothers you that’s fine, you’re entitled to your opinion. But let’s not pretend that this is because of some logical or quantifiable metric that proves that AI is so dangerous or bad that it can’t be used even to help university students with some basic tasks.