The Naked Scientists
Toggle navigation
Login
Register
Podcasts
The Naked Scientists
eLife
Naked Genetics
Naked Astronomy
In short
Naked Neuroscience
Ask! The Naked Scientists
Question of the Week
Archive
Video
SUBSCRIBE to our Podcasts
Articles
Science News
Features
Interviews
Answers to Science Questions
Get Naked
Donate
Do an Experiment
Science Forum
Ask a Question
About
Meet the team
Our Sponsors
Site Map
Contact us
User menu
Login
Register
Search
Home
Help
Search
Tags
Member Map
Recent Topics
Login
Register
Naked Science Forum
General Discussion & Feedback
Just Chat!
The fair use of LLM and AI in education.
« previous
next »
Print
Pages: [
1
]
Go Down
The fair use of LLM and AI in education.
7 Replies
3065 Views
0 Tags
0 Members and 1 Guest are viewing this topic.
Eternal Student
(OP)
Naked Science Forum King!
1696
Activity:
8%
Thanked: 433 times
The fair use of LLM and AI in education.
«
on:
14/04/2024 02:45:59 »
Hi.
Did you know that a lot of Universities run quite a lot of submitted material, like essays, through a computer to check for plagiarism? You probably did know that.
Did you know they now also include a quick assessment of the probability that this content was substantially created by some LLM (Large language Model - like Gemini, ChatGPT etc.)? You may have known that but LLM's are quite new so they've only recently started it.
If you submit an essay with a high chance of having been generated by a LLM then it can be rejected or the person could be called in for a viva examination.
However, it seems to be bit one sided. There's one student I know who felt strongly that the feedback and comments on their essays were very generic and not especially useful. They ran the feedback through a LLM checker and - well, you guessed it. Perhaps they should reject the feedback and insist the marker comes in for a face-to-face meeting where the essay is properly discussed for 10 minutes?
On the other hand, perhaps it's not all bad. Eduaction may end up becoming a game or a test to see who can game the system better - but we will have grauates who can play the game very,
very
well. Students will be working not just to prevent triggering an LLM detection but to also get future iterations of their LLM assisted essays adjusted in line with output generated by the markers LLM system. This will help the markers AI system pick up key elements and hopefully grade it more highly. We'll have future graduates who are highly skilled with this new LLM technology.
That will be important because LLM and AI will only continue to develop, so our new graduates will need to be good with this technology. There's got to be a bright side to everything.
Best Wishes.
«
Last Edit: 14/04/2024 03:32:03 by
Eternal Student
»
Logged
Halc
Global Moderator
Naked Science Forum King!
2341
Activity:
3.5%
Thanked: 973 times
Re: The fair use of LLM and AI in education.
«
Reply #1 on:
14/04/2024 14:32:34 »
Quote from: Eternal Student on 14/04/2024 02:45:59
a LLM checker
What is a LLM checker? What are the markers that trigger a positive?
How much better are future LLMs going to get such that such tests need to be ever more complex? After all, what a student does is essentially what an LLM does: Research some source(s) and regurgitate the material in altered words. Short of an actual 'new reasearch' project, that's pretty much it.
The checker seems to be a form of the Turning test. Was this text generated by a human or no? All the harder since there is only one-way text and no interaction.
Logged
The following users thanked this post:
Eternal Student
alancalverd
Global Moderator
Naked Science Forum GOD!
19431
Activity:
66%
Thanked: 1655 times
life is too short to drink instant coffee
Re: The fair use of LLM and AI in education.
«
Reply #2 on:
14/04/2024 16:50:34 »
Literary criticism, and to some extent the whole subject of literary studies, has been neatly summed up as "hundreds of pages of bad English written about a few pages of good English", and I guess the same could be said in any other language, so the value of essays in that area must be close to zero and their purpose is difficult to imagine.
Undergraduate essays in factual (e.g. geography) or presumed factual (history) subjects really boil down to a single question: "have you read the book?" So AI has revealed that a great deal of student and teacher time is, frankly, wasted by regurgitation of secondhand facts and opinions.
On the presumption that studying the humanities has some purpose, it is therefore incumbent on those who profit from teaching them to introduce some form of laboratory practical. The pub quiz format is ideal since it tends to the requisite gaussian curve of marks with a large number of questions and candidates, and does test retention of facts (including the written opinions of third parties).
Having established the candidate's diligence in reading or attending lectures, we now need to determine whether he can form and express an opinion clearly, so a very brief viva will waste less of everyone's time than writing and marking a thousand word essay. Though written, Churchill's memo to Montgomery: "Pray let me have, on half a sheet of paper, your plan for invading Italy" sets the tone for a good viva question. From the examiner's point of view it would be a lot more fun than adjudicating a brass band contest, where the scope for interpretation of the test piece is rather limited.
Logged
helping to stem the tide of ignorance
The following users thanked this post:
Eternal Student
alancalverd
Global Moderator
Naked Science Forum GOD!
19431
Activity:
66%
Thanked: 1655 times
life is too short to drink instant coffee
Re: The fair use of LLM and AI in education.
«
Reply #3 on:
15/04/2024 10:09:27 »
Postscript:
The pub quiz idea is even better than I thought. Traditionally, you choose a team name or personal nom de plume, and everyone marks someone else's papers. Now if the identifiers are properly anonymised and coded, you could do this for a hundred students and save thousands of hours of examiners' time.
Reshuffle four times, ignore the highest and lowest, and award the sum of the middle three marks, and you have a completely fair marking system. It's used in ice skating and boxing, where it really matters because money and medals are at stake, unlike Eng Lit where your future career as a politician, bus driver or playgroup knicker lady doesn't depend on the class or even the subject of your degree.
«
Last Edit: 15/04/2024 12:32:16 by
alancalverd
»
Logged
helping to stem the tide of ignorance
paul cotter
Naked Science Forum King!
1800
Activity:
24%
Thanked: 216 times
forum grump
Re: The fair use of LLM and AI in education.
«
Reply #4 on:
15/04/2024 13:22:28 »
Computers assessing the output of other computers. No need for students.
Logged
Did I really say that?
alancalverd
Global Moderator
Naked Science Forum GOD!
19431
Activity:
66%
Thanked: 1655 times
life is too short to drink instant coffee
Re: The fair use of LLM and AI in education.
«
Reply #5 on:
15/04/2024 15:03:41 »
There is already an absurd "public outcry" (journalise for a couple of crank letters to the Daily Mail) because some UK schools will have to close for lack of pupils. Without students, what would Eng Lit, Media Studies and Philosophy lecturers do all day? And how would teenagers fill their time between school and adult unemployment?
«
Last Edit: 15/04/2024 15:05:57 by
alancalverd
»
Logged
helping to stem the tide of ignorance
Petrochemicals
Naked Science Forum King!
3494
Activity:
10%
Thanked: 169 times
forum overlord
Re: The fair use of LLM and AI in education.
«
Reply #6 on:
01/05/2024 00:55:28 »
Quote from: alancalverd on 15/04/2024 15:03:41
Media Studies
Media studies is all the rage now, the internet has made it a mandatory feature for every business, internet site etc. Who needs maths when you have social media?
Logged
For reasons of repetitive antagonism, this user is currently not responding to messages from;
BoredChemist
To ignore someone too, go to your profile settings>modifyprofie>ignore!
alancalverd
Global Moderator
Naked Science Forum GOD!
19431
Activity:
66%
Thanked: 1655 times
life is too short to drink instant coffee
Re: The fair use of LLM and AI in education.
«
Reply #7 on:
01/05/2024 07:12:47 »
Very timely. Last night I bought a drink for ?4.50 and offered the bartender a ?20 note as I had left my credit card at home. He opened the cash register and stood completely baffled for at least a minute, confessing that he couldn't work out what to do next.
The world has changed hugely in the last four years.
Logged
helping to stem the tide of ignorance
Print
Pages: [
1
]
Go Up
« previous
next »
Tags:
There was an error while thanking
Thanking...