[scientizscht (OP)]

Hello!

Obviously ChatGPT is something unprecedented from the machine learning sector.

Do you follow these advances closely to explain what is actually happening?

Is it simply training statistically a model so that it can put together words that make sense? I am trying to figure out if it is something more than that, because it can be quite advanced in its responses though I do not see much of originality of course.

Thank you!

[Eternal Student]

Hi.

Do you follow these advances closely to explain what is actually happening?

   No, not really.   Just enough not to be completely left behind.   It's something that is quite likely to have some impact on most people in most walks of life. 

Is it simply training statistically a model so that it can put together words that make sense?

    What do you mean by "statistical model"?    Some aspects are statistically based and machine learning is very much about training a system on lots of input and allowing it to modify the processing until a suitable output is reliably obtained.  The modifications are not always random but guided by a return or feedback from the output in various ways.   In the early stages of training ChatGPT there was human supervised learning so that humans could influence the processing directly and/or certainly provide more feedback to rapidly shape the neural network.   In later stages the Machine Learning becomes much more autonomous.

ChatGPT is a neural network system based on artificial (electronic) components.
     ChatGPT initially used a Microsoft Azure supercomputing infrastructure, powered by Nvidia GPUs, that Microsoft built specifically for OpenAI and that reportedly cost "hundreds of millions of dollars". Following the success of ChatGPT, Microsoft dramatically upgraded the OpenAI infrastructure in 2023.
   

[Taken from Wikipedia:  https://en.wikipedia.org/wiki/ChatGPT ]

    It's not so much the hardware that's important but the neural network architecture implemented on it that is interesting and ultimately makes it work.   The hardware just allows it to work fast enough to be usefull.

   GPT models are artificial neural networks that are based on the transformer architecture (see below), pre-trained on large datasets of unlabelled text, and able to generate novel human-like text   

Further details:  https://en.wikipedia.org/wiki/Generative_pre-trained_transformer

   A transformer is a deep learning model that adopts the mechanism of self-attention, differentially weighting the significance of each part of the input (which includes the recursive output) data. 

Further details:  https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)

    If you're not all that familiar with what a basic "neural network" is then you'd actually be better of starting here:
https://en.wikipedia.org/wiki/Neural_network#Artificial_intelligence

     There have been several threads in this forum discussing ChatGPT recently.   Some of them were discussing AI in a very general sense.   Some were posts entirely generated by ChatGPT and usually prompted by what might be officially termed as "garbage" input.   This comes from the acronym  G.I.G.O.  that is used in Computer Science and just means that if you put Garbage in to a system then you are very likely to get garbage produced as an output.  These sorts of posts are now almost becoming regarded as SPAM.  At least once, ChatGPT has been used to generate a reply to someone else (I did that - just to see how well it would do).   
    At the moment ChatGPT has been very one-sided and is responsible for generating many of the original posts that are in the forum    BUT   we can use ChatGPT to replace the need for anyone to actually be here and give replies to those (or any) posts.   This is something I've been considering quite a lot recently but I'm not staff or involved in the administration of this website. 

Best Wishes.

[alancalverd]

we can use ChatGPT to replace the need for anyone to actually be here and give replies to those (or any) posts. 

As a moderator, I'd be very worried if anyone thought the kind of repetitive and unreferenced crap that ChatGPT produces was characteristic of the level of intelligence and clarity that characterises the better replies in this forum.

[evan_au]

Obviously ChatGPT is something unprecedented from the machine learning sector.

I wouldn’t call it unprecedented. Chat GPT 4 is merely an extension of Chat GPT 3.5 - with more input data, a more powerful training processor, and a much larger in-memory model during execution.
One worrying trend is that the amount of energy spent on training seems to be growing exponentially..

[Zer0]

we can use ChatGPT to replace the need for anyone to actually be here and give replies to those (or any) posts. 

As a moderator, I'd be very worried if anyone thought the kind of repetitive and unreferenced crap that ChatGPT produces was characteristic of the level of intelligence and clarity that characterises the better replies in this forum.

Umm...keeping my excessive Bias against A.I. & A.G.I. aside...

MayBee...

If ChatGPT could be Embedded into the Forum, if the Implementation could be Free of Monetary costs, then perhaps it's Worth a thought.

It might Attract more New users/members.

It's Output could be assessed in Real Time & Compared with Human generated answers.

MODS could Utilize the A.I.s rewards & corrections Inputs to Teach it to be Better.

Perhaps, OpenAI should be the ones to ideally put forward the proposal first from their end to TNS.

Donno if It could help with Solving Spam & Other issues the Mods face.

We could learn a thing or two from it, & in return, it could learn Alot from Us.

[Halc]

If ChatGPT could be Embedded into the Forum

It can't.see the forum. It has no internet access. It is strictly a server, answering incoming requests and putting out no queries of its own.

MODS could Utilize the A.I.s rewards & corrections Inputs to Teach it to be Better.

It is incapable of learning. You can correct it and it might acknowledge the correction, but ask the same question again tomorrow and it's just as likely to get it wrong again, but of course worded differently.

it could learn Alot from Us.

It can't unfortunately, hence it can never qualify as a general intelligence, or an AGI. It's why they keep coming out with new versions with more up to date training data and more powerful servers. The new versions still can't learn, but they supposedly give a lower percentage of wrong answers and can hold a conversation longer without forgetting how it started.

[Eternal Student]

Hi.

    Just to clarify what @Halc has just said:

1.  The moderators can't teach ChatGPT anything but the development team at OpenAI can do that.

2.   

It is incapable of learning.

     From us out here.   In development by OpenAI some of the learning was human supervised but the later development was much more autonomous.   A lot of this was already done just for the transformer architecture rather then the very specific implementation and incorporation of that into the ChatGPT system.   For example, it was trained on most of the contents of the internet (from an offline archive of the contents of the internet) as it was about 3 years ago.   The precise details of what was done is not known to me but we can simplify it to something like this:   I want to be very clear here - I am just presenting a very simple version of something that can be done just because it will be easier to understand it this way.   If you want actual details then you need to review the links already given in earlier posts etc.
       The system could sample the first 200 words of an article.    Then make an attempt to generate what should be the next 100 words of the article.  Then compare what was generated by itself with what was said in the article.   Various statistical analysis would be applied automatically to asses "how close" the system had got.   Various numerical values are then obtained which act as feedback for the system.   The feedback will then automatically alter the neural network.    The system will then move on to another article and repeat the process.  If it scores slightly better on matching the next 100 words then it keeps the modifications that were made just previously to the neural network, if not it will undo the modifications.   As I've said, this is very simplified, the exact process of modifying the neural network is not going to based on just the previous attempt and the current attempt.   The system will be a vast and complicated interconnection of nodes with an elaborate process for modifying these connections in response to an array of feedback signals produced after evaluation of its objective function(s).
       What you have to understand is that all of this process,  sampling an article, generating output text, comparing and scoring against the real text, feedback to the neural network  etc.   -  goes on automatically and entirely without a human being needing to be there.    You just set up the system to start, give it access to the archive and tell it to go.... wait for long enough and you have a reasonably well evolved neural network with considerable ability.   That is what we can reasonably describe as autonomous learning.
    However it is not a genuine  AGI (Artificial General Intelligence),  it has an objective function (or many of them) which involve writing better or more human-like text.   It does not have an objective function that would modify the neural network to solve a different sort of problem, or even to have any way of identifying what another problem might be.   For example, if a rat got into the computer room and starting eating the cables, then it has no objective about eliminating rats.  The whole event was not any sort of text input, it does not trigger the processing of any objective function and so it won't generate feedback that can modify the neural network.   It will not learn anything about rats running around in the computer room.
    I hope that makes some sense.

Best Wishes.

[Zer0]

Hmm...Sorry i had a different incorrect Understanding of ChatGPT.
Thanks for the Corrections!

I've never used it.
I went on the OpenAI page once or twice, read the T&C, n called it Quits.

All my little info on the Topic comes from pop sci media articles & Utube.

I had read about RLHF(Reinforced Learning from Human Feedback).
Thought OpenAI would feel Obliged to provide Site Mods with that access power.
Coz Mods here are Smart & Wise, so why not.

Had also read about some Plugins stuff.
Like video, audio, inputs etc.
Thought it was capable of image scanning n understanding contents.
Like feed it an image, n it could recognize n provide output responses.

I suppose Bing uses it or has it integrated.
Wonder how folks use it to make Appointments.
Donno how it sends emails n even books a restaurant table on the User's behalf.
(Without Internet or Input Access)

So if it has no Internet access, then it didn't really Directly fool TaskRabbit to solve a Captcha.
There was a Human involvement in the middle of the process.

Had heard when IT was unable to solve a Calculation, it requested for access to an online Calculator...huh!
(& Yaa, it solved it n got it right lol)

Recently saw an article header stating it can fight Spam, but sadly i  did not read it.

https://arxiv.org/abs/2303.12712

(Soo, just sparks...no smoke, no fire)

Guess now I'll just wait n watch what DeepMind is capable of.

[hamdani yusuf]

Here's a first hand information intended for general audience.

The Inside Story of ChatGPT’s Astonishing Potential | Greg Brockman | TED

In a talk from the cutting edge of technology, OpenAI cofounder Greg Brockman explores the underlying design principles of ChatGPT and demos some mind-blowing, unreleased plug-ins for the chatbot that sent shockwaves across the world. After the talk, head of TED Chris Anderson joins Brockman to dig into the timeline of ChatGPT's development and get Brockman's take on the risks, raised by many in the tech industry and beyond, of releasing such a powerful tool into the world.

[Zer0]

Why do They not provide IT with Sensors?

Visual, audio, temperature, infrared, night vision, motion detection etc etc.

Then when it senses Movement, it could Analyze what it is, n if it's a Rat, then IT could sound an Alarm.

Meow Meow or Boo BooH!

[hamdani yusuf]

Why do They not provide IT with Sensors?

Visual, audio, temperature, infrared, night vision, motion detection etc etc.

Then when it senses Movement, it could Analyze what it is, n if it's a Rat, then IT could sound an Alarm.

Meow Meow or Boo BooH!

They will, eventually. Tesla's Optimus and Google's robot are going in that direction.
I guess they are preparing risk mitigation strategies before launching their products in a larger scale.

[alancalverd]

Then when it senses Movement, it could Analyze what it is, n if it's a Rat, then IT could sound an Alarm.

Why not? Because a dog will get off its backside and kill the rat. And deal with any other intruder. And sense if you are sick or miserable. And keep you warm at night. 

[hamdani yusuf]

Let's hear what an AI researcher say about it.

GPT 4 is Smarter than You Think: Introducing SmartGPT

In this video, I will not only show you how to get smarter results from GPT 4 yourself, I will also showcase SmartGPT, a system which I believe, with evidence, might help beat MMLU state of the art benchmarks.

This should serve as your ultimate guide for boosting the automatic technical performance of GPT 4, without even needing few shot exemplars.

The video will cover papers published in the last 72 hours, like Automatically Discovered Chain of Thought, which beats even 'Let's think Step by Step' and the approach that combines it all.

Yes, the video also touches on the OpenAI DeepLearning Prompt Engineering Course but the highlights come more from my own experiments using the MMLU benchmark, and drawing upon insights from the recent Boosting Theory of Mind, and Let?s Work This Out Step By Step, and combining it with Reflexion and Dialogue Enabled Resolving Agents.

Prompts Frameworks:

Answer: Let's work this out in a step by step way to be sure we have the right answer

You are a researcher tasked with investigating the X response options provided. List the flaws and faulty logic of each answer option. Let's work this out in a step by step way to be sure we have all the errors:

You are a resolver tasked with 1) finding which of the X answer options the researcher thought was best 2) improving that answer, and 3) Printing the improved answer in full. Let's work this out in a step by step way to be sure we have the right answer:

[ArthurArts]

Let's hear what an AI researcher say about it.

GPT 4 is Smarter than You Think: Introducing SmartGPT

In this video, I will not only show you how to get smarter results from GPT 4 yourself, I will also showcase SmartGPT, a system which I believe, with evidence, might help beat MMLU state of the art benchmarks.

This should serve as your ultimate guide for boosting the automatic technical performance of GPT 4, without even needing few shot exemplars.

The video will cover papers published in the last 72 hours, like Automatically Discovered Chain of Thought, which beats even 'Let's think Step by Step' and the approach that combines it all.

Yes, the video also touches on the OpenAI DeepLearning Prompt Engineering Course but the highlights come more from my own experiments using the MMLU benchmark, and drawing upon insights from the recent Boosting Theory of Mind, and Let?s Work This Out Step By Step, and combining it with Reflexion and Dialogue Enabled Resolving Agents.

Prompts Frameworks:

Answer: Let's work this out in a step by step way to be sure we have the right answer

You are a researcher tasked with investigating the X response options provided. List the flaws and faulty logic of each answer option. Let's work this out in a step by step way to be sure we have all the errors:

You are a resolver tasked with 1) finding which of the X answer options the researcher thought was best 2) improving that answer, and 3) Printing the improved answer in full. Let's work this out in a step by step way to be sure we have the right answer:

Honestly, reading about how quickly AI is evolving makes me a little nervous. I'm not a conspiracy theorist, but people seem too invested in this technology and could pay for it.

[alancalverd]

Frankly, if that is an example of GPT4 at work, you have nothing to fear. Yesterday whilst waiting for a shop to open I was accosted by a man who spoke pretty much like that: repetitive, coherent, but meaningless. As he walked away the woman behind me said "Poor old Jeff hasn't been himself since his dad died".

[chris]

In case you did not catch it, we recently published an episode of the Naked Scientists podcast on Artificial Intelligence and chatGPT, which might be helpful.

[remotemass]

I recommend you the book "What Is ChatGPT Doing ... and Why Does It Work?", by Stephen Wolfram.
He has a youtube videos channel and does live streams on a regular basis explaining different topics, mainly in Physics, Complexity, Chaos and Dynamic Systems and Computer Science. He was motivated to start that enterprise of videos and live streams by the very young mathematician Jonathan Gorard. Thank you, both!

It will probably answer you that question and more.

- remotemass

[alancalverd]

Why Does It Work?

If by "work" you mean recycling other people's text, it works because that is what it is designed to do. If you mean creating useful materials, objects or ideas, it doesn't.

[evan_au]

recycling other people's text

Most science is just recycling other people's words.

Perhaps it ChatGPT permutes those same words into a new (but not totally unlikely) sequence, it may, perchance, generate something useful.
- Separating the gold and gems from the stream of derivative drivel is left as an exercise for the reader...

[alancalverd]

Aha, the true Aussie spirit!

"OK, mate, you can have as much outback as you want for a dollar a square meter. There's bound to be a nugget of something somewhere."

Amazingly, there sometimes is, but I wouldn't bet on ChatGPT recognising it!