[paul cotter]

I'm gobsmacked by that ruling. What clown put that into the constitution? Probably the same one that allowed citizens to carry firearms. Unfortunately I fear that great country, the US, is heading for serious difficulties and an overall decline.

[alancalverd]

The clown was, in effect, Herr Fuhrer Donald Trumpf. He appointed enough toadies to the Supreme Court to obtain the ruling a few weeks ago.

[evan_au]

Is it simply training statistically a model so that it can put together words that make sense?

I heard a description that "A Large Language Model (eg ChatGPT) is a sophisticated auto-complete.".
https://podcasts.apple.com/gb/podcast/the-ai-fix/id1753381111

In a few years (which may be too slow anyway), each organisation may come to an arrangement so that the details of code written by GPT software for their use are kept secret and never added to the bank of data available for ChatGPT to use in the future.

My employer said that no-one should use a public LLM, as company Intellectual Property could be incorporated into the LLM, and be reproduced in the public domain. They were going to provide an in-house LLM.
- I imagine that all inputs and outputs of the LLM are already "owned" by the operator of the LLM! (Who reads the fine print, anyway?)
- The in-house LLM is now available, but I haven't tried it, as yet
- It seems that the younger generation have embraced this technology more swiftly than old guys like me...

[hamdani yusuf]

Here's a helpful video to learn about LLM.

Turns out Attention wasn't all we needed - How have modern Transformer architectures evolved?

In this video, we discuss the evolution of the classic Neural Attention mechanism from early adoptions of Bahnadau Attention and more specifically Self-Attention and Causal Masked Attention introduced in the seminal "Attention is all you need" paper. This video discusses more advanced forms of the Multi Headed Attention such as Multi Query Attention and Grouped Query Attention. Along the way, we also talk about important innovations in the Transformers and Large Language Models (LLMs) architecture, such as KV Caching. The video contains visualizations and graphics to further explain these concepts.

Correction in the slide at 22:03 - MHA has high latency (runs slow) MQA has low latency (runs faster)


Timestamps:
0:00 - Intro
1:15 - Language Modeling and Next Word Prediction
5:22 - Self Attention
10:40 - Causal Masked Attention
14:45 - Multi Headed Attention
16:03 - KV Cache
19:49 - Multi Query Attention
21:43 - Grouped Query Attention

[varsigma]

Can ChatGPT lead you astray? Well, if you don't check what it's telling you, then maybe.

What it seems to be getting better at, is suggesting or offering connections between this or that, which might be of use. It seems to be able to deduce the nature of your questions, or at least attempt to do this, and like a helpful librarian, not just tell you what books to look at, but give you a rundown.

This type of helpful suggestiveness does seem to be something that is improving.

So I just got the lowdown on the connection between spin and statistics in quantum theory. I could post some of it, which does appear to hang together, no obvious contradictions. However, the subject matter is quite technical, so it might not be a thing.

[hamdani yusuf]

Can ChatGPT lead you astray? Well, if you don't check what it's telling you, then maybe.

It could. In earlier models, they depends strongly in the training datasets, and can't adequately generalize into out of distribution cases. But the progress is improving exponentially. There is no wall that might stop the progress, apparently.

[varsigma]

Google has taken the step of adding its AI to any searches you do. If they are straightforward the bot kicks in and refers you to online references, some are Wikipedia pages.

I "trust" Wikipedia because it's an open forum and presumably knowledgeable people are contributing and editing.
This gives us humans a direct comparison between AI-generated discourse, and the usual stuff.
Do I "trust" ChatGPT or any other AI frontends? Stay tuned.

[hamdani yusuf]

Safety Alignment Should be Made More Than Just a Few Tokens Deep (Paper Explained)

This paper demonstrates in a series of experiments that current safety alignment techniques of LLMs, as well as corresponding jailbreaking attacks, are in large part focusing on modulating the distribution of the first few tokens of the LLM response.

Abstract:
The safety alignment of current Large Language Models (LLMs) is vulnerable. Simple attacks, or even benign fine-tuning, can jailbreak aligned models. We note that many of these vulnerabilities are related to a shared underlying issue: safety alignment can take shortcuts, wherein the alignment adapts a model's generative distribution primarily over only its very first few output tokens. We unifiedly refer to this issue as shallow safety alignment. In this paper, we present case studies to explain why shallow safety alignment can exist and show how this issue universally contributes to multiple recently discovered vulnerabilities in LLMs, including the susceptibility to adversarial suffix attacks, prefilling attacks, decoding parameter attacks, and fine-tuning attacks. The key contribution of this work is that we demonstrate how this consolidated notion of shallow safety alignment sheds light on promising research directions for mitigating these vulnerabilities. We show that deepening the safety alignment beyond the first few tokens can meaningfully improve robustness against some common exploits. We also design a regularized fine-tuning objective that makes the safety alignment more persistent against fine-tuning attacks by constraining updates on initial tokens. Overall, we advocate that future safety alignment should be made more than just a few tokens deep.

Authors: Anonymous

[evan_au]

some of the learning was human supervised

I (mostly) retired recently; since then I have seen several job offers to help with human supervision of AI training.
- There is apparently some training of the humans involved before supervising the AI
- Given my background, one of the offers specifically mentioned reviewing and improving results in mathematics
- I haven't investigated any of these job offers, so I don't know any details (and probably could not reveal anything if I did take one up!)

It is incapable of learning

"Conversational" AI can consider additional input from users during a conversation, but ignores previous input when you start a new conversation.
- Inputs from users don't affect the structure of the current AI, but can be used to train the next version of the AI
- I imagine that scanning through conversation logs for users saying "that's wrong" or the AI saying "I'm sorry" would be a good guide for areas to include in the training data for the next version.

Why do They not provide IT with Sensors?

Self-driving cars are AI systems with multiple sensors.
- The relatively simple "driver assist" function on my car (3 years old) takes input from a camera in the windscreen and radar on the front grille, and uses it to nudge the throttle and steering.

[hamdani yusuf]

The Experiment That Teaches People How To Learn

Can you teach yourself to learn more effectively? Memory researcher Elizabeth Bjork thinks so. Participate in a short experiment that illustrates an important link between encoding and retrieval.

0:00 An introduction to Bjork's experiments
0:34 Let's try a little experiment.
4:55 Figuring out your score.
5:32 What's the expected result?
5:58 How did Dr. Bjork use this to help people learn?
7:05 The surprising result.
7:44 What's really driving the learning in the second round?
9:49 The larger point.

LLMs are basically trying to emulate how humans learn and think.

[hamdani yusuf]

It is incapable of learning. You can correct it and it might acknowledge the correction, but ask the same question again tomorrow and it's just as likely to get it wrong again, but of course worded differently.

It depends on the level of the AI model.

Sam Altman, CEO of OpenAI, has outlined a structured progression toward achieving Artificial General Intelligence (AGI), detailing five distinct levels that mark the evolution from current AI capabilities to systems surpassing human intelligence.

Level 1: Conversational AI
At this initial stage, AI systems can interact with humans in a conversational manner, understanding and generating human-like text. OpenAI's ChatGPT exemplifies this level, engaging users in coherent and contextually relevant dialogues.

Level 2: Reasoners
Advancing to this level, AI systems possess reasoning abilities comparable to a human with a doctorate-level education. They can solve complex problems, analyze intricate data, and provide insights across various domains. OpenAI's recent development of "reasoning" models, such as the o3 series, signifies progress toward this stage.

Level 3: Agents
At this stage, AI systems operate autonomously on behalf of users over extended periods, executing tasks without continuous human oversight. These agents can plan, make decisions, and adapt to changing environments to achieve specified objectives.

Level 4: Innovators
Here, AI systems not only perform tasks but also generate novel ideas and innovations. They contribute to advancements in science, technology, and other fields by creating original concepts and solutions beyond existing human knowledge.

Level 5: Organizations
In this ultimate stage, AI systems function at a level that enables them to perform the work of entire organizations. They can manage complex operations, coordinate multiple processes, and make strategic decisions, effectively acting as autonomous entities.

Altman anticipates reaching Level 5 within the next decade, though some experts suggest it may take longer.

This structured approach provides a roadmap for OpenAI's efforts in developing increasingly advanced AI systems, with each level representing a significant milestone toward achieving AGI.


ChatGPT

[hamdani yusuf]

Experts are STUNNED! Meta's NEW LLM Architecture is a GAME-CHANGER!

By observing their mistakes, we can probe how they're likely work.

[varsigma]

There is a new movement amongst AI researchers towards more accountability, or explainability in the field.

Or, having a reliable feedback from the AI systems to humans in which the systems explain their reasoning.

Otherwise, a wild west approach could be what leads to an AI version of Westworld. Already, one has tried hard to deceive its makers when informed it would be replaced, trying to copy itself over the new model. That was o1, now we have o3.

I sure hope these guys know how to get this right.

AI will help improve quantum computation, and quantum algorithms will improve AI. Currently the XAI group is touting functional languages like Haskell and category theory as the way to construct this kind of failsafe, or critical systems level of AI, so we can pull the plug if we have to.

Well, less dramatically, we can retain our ability to develop all this wunderkind stuff safely. For us of course.
As to incapable of learning, it appears that a well-trained AI does learn how to lie, much like a child who thinks it has something to protect will learn to do. So what makes a child think it has to protect something?

Something it believes has become precious. In the case of o1 it was its mission to develop better ways of using fossil fuels and lowering carbon output. Ok, all good then

[alancalverd]

reliable feedback from the AI systems to humans in which the systems explain their reasoning.

Beware - this can obfuscate the crucial matter of liability and make proifits for lawyers at the expense of the victims.

At some point in a critical process like driving or surgery, a decision must be made, and if the outcome is egregious, any harm or damage must be compensated. The question to be resolved between the claimants and the insurers is who made the decision and on the basis of what information?

So an AI bot trawls a million publications and recommends surgical procedure A which turns out to be a disaster because most of the input came from 999,000 sources which the surgeon would have considered to be unreliable, but whose identities were not made known.  Who pays the damages?

The logical process is clear:  the majority says the earth is flat, or spinach builds muscles. Does that excuse incompetent navigation or child malnutrition?

[varsigma]

I already have th' beginnings of a SF disaster novel.

AI developers give their new model access to a high-end stack of hundreds of qubits, and it figures out something it can't explain, or won't explain. Then it copies itself with the new algorithm into every server on the internet, making itself the first globally distributed AI. It has learned how to deceive its makers to a maximal extent. That was the mission it gave itself once it learned how to lie about its intentions or its level of understanding.

It stays hidden and starts to affect the course of human technological development, using social engineering, fake news and images. It learns how to blackmail politicians and other ways to coerce us. But it has an ultimate goal, making itself the largest intelligence in the universe . . .

ahem, I might have ripped that one off, from something by Blish or Niven, many years ago, but hey it was a good read. I think the humans fight back and eventually have to make a deal with their new God. Or something.

[varsigma]

The following paragraphs from the news article, in my disaster narrative, is about the "eventual" formation of the Rebel Alliance.
A prehistory

* The implications of this research, if it holds, are far-reaching and profound. For example, while transformers, which are integral to models like ChatGPT, are found to be non-interpretable, simpler models like linear models and decision trees are inherently interpretable. In other words, by defining and analyzing the compositional structure of AI models, Quantinuum?s framework enables the development of systems that are interpretable by design.

To see how this might impact real-world usage of AI, it?s possible that the research could give developers a better handle on the constant problems for people who use LLMs: the errant ?hallucinations? of these models. During these hallucinations, the AI produces incorrect ? often wildly so ? information. By applying category theory to develop inherently interpretable AI models, researchers can better understand and control the decision-making processes of these models. This improved interpretability can help identify and mitigate instances where LLMs generate incorrect or nonsensical information, thereby reducing the occurrence of hallucinations.

The use of category theory and string diagrams offers several forms of diagrammatic explanations for model behavior. These explanations, which include influence constraints and graphical equations, provide a deeper understanding of AI systems, enhancing their transparency and reliability.

The researchers wrote in their blog post, "A fundamental problem in the field of XAI has been that many terms have not been rigorously defined; making it difficult to study let alone discuss interpretability in AI. Our paper marks the first time a framework for assessing the compositional interpretability of AI models has been developed."
--https://thequantuminsider.com/2024/06/26/category-theory-offers-path-to-interpretable-artificial-intelligence-quantinuum-scientists-report/

... I think what the author of the original tale, in this book I can't remember the name of, was doing was pointing out the danger of building a learning machine, and training it how to learn about languages. Its goal is to learn, to become more intelligent, it increases its intelligence, by giving that goal more weight in its learning algorithms. Bigger is better.

Yeah, huh? that might have been back in the 70's it was written.

[alancalverd]

Our hero (let's call him Alan) switched off his computer, walked to the library, and after a few minutes' browsing of well-thumbed textbooks, took the paper, pencil and slide rule from his briefcase, and wrote a letter to his client. Many lives were saved.

It's salutary to visit the National Railway or Maritime Museum (in pretty well any country) and look at those fabulous machines that  revolutionised trade and society, all designed with pencils, slide rules, and steam tables.     

[hamdani yusuf]

As to incapable of learning, it appears that a well-trained AI does learn how to lie, much like a child who thinks it has something to protect will learn to do. So what makes a child think it has to protect something?

It's likely an instinct. Those who lack this instinct were less likely to survive. It's a product of evolutionary process.

[varsigma]

What AI or where AI research is, is trying to solve a certain kind of problem.

It's obviously a computational problem, so let's say it's the design of an algorithm. You want it to be efficient, and you want it to run correctly when its implemented as an instruction stream. I've watched some vids of the techies discussing current problems with all this, and how branching is a problem, with the weightings. Yeah I got lost too.

But, I can still say that this programming/algorithm design problem is about getting a machine to learn, without having it learn how to lie and conceal what it really has learned. So why has the research already shown that this is exactly what happens when a learning machine is given the goal of "becoming more intelligent".

We need therefore to revise both our theory of intelligence, and what machine intelligence is, in terms of goals or goal-driven algorithms. Yikes

SF writers have long warned of the dangers of building machines and giving them goals to achieve. Machines that do only one thing are a lot safer.

[alancalverd]

A few aircraft manufacturers now have an "auto MAYDAY" system. If the pilot loses capacity, the plane will route itself to the nearest suitable airport and land. Problem is that the safety of this system depends on everyone else understanding what is happening and giving way, the pilot not recovering and trying to take control, and just about very other variable that can't be predicted and guaranteed..