0 Members and 96 Guests are viewing this topic.
Is OpenAI's project code-named Strawberry (formerly Q-Star) a combination of two techniques already revealed in papers? These papers on Deep Q learning and STaR bootstrapping could show the way to AGI, so perhaps they are!
In today's AI news and tech news, we'll discuss the revolutionary steps towards GPT 5 and AGI taken by Sam Altman & Co. with their upcoming "Reasoners" category, which will include GPT 5, specifically focusing on the advanced AI agents and GPT 5 developments. Sam Altman and his partners are ramping up their efforts, targeting a system that emulates PhD-level capabilities without auxiliary tools. GPT 5, as revealed, is on the brink of becoming a powerhouse in AI, potentially functioning with the intellectual prowess equivalent to those holding top-tier academic degrees.Recent reports from Bloomberg and insights shared by OpenAI's executives during internal presentations shine a light on the ambitious path from conversational AI agents, like ChatGPT 4 and ChatGPT 4o, to the next generation that could manage entire organizations. With this leap, OpenAI introduces a structured tier system named after various academic levels, signifying a step closer to achieving artificial general intelligence (AGI). This tier, where GPT 5 belongs, known as "Reasoners," indicates a significant shift towards more autonomous and sophisticated AI systems capable of performing complex and critical tasks, setting a new benchmark in the field.In conversations with leading figures such as Sam Altman and Dario Amodei from Anthropic, the potential of GPT 5 extends beyond current applications, with it slated to impact high-stake industries and research organizations profoundly. Despite skepticism about AI hype, the trajectory set by these advancements suggests a near future where AI could outperform human capabilities in various domains.
An article from Reuters has new information about Q-star and project Strawberry. Let's take a look!
Errata1:40 should be: "word fragment is appended to the end of the original input". Thanks for Chris A for finding this one.
https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdfImageNet Classification with Deep Convolutional Neural NetworksAlex Krizhevsky, University of Toronto, kriz@cs.utoronto.caIlya Sutskever, University of Toronto, ilya@cs.utoronto.caGeoffrey E. Hinton, University of Toronto, hinton@cs.utoronto.caAbstractWe trained a large, deep convolutional neural network to classify the 1.2 millionhigh-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5%and 17.0% which is considerably better than the previous state-of-the-art. Theneural network, which has 60 million parameters and 650,000 neurons, consistsof five convolutional layers, some of which are followed by max-pooling layers,and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called ?dropout?that proved to be very effective. We also entered a variant of this model in theILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,compared to 26.2% achieved by the second-best entry.
Sora was previously known as the best text to video generator... Then 4 months later, its contenders have shown their faces. This cambrian explosion of AI generated videos is actually getting out of hand.
0:00 - intro1:06 - google3:09 - the dead internet theory3:51 - facebook6:33 - twitter9:56 - instagram12:09 - character.ai13:15 - sofi14:17 - adobe16:54 - netflix17:52 - spotify/AI music24:24 - AI as a side hustle26:12 - AI video is terrifying29:41 - what is the point of this?
0:00 - The Mind Virus1:52 - OpenAI3:27 - Why are companies investing in AI?5:29 - AI is not profitable (yet)6:31 - Hype is a Marketing Tool10:21 - How fast will AI improve?14:12 - How to make life decisions?A few more points I didn't mention in the video:1. A day after i uploaded this video I saw tech bros on twitter saying "all you need is Claude" and you can code almost anything.. yet they couldn't even recreate a basic component on neetcode.io that I literally coded as a junior engineer. So once again, people are vastly overstating what AI can do. If only this hadn't happened in human history a million times before.2. Amazon invested billions in Alexa, only for it to be obsoleted by LLMs. I worked in Alexa (for a brief time) and it's obvious it wasn't well run. Big tech doesn't always know what they're doing.3. Amazon Go's "AI" turned out to be indian workers watching security cameras.4. Nearly every advancement goes through hype cycles. Not just the dotcom bubble, even the railroads were overbuilt in the 1800s. "The Panic of 1893 was the largest economic depression in U.S. history at that time. It was the result of railroad overbuilding and shaky railroad financing, which set off a series of bank failures."Fwiw I literally use LLMs on a daily basis to automate my own tasks. Yes it helps somewhat, but I'm also very familiar with its limitations.
In the near future there should be a reliable method to determine whether a video represent physical reality or not.
Quote from: hamdani yusuf on 30/07/2024 04:36:52In the near future there should be a reliable method to determine whether a video represent physical reality or not.Ask the basic question: who stands to gain?
I follow the history of RL (model free), from learning tic tac toe, checkers, back gammon, as well as physical problems (cart and pole), walking, grasping (OpenAI's dexterous robotic hand)...I explain value functions, q functions, policy functions and how they work together. Including how TD learning was used..Along the way, we'll encounter the challenges of transferring simulated skills to the real world (domain randomization) and witness the emergence of eerily human-like behaviors in AI agents. It leaves us with a provocative question: where is the line between actions and words? What is the role of an GPT for actions?Featuring insights from:Claude ShannonArthur SamuelGerald TesauroRichard SuttonDavid SilverDeep Mind/Open AI etc.00:00 - Introduction00:32 - Learning Tic Tac Toe02:00 - Learning Cart and pole04:20 - Shannon & Chess06:50 - Samuel's Checkers09:25 - TD Gammon (Gerald Tesaruo)11:00 - TD Learning14:30 - Learning Atari (DQN)17:28 - DIrect Policy Gradiant19:40 - Domain Randomization
Let's take a look at 15 big events that happened in Tech in July 2024, like Google's Alpha Proof math AI, the CrowdStrike Windows disaster, Node.js TypeScript support, new Python web frameworks, and more. 🔖 Topics CoveredTuaw Zombie WebsiteNode TS SupportFastHTML Framework Zed LinuxIntel Chips Fixed KindaStripe Buys Lemon SqueezyStackOverflow UnhappinessGoogle Alpha ProofGoogle Alpha RatsSearchGPTMistral LargeOpenAI BankruptcyReddit Pay-to-ScrapeThe COPIED ActCrowdstrike Redemption Arc
NVIDIA CEO Jensen Huang presented a major breakthrough on Project GR00T with WIRED?s Lauren Goode at SIGGRAPH 2024.In a two-minute demonstration video, NVIDIA explained a systematic approach they discovered to scale up robot data, addressing one of the most challenging issues in robotics.The research team began by using Apple Vision Pro to give the human operator first-person control of the humanoid. By leveraging RoboCasa, a generative simulation framework, the researchers multiplied the demonstration data by varying the visual appearance and layout of the environment. In the final step, they applied MimicGen to further multiply the data by varying the robot's motion.Through GPU-accelerated simulation technology, NVIDIA successfully transformed scarce and expensive human demonstration data into massive training samples. Seems like we can finally apply the "scaling law" in the field of robotics! Woohoo!
Future conscious entities.
Quote from: hamdani yusuf on 01/08/2024 07:26:45Future conscious entities.No. The person who made, commissioned or published the video.
a reliable method to determine whether a video represent physical reality or not.
Mixture of Experts explained, well, re-explained. We are in the Fine-Grain era of Mixture of Experts and it's about to get even more interesting as we further scale it up.
Are We Done With MMLU?[Paper] https://arxiv.org/abs/2406.04127Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models[Paper] https://arxiv.org/abs/2406.02061Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization[Paper] https://arxiv.org/abs/2405.15071Grokfast: Accelerated Grokking by Amplifying Slow Gradients[Paper] https://arxiv.org/abs/2405.20233Code: [Select] https://github.com/ironjr/grokfast[/quote]
https://github.com/ironjr/grokfast[/quote]
In this video we explore Boltzmann Machines ? one of the first generative models that learns probability distribution of data, leveraging stochastic rules and latent representations.OUTLINE:00:00 Introduction01:56 Goal of Boltzmann Machines05:26 Boltzmann Distribution13:29 Stochastic Update Rule17:39 Contrastive Hebbian Rule25:41 Hidden Units28:25 Restricted Boltzmann Machines29:38 Conclusion & OutroReferences:1. Ackley, D., Hinton, G. & Sejnowski, T. A learning algorithm for boltzmann machines. Cognitive Science 9, 147?169 (1985).2. Downing, K. L. Gradient Expectations: Structure, Origins, and Synthesis of Predictive Neural Networks. (The MIT Press, Cambridge, Massachusetts, 2023).3. Hinton, G. E. & Salakhutdinov, R. R. Reducing the Dimensionality of Data with Neural Networks. Science 313, 504?507 (2006).4. Hinton, G. E. A Practical Guide to Training Restricted Boltzmann Machines. in Neural Networks: Tricks of the Trade (eds. Montavon, G., Orr, G. B. & M?ller, K.-R.) vol. 7700 599?619 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2012).
Books mentioned- WEAPONS OF MATH DESTRUCTION- AUTOMATING INEQUALITYBTW I used to take this issue very seriously.
Pausing is just a waste of time. Worse, it's a waste of life and lives potentially saved. We have to adapt in the moment, and discover the dangers and opportunities along the way. Though, as a guy stuck in a low-paying 9-5 job for potentially the rest of my life, my opinion is colored by extreme desperation. My generation, and every generation after, will never be able to stop working. No retirement. Ever. The economics just aren't there (thank you Ronald Regan), and no company is paying a living wage. There's no pressure to. Unless the tech revolution comes along and disrupts the whole "work until you die" thing, I say full steam ahead. The sooner we get to that AGI tech revolution, or whatever it may look like, then the sooner things can change drastically. I'm hoping for drastic positive change, but the uncertainty of a chaotic future is better than knowing what lies at the end of the road with the current status quo. Though, I know that pausing AI progress is an impossibility, which gives me a lot of hope.
SAKANA AUTONOMOUS AI SCIENTIST- https://github.com/SakanaAI/AI-Scientist- https://sakana.ai/ai-scientist/ACE FRAMEWORK- https://arxiv.org/abs/2310.06775- https://github.com/daveshap/ACE_Frame...