[hamdani yusuf (OP)]

New Caledonia Open Data as a Whole Big Graph

The New-Caledonian government manages an Open Data platform that hosts and promotes all the Open Data of New-Caledonia on data.gouv.nc. As the underlying solution relies on OpenDataSoft, it makes interacting with all the data possible. In this session, Adrien will delve into all these metadata datasets to explore his country from a purely data-centric and graph perspective at a substantial scale.


00:00 : Intro about New-Caledonia
01:08 : How do countries are connected to the world (and United Nations)
01:27 : Benchmarking countries Open Data maturity w/ the "Global Data Barometer" tool
02:17 : Introducing our data journey
02:52 : Ideating w/ AI : create a storytelling plan with AI crowd (Langchain SmartLLMChain & OpenAI gpt-4)
05:02 : Discovering storytelling keypoint 1/5 : Map Country's Open Data to UN SDGs
05:35 : 2/5 Identifying relationships
06:07 : 3/5 predictive analysis
06:16 : 4/5 Gap Analysis
06:30 : 5/5 Storytelling with Data
06:52 : Get smart title and subtitle for the experience
07:18 : About UN SDGs
07:36 : The data pillars we used to connect country data to UN SDGs
07:59 : About data.gouv.nc (Open Data portal from New-Caledonia)
08:22 : UN SDGs Open Data Portal
08:36 : About Pacific Data Hub from The Pacific Community (SPC)
08:58 : Project workflow, schedule and stack
09:52 : Connecting datasets to UN SDGs
10:18 : Building and putting the data together on Kaggle/DuckDb
10:30 : Discovering the schema of our knowledge graph
11:26 : About SDG goal hierarchy and "foundational goals"
11:51 : Introducing RAG (Retrieval AUgmented Generation) w/ LangChain
15:19 : RAG with LLama_Index
17:29 : SDGs interactive discovery : storytelling on Neo4J Aura DB/Bloom
20:27 : Explore Gender Equity in 3D with Kineviz GraphXR
22:22 : Dataviz with Gephisto
22:38 : Kudos to the biggest Open Data Publishers
22:59 : About the stack
23:39 : Data art and contemplation (Gephi with Runway experiment) cf   
25:49 : Conclusion

The more accurate and precise model of the world is needed to make better informed decisions.

[alancalverd]

The more accurate and precise model of the world is needed to make better informed decisions.

True. What you are saying is that the more information we have, the better informed we are, which is just a tautology.

But what matters is the ability to make better decisions.

It is characteristic of autocratic governments that they collect and index vast amounts of information about everyone and everything.  It is also characteristic that they make very bad decisions. Here's an old story about a meeting between Kruschev and Eisenhower:

K: "You accuse the Soviet Union of being antisemitic. 30% of our musicians, 25% of our scientists, 19% of our journalists and 10% of our military officers and senior civil servants are Jews. Can you say the same about the USA?"

E: "Our constitution does not allow the President to count Jews."   

[hamdani yusuf (OP)]

It is characteristic of autocratic governments that they collect and index vast amounts of information about everyone and everything.  It is also characteristic that they make very bad decisions.

Democratic governments collect data as well. The difference is the transparency in which and how the data is collected and how it's used.
Bad decisions mean they don't support of achieving their goals. More accurate and precise models of reality allow for better alignment between goals and decisions.

[hamdani yusuf (OP)]

Are we heading for a digital prison? - Panopticon (Foucault, Bentham, Cave)

Today we talk about Jeremy Bentham's concept of the Panopticon. Michel Foucault's comparison to society in 1975. The historical role of intelligence as a justification for dominance. The anatomy of free will, and how a digital world may systematically limit our free will without us knowing it.

[alancalverd]

More accurate and precise models of reality allow for better alignment between goals and decisions.

But they don't legitimise the goal. Knowing exactly where all the Uyghurs or Sunnis live makes it a lot easier to exterminate them, but it doesn't make extermination a Good Thing.

[hamdani yusuf (OP)]

More accurate and precise models of reality allow for better alignment between goals and decisions.

But they don't legitimise the goal. Knowing exactly where all the Uyghurs or Sunnis live makes it a lot easier to exterminate them, but it doesn't make extermination a Good Thing.

It depends on how far in the future you determine your goal. Genocides can only be good for short term, one generation at most. Longer than that, the undesired side effects will outweigh the short term benefits.

[alancalverd]

Good for whom in the short term? Only the politicians that order them, surely?

[hamdani yusuf (OP)]

Physics Informed Machine Learning: High Level Overview of AI and ML in Science and Engineering

This video describes how to incorporate physics into the machine learning process.  The process of machine learning is broken down into five stages: (1) formulating a problem to model, (2) collecting and curating training data to inform the model, (3) choosing an architecture with which to represent the model, (4) designing a loss function to assess the performance of the model, and (5) selecting and implementing an optimization algorithm to train the model. At each stage, we discuss how prior physical knowledge may be embedding into the process.
Physics informed machine learning is critical for many engineering applications, since many engineering systems are governed by physics and involve safety critical components.  It also makes it possible to learn more from sparse and noisy data sets.

%%% CHAPTERS %%%
00:00 Intro
03:53 What is Physics Informed Machine Learning?
06:41 Case Study: Encoding Pendulum Movement
09:19 The Five Stages of Machine Learning
16:09 A Principled Approach to Machine Learning
20:00 Physics Informed Problem Modeling
21:48 Physics Informed Data Curation
25:34 Physics Informed Architecture Design
28:59 Physics Informed Loss Functions
30:55 Physics Informed Optimization Algorithms
34:56 What This Course Will Cover
46:48 Outro

[alancalverd]

many engineering systems are governed by physics

We used to think that everything was governed by physics, but clearly the proponents of whatever they are selling think some are governed by consensus.  Kruger-Dunning strikes again.

[hamdani yusuf (OP)]

Good for whom in the short term? Only the politicians that order them, surely?

Some of their followers might also benefit.

[hamdani yusuf (OP)]

many engineering systems are governed by physics

We used to think that everything was governed by physics, but clearly the proponents of whatever they are selling think some are governed by consensus.  Kruger-Dunning strikes again.

When the consensus don't match with reality, sooner or later they will face reality check, and be forced to shift their paradigm.

[hamdani yusuf (OP)]

Elon Musk's STUNNING Prediction | Sam Altman Attempts to Harness the Power of a Thousand Suns

Wes Roth's video delves into the rapidly evolving landscape of Artificial General Intelligence (AGI), highlighting the potential for digital intelligence to surpass human cognitive capabilities within the next decade. It discusses the challenges and advancements in computing and energy resources necessary for AI development, focusing on the energy demands of training large AI models and the exploration of fusion energy as a sustainable solution. The video also touches on the impact of AI and technology on creative industries, including film, gaming, and art, emphasizing the transformative potential of AI tools for independent creators. Additionally, it explores the societal and economic implications of AI advancements, proposing equitable wealth distribution models to mitigate the risks of mass unemployment. The video concludes by addressing concerns and optimism surrounding AI's role in creative endeavors, emphasizing the importance of adapting to and embracing technological change.

Video Chapters:

[00:00:00] Introduction to AGI Predictions
Discussion on the difficulty of predicting the future due to rapid changes.
Predictions about AGI surpassing human cognitive tasks within a few years.

[00:01:36] The Energy Challenge in AI Development
The transition from computational to energy limitations in AI advancements.
Insights into the difficulties of scaling AI infrastructure without impacting energy resources.
[00:02:39] Fusion Energy: A Potential Solution
Introduction to Helion Energy's fusion project, supported by notable tech figures. Fusion vs. fission energy, with a focus on fusion's potential to provide sustainable energy for AI development.

[00:06:08] Criticism and Optimism Towards Tech Leaders
Discussion on the negative media portrayal of tech figures and the rationale behind it.

The importance of focusing on the positive impacts of technological advancements rather than the controversies.

[alancalverd]

When the consensus don't match with reality, sooner or later they will face reality check, and be forced to shift their paradigm.

At the cost of how many lives? Engineering isn't a game.

[hamdani yusuf (OP)]

When the consensus don't match with reality, sooner or later they will face reality check, and be forced to shift their paradigm.

At the cost of how many lives? Engineering isn't a game.

Engineering is a part of life. Life is a game for everyone. The prize is the survival of one's successors.
If you think about it, even our current selves are successors of our own pasts. Likewise, our future selves are successors of our current selves. Our bodies consist of different atoms than they were decades ago. They will also be different next decades, whether or not we still alive.

[hamdani yusuf (OP)]

Grok Vision - First Multimodal Model from XAi

X.ai just made an announcement about Grok-1.5 Vision. Its their new multimodal model that can understand images and can write code based on flow diagrams just like GPT-4

Grok 1.5 Vision Shows STUNNING Performance | Beats GPT-4, Claude and Gemini 1.5

GROK:
https://x.ai/blog/grok-1.5v
Introducing Grok-1.5V, our first-generation multimodal model. In addition to its strong text capabilities, Grok can now process a wide variety of visual information, including documents, diagrams, charts, screenshots, and photographs. Grok-1.5V will be available soon to our early testers and existing Grok users.

Capabilities
Grok-1.5V is competitive with existing frontier multimodal models in a number of domains, ranging from multi-disciplinary reasoning to understanding documents, science diagrams, charts, screenshots, and photographs. We are particularly excited about Grok?s capabilities in understanding our physical world. Grok outperforms its peers in our new RealWorldQA benchmark that measures real-world spatial understanding. For all datasets below, we evaluate Grok in a zero-shot setting without chain-of-thought prompting.

The universe is a dynamic system, thus an accurate virtual universe must also be dynamic, i.e. change with time to reflect the real universe. The accurate and dynamic virtual universe must have the ability to understand information they get from their sensors and other inputs. RealWorldQA benchmark is a way forward.

[alancalverd]

The universe is a dynamic system, thus an accurate virtual universe must also be dynamic, i.e. change with time to reflect the real universe.

Including itself.
Fact is that any mapping is necessarily incomplete. A database can be accurate or up to date, but not both.

[hamdani yusuf (OP)]

The universe is a dynamic system, thus an accurate virtual universe must also be dynamic, i.e. change with time to reflect the real universe.

Including itself.
Fact is that any mapping is necessarily incomplete. A database can be accurate or up to date, but not both.

It's a false dichotomy. It can be sufficiently accurate, precise, and updated in certain practical limits. The remaining inaccuracies can be assigned to random factors with computable probabilitas.

[alancalverd]

"Sufficiently" is what it's all about. See Heisenberg!

[hamdani yusuf (OP)]

"Sufficiently" is what it's all about. See Heisenberg!

The universe could be infinite, both in space and time. Simulating it with infinite precision and accuracy would need infinite computational resources, which is not practical.

[alancalverd]

Even if it were finite, you can't simulate it completely because the simulation would then be part of the universe, leading to an infinite recursion.