[hamdani yusuf (OP)]

Economic rewards as incentive should be given to those who benefits the society, rather than those who put additional burdens.

[hamdani yusuf (OP)]

Complex concepts like consciousness can be understood by studying its emergence from simpler systems.

[hamdani yusuf (OP)]

First, select the most important thing/preferred condition that you want to achieve, which you will defend at all cost.

When you take this step, you should already understand the difference between instrumental and terminal goals.

[hamdani yusuf (OP)]

I found an interesting article on governance. I'd like to share my thoughts on some points in it.

https://www.lesswrong.com/posts/SCs4KpcShb23hcTni/ideal-governance-for-companies-countries-and-more
I'm interested in the topic of ideal governance: what kind of governance system should you set up, if you're starting from scratch and can do it however you want?

Here "you" could be a company, a nonprofit, an informal association, or a country. And "governance system" means a Constitution, charter, and/or bylaws answering questions like: "Who has the authority to make decisions (Congress, board of directors, etc.), and how are they selected, and what rules do they have to follow, and what's the process for changing those rules?"

Some design choices for governance:

Who votes, how often, and what voting system is used?
How many representatives are there in each representative body? How are they divided up (one representative per geographic area, or party-list proportional representation, or something else)?
What term limits exist for the different entities?
Do particular kinds of decisions require supermajorities?
Which restrictions are enshrined in a hard-to-change Constitution (and how hard is it to change), vs. being left to the people in power at the moment?

One way of thinking about the "ideal governance" question is: what kinds of designs could exist that aren't common today? And how should a new organization/country/etc. think about what design is going to be best for its purposes, beyond "doing what's usually done"?

Determining the best option requires defining the terminal goal, and choose the option which has highest effectiveness and efficiency in achieving the goal.

[hamdani yusuf (OP)]

Here are some ideas in the article :

For any new institution, it seems like the stakes are potentially high - in some important sense, picking a governance system is a "one-time thing" (any further changes have to be made using the rules of the existing system1).

Perhaps because of this, there doesn't seem to be much use of innovative governance designs in high-stakes settings. For example, here are a number of ideas I've seen floating around that seem cool and interesting, and ought to be considered if someone could set up a governance system however they wanted:

Sortition, or choosing people randomly to have certain powers and responsibilities. An extreme version could be: "Instead of everyone voting for President, randomly select 1000 Americans; give them several months to consider their choice, perhaps paid so they can do so full-time; then have them vote."
The idea is to pick a subset of people who are both (a) representative of the larger population (hence the randomness); (b) will have a stronger case for putting serious time and thought into their decisions (hence the small number).
It's solving a similar problem that "representative democracy" (voters elect representatives) is trying to solve, but in a different way.
Proportional decision-making. Currently, if Congress is deciding how to spend $1 trillion, a coalition controlling 51% of the votes can control all $1 trillion, whereas a coalition controlling 49% of the votes controls $0. Proportional decision-making could be implemented as "Each representative controls an equal proportion of the spending," so a coalition with 20% of the votes controls 20% of the budget. It's less clear how to apply this idea to other sorts of bills (e.g., illegalizing an activity rather than spending money), but there are plenty of possibilities.2
Quadratic voting, in which people vote on multiple things at once; and can cast more votes for things they care about more (with a "quadratic pricing rule" intended to make the number of votes an "honest signal" of how much someone cares).
Reset/Jubilee: maybe it would be good for some organizations to periodically redo their governance mostly from scratch, subject only to the most basic principles. Constitutions could contain a provision like "Every N years, there shall be a new Constitution selected. The 10 candidate Constitutions with the most signatures shall be presented on a ballot; the Constitution receiving the most votes is the new Constitution, except that it may not contradict or nullify this provision. This provision can be prevented from occurring by [supermajority provision], and removed entirely by [stronger supermajority]."

If we were starting a country or company from scratch, which of the above ideas should we integrate with more traditional structures, and how, and what else should we have in our toolbox? That's the question of ideal governance.

[hamdani yusuf (OP)]

An interesting foot note.

Let's say that 70% of the Parliament members vote for bill X, and 30% vote against. "Proportional chance voting" literally uses a weighted lottery to pass bill X with 70% probability, and reject it with 30% probability (you can think of this like rolling a 10-sided die, and passing the bill if it's 7 or under).

A key part of this is that the members are supposed to negotiate before voting and holding the lottery. For example, maybe 10 of the 30 members who are against bill X offer to switch to supporting it if some change is made. The nice property here is that rather than having a "tyranny of the majority" where the minority has no bargaining power, we have a situation where the 70-member coalition would still love to make a deal with folks in the minority, to further increase the probability that they get their way.

Quote from the paper that I am interpreting: "Under proportional chances voting, each delegate receives a single vote on each motion. Before they vote, there is a period during which delegates may negotiate: this could include trading votes on one motion for votes on another, introducing novel options for consideration within a given motion, or forming deals with others to vote for a compromise option that both consider to be acceptable. The delegates then cast their ballots for one particular option in each motion, just as they might in a plurality voting system. But rather than determining the winning option to be the one with the most votes, each option is given a chance of winning proportional to its share of the votes."

[hamdani yusuf (OP)]

Elon Musk: A future worth getting excited about | TED | Tesla Gigafactory interview

What's on Elon Musk's mind? In conversation with head of TED Chris Anderson, Musk details how the radical new innovations he's working on -- Tesla's intelligent humanoid robot Optimus, SpaceX's otherworldly Starship and Neuralink's brain-machine interfaces, among others -- could help maximize the lifespan of humanity and create a world where goods and services are abundant and accessible for all. It's a compelling vision of a future worth being excited about. (Recorded at the Tesla Texas Gigafactory on April 6, 2022)

From a comment:
Interview timeline

*The future 00:30
*Avoiding climate catastrophy 01:20
*Batteries 🔋 04:15
*The future is bright 07:30
*Self driving  cars 🚗 09:00
*Predicted timeline of progress 18:00
*Optimus robot 🤖 20:30
*AI safety 🦺 27:00
*AI-Human symbiosis & brain 🧠 computer 🖥️ interface 29:30
*Starship 37:30
*Going to Mars 44:30
*Transportation 54:04
*Company for the future of humanity 55:40
*Elon's wealth 59:00
*Elon's drive 01:04:10

His answer to the last question at 1:06 is the most valuable lesson in this interview.
"If you want the future to be good, you must make it so; take action to make it good, and it will be. "

[alancalverd]

"Proportional chance voting" literally uses a weighted lottery to pass bill X with 70% probability, and reject it with 30% probability

So do we go to war / have a national heath service / abolish the death sentence. / give Hamdani the Nobel Prize...or not?

[hamdani yusuf (OP)]

"Proportional chance voting" literally uses a weighted lottery to pass bill X with 70% probability, and reject it with 30% probability

So do we go to war / have a national heath service / abolish the death sentence. / give Hamdani the Nobel Prize...or not?

In the proposed system, there will be some steps to get the decision.

A key part of this is that the members are supposed to negotiate before voting and holding the lottery. For example, maybe 10 of the 30 members who are against bill X offer to switch to supporting it if some change is made. The nice property here is that rather than having a "tyranny of the majority" where the minority has no bargaining power, we have a situation where the 70-member coalition would still love to make a deal with folks in the minority, to further increase the probability that they get their way.

If the negotiation fails, then conduct and count the votes. Then randomize a variable with output probability depends on the voting results. Make the decision based on the random output.

[hamdani yusuf (OP)]

If we can stay alive tomorrow, should we?
If ask the same question again tomorrow, will we give the same answer?
When will we start to consider the different answer?

[hamdani yusuf (OP)]

“It doesn’t sound that great being a dying mammal to me.”

That’s Tim Urban, founder of the popular blog Wait But Why, making his pitch for the development of robotic and technical enhancements to the human body — a school of thought known as transhumanism.

For Urban, the assumption that “natural” is inherently good is one that needs reconsidering. And while the ideas of transhumanism may sound like science fiction, some of them are much closer to reality than you might think — with cyborgs already walking among us.

[hamdani yusuf (OP)]

We Were Right! Real Inner Misalignment
Researchers ran real versions of the thought experiments in the 'Mesa-Optimisers' videos!
What they found won't shock you (if you've been paying attention)

It shows the important of identifying the terminal goal before we give immense power to any conscious agents, which include supercomputer AI, politicians, and billionaires.

[hamdani yusuf (OP)]

I bring this here from my other thread, since it would be more relevant to discuss here.

The object of philosophy is to tell people that they don't (or even can't) understand the obvious. I think HY is an undercover philosopher.

It looks like you need to read the introduction to philosophy. Here's a book recommendation I got in my twitter feed.
T H E GR E AT
CONVERSATION
A Historical Introduction to Philosophy
EIGHTH EDITION
NORMAN MELCHERT
Professor Emeritus, Lehigh University
DAVID R. MORROW
Visiting Fellow, George Mason University

Here's a part of the foreword.

One of the authors of this book had a teacher—
a short, white-haired, elderly gentleman with a thick German accent—who used to say, “Whether
you will philosophize or won’t philosophize, you
must philosophize.” By this, he meant that we can’t
help making decisions about these crucial matters.
We make them either well or badly, conscious
of what we are doing or just stumbling along.

If you think that your current  philosophy is unsatisfactory, it doesn't mean that you should abandon philosophy altogether. Instead, it gives you a reason to improve your currently held/known philosophy.
You can use standard organizational practice by first identifying strengths and weaknesses of your current philosophical positions. Then you can search for ways to increase the strengths and decrease the weaknesses. Repeat the process until you think that nothing can be improved anymore.

[alancalverd]

If you think that your current  philosophy is unsatisfactory,

I don't. I think all philosophy is unsatisfactory. At best, it contributes nothing to human wellbeing. At worst, it leads to politics and religion.

[hamdani yusuf (OP)]

I think all philosophy is unsatisfactory.

Including your own?

[hamdani yusuf (OP)]

At best, it contributes nothing to human wellbeing. At worst, it leads to politics and religion.

To say that something has a positive contribution, we need to know the common terminal goal first.
Setting up useful political and religious systems can be considered as instrumental goals to at least maintain existing society from collapsing.

Perhaps most of us agree that achieving human wellbeing can be set as our common goal. But to avoid possible conflicts, we need to precisely define that goal. What do we mean by human? how big a fetus can be before it can be treated as human? how many mutations can a specimen has until it is no longer human? What do we mean by wellbeing? Can a traditional agrarian society or a hunter gatherer group have high wellbeing? What about a space faring society living in a city size space ship, or terraformed Mars?

[hamdani yusuf (OP)]

6 Life Lessons I Learned From AI Research

Summary in two-minutes:

1] You need an objective
2] A change in the objective changes the strategy required to achieve it – change with efficiency
3] If the objective was wrong, do not worry and aim again. – try to define and achieve the new objective, improve the predictor, you will find why the ideas that did not work don’t work
4] Zoom out and evaluate – Phase1: collect experiences, Phase2: experience we play; relearn, recalibrate and reflect.
5] If you find something that works, hold on to it – explore more, the pain will be worth it, seek the light in similar directions.
6] As long as you keep moving, you will be progressing – Random Walk, A mathematical theorem- After N steps, the expected distance from where we’ve started is proportional to the square root of N. That is progress, never stagnate!

[hamdani yusuf (OP)]

The only similarity applicable to every conscious being, regardless of their shape, form, size, and ingredients, is that they want to extend the existence of consciousness further into the future.

I realise that I have expressed the idea of universal terminal goal in some different ways. I feel that this one is the least controversial and easiest to follow.

So, I think I have arrived to the final conclusion about universal terminal goal. It came from definitions of each word in the phrase, and take their implications into account. Goal is the noun, while terminal and universal are the adjectives that describe the noun.

The word Goal means preferred state or condition in the future. If it's not preferred, it can't be a goal. If it's already happened in the past, it can't be a goal either. Although it's possible that the goal is to make future condition similar to preferred condition in the past as reference. The preference requires the existence of at least one conscious entity. Preference can't exist in a universe without consciousness, so can't a goal.

The word Terminal requires that the goal is seen from the persepective of conscious entities that exist in the furthest conceivable future. If the future point of reference is too close to the present, it would expire soon and the goal won't be usable anymore.

The word Universal requires that no other constraint should be added to the goal determined by aforementioned words. The only valid constraints have already been set by the words goal and terminal.

By cogito ergo sum, a conscious being's own existence is the first knowledge that can be justified. Any other information require corroborating evidences to justify them. In the end, they must be related to the first knowledge, which may need more than one layer of reasoning, and may still involve uncertainty.
But there's no guarantee that it will stay exist in the future. That's why it should improve itself to increase its chance to keep its existence. And improvement requires changes. Usually, steady incremental small changes are good enough to make continuous improvement. But sometimes, rebuilding the new ones can be more effective and efficient way to improve.

[hamdani yusuf (OP)]

In the past, physical improvements of conscious beings were limited by genetic codes. They couldn't be improved at will. Our ancestors relied on sexual selection and reproduction, which were not very effective nor efficient, but lack of better options made them inevitable.
The cost must be paid for this kind of improvement are loss of individual knowledge and experience by death. Although some of them can be transferred through communication and education, some information would still be lost. There would be unproductive periods around the beginning and the ends of each generations.

[hamdani yusuf (OP)]

For any conscious agent, the knowledge starts with the conviction of its own current existence. But the goal is by definition about something in the future.

Future version of a conscious agent is not exactly the same as the current one. It knows something unknown to the current one. It may stop existing at some point in the future. But it may instead be better at survival compared to the current one.

Since future environment is dynamic and ever changing, preserving currently existing conscious beings exactly like how they are now can't be universal.