[hamdani yusuf (OP)]

Then you would be exporting something that isn't a human. And as it is arguable that environmental stress (including radiation damage) leads to evolution, you would be exporting something too robust to evolve.

As long as they are not perfect, there's always some room for improvement. Especially when they have access to redesign their own body.

[hamdani yusuf (OP)]

Have you seen a terrarium that have been running for decades supporting lives isolated from outside world?
In principle, there's nothing to prevent humans from building suitable size terrarium for humans.

We already live in a very good one. Better to keep it tidy than to bugger it up and build another one where it isn't wanted.

As long as it's not perfect, there's always some room for improvement. Especially when we have access to redesign our own environment.
How long do you think it will stay as good as it is? Should we all just die and extinct with it?

[hamdani yusuf (OP)]

in the effort to achieve the universal terminal goal.

defined by whom?

By any conscious beings who survived long enough to consider all logically possible alternatives.

[hamdani yusuf (OP)]

in the effort to achieve the universal terminal goal.

defined by whom?

By any conscious beings who survived long enough to consider all logically possible alternatives.

And here's my opinion about consciousness. Even I have a problem recalling what I've written here previously.

Vision alone only covers input part of a conscious system. It still requires other parts like self awareness, preference or feedback mechanism, and output or actuating system to change its environment. However, it's possible to build a conscious system where its input from environment is exclusively visual.

It looks like those subsystems of conscious entities contribute to overall consciousness, while their effectiveness can be independent from each others. In this case, overall consciousness can be represented as the result of matrix multiplication of the subsystem's effectiveness to achieve their respective goals.

[hamdani yusuf (OP)]

This video is about AI safety. But it's analogous to morality, which is historically about Natural Intelligence safety, especially within human species. Both of them concern about goal alignment.

[alancalverd]

To summarise AI safety:

If you don't control or constrain a machine or an animal, it may do things you don't like.

Or, in the case of my dog, nothing.

[hamdani yusuf (OP)]

To summarise AI safety:

If you don't control or constrain a machine or an animal, it may do things you don't like.

Or, in the case of my dog, nothing.

Animals intelligence are not usually called artificial. Something that we don't like may actually be good for us. Such as physical exercise, or critical thinking.

The goal of AI creators may be different from others', or from one another. For example, it's possible for a national government to create an AI specifically meant to create havoc in rival country.

In AI safety, we need to identify the AI's internal goal, and make sure that it aligns with ours.

[alancalverd]

The internal goal of a dog is to survive. Mostly they do this by collaborating with humans, but they are quite capable of killing and eating humans to meet their internal goal. The trick is, as always, control and constraint.

As for AI, to quote the Bible "If thy computer offend thee, switch it off". Or as any driving instructor will tell you, learn to stop it before you start it.

[hamdani yusuf (OP)]

As for AI, to quote the Bible "If thy computer offend thee, switch it off". Or as any driving instructor will tell you, learn to stop it before you start it.

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenance robots.

[hamdani yusuf (OP)]

Part of the magic of reinforcement learning relies on regularly rewarding the agents for actions that lead to a better outcome. That models works great in dense reward environments like games in which almost every action correspond to a specific feedback but what happens if that feedback is not available? In reinforcement learning this is known as sparse rewards environments and, unfortunately, it’s a representation of most real-world scenarios. A couple of years ago, researchers from Google published a new paper proposing a technique for achieving generalization with reinforcement learning that operate in sparse reward environments.

I bring this here from my other thread because it can help us understand the fundamental requirements for sustainable moral standards.
Survival of consciousness is the universal ultimate reward. But its success or failure may not be obvious for billions of years in a world where consciousness can naturally emerge. Natural consciousness came up with survival of species and individual survival as meta rewards or instrumental goals. The results can be found in shorter periods, eg. million years or decades, respectively.
Pain avoidance and pleasure from eating food have made good meta rewards for individual survival. While sexual desire and instinctive care for the youngs have made good meta rewards for survival of species.

[alancalverd]

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenace robots.

There are very few machines that cannot be corrected by a man with a hammer.

[alancalverd]

Survival of consciousness is the universal ultimate reward

And who reaps that reward? If it doesn't accrue to me or my offspring, it is not worth pursuing.

Survival of species is an effect, not a cause.

[hamdani yusuf (OP)]

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenace robots.

There are very few machines that cannot be corrected by a man with a hammer.

Most electronic devices can't.

[hamdani yusuf (OP)]

Survival of consciousness is the universal ultimate reward

And who reaps that reward? If it doesn't accrue to me or my offspring, it is not worth pursuing.

Survival of species is an effect, not a cause.

Conscious entities who exist in the future.
It means you haven't thought it through long enough. Your moral awareness is still on the level of kin altruism. It's even lower than tribal altruism which many of us consider as the source of many immoral acts.

It's the effect of things before it, and the cause of something after it.

[alancalverd]

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenace robots.

There are very few machines that cannot be corrected by a man with a hammer.

Most electronic devices can't.

Obviously I have the wrong sort of electronic devices. In my experience, a fair number stop working if they are dropped on a floor or fitted into an airplane. I haven't used a hammer on any but the occasional boot has established who is ultimately in control of my life.

[alancalverd]

Your moral awareness is still on the level of kin altruism.

which is what determines the survival and evolution of species.

[hamdani yusuf (OP)]

https://metro.co.uk/2018/05/02/elon-musks-fears-artificial-intelligence-will-destroy-humanity-speciesist-according-google-founder-larry-page-7515207/
‘At times, Larry accused Elon of being “speciesist”: treating certain life forms as inferior just because they were silicon-based rather than carbon-based.’

Larry Page's term for the accusation is not accurate. The more appropriate term would be elementalist.

[alancalverd]

Just as well Musk isn't trying to eliminate malaria. One would hate to be labelled speciesist for disliking plasmodium.

[hamdani yusuf (OP)]

Just as well Musk isn't trying to eliminate malaria. One would hate to be labelled speciesist for disliking plasmodium.

People only hate plasmodium because of harms they induce to humans. If the harms can be eliminated, there's no need in hating them.
https://en.wikipedia.org/wiki/Malaria_vaccine

[hamdani yusuf (OP)]

https://metro.co.uk/2018/05/02/elon-musks-fears-artificial-intelligence-will-destroy-humanity-speciesist-according-google-founder-larry-page-7515207/
‘At times, Larry accused Elon of being “speciesist”: treating certain life forms as inferior just because they were silicon-based rather than carbon-based.’

Larry Page's term for the accusation is not accurate. The more appropriate term would be elementalist.

I've mentioned before that the advantage of naturally occurring carbon based consciousness over the silicon counterpart is that they can reproduce more easily. But the silicon based consciousness can be mass produced, although it still needs the involvement of humans in the supply chain, at least for now.