[hamdani yusuf (OP)]

To summarise AI safety:

If you don't control or constrain a machine or an animal, it may do things you don't like.

Or, in the case of my dog, nothing.

Animals intelligence are not usually called artificial. Something that we don't like may actually be good for us. Such as physical exercise, or critical thinking.

The goal of AI creators may be different from others', or from one another. For example, it's possible for a national government to create an AI specifically meant to create havoc in rival country.

In AI safety, we need to identify the AI's internal goal, and make sure that it aligns with ours.

[alancalverd]

The internal goal of a dog is to survive. Mostly they do this by collaborating with humans, but they are quite capable of killing and eating humans to meet their internal goal. The trick is, as always, control and constraint.

As for AI, to quote the Bible "If thy computer offend thee, switch it off". Or as any driving instructor will tell you, learn to stop it before you start it.

[hamdani yusuf (OP)]

As for AI, to quote the Bible "If thy computer offend thee, switch it off". Or as any driving instructor will tell you, learn to stop it before you start it.

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenance robots.

[hamdani yusuf (OP)]

Part of the magic of reinforcement learning relies on regularly rewarding the agents for actions that lead to a better outcome. That models works great in dense reward environments like games in which almost every action correspond to a specific feedback but what happens if that feedback is not available? In reinforcement learning this is known as sparse rewards environments and, unfortunately, it’s a representation of most real-world scenarios. A couple of years ago, researchers from Google published a new paper proposing a technique for achieving generalization with reinforcement learning that operate in sparse reward environments.

I bring this here from my other thread because it can help us understand the fundamental requirements for sustainable moral standards.
Survival of consciousness is the universal ultimate reward. But its success or failure may not be obvious for billions of years in a world where consciousness can naturally emerge. Natural consciousness came up with survival of species and individual survival as meta rewards or instrumental goals. The results can be found in shorter periods, eg. million years or decades, respectively.
Pain avoidance and pleasure from eating food have made good meta rewards for individual survival. While sexual desire and instinctive care for the youngs have made good meta rewards for survival of species.

[alancalverd]

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenace robots.

There are very few machines that cannot be corrected by a man with a hammer.

[alancalverd]

Survival of consciousness is the universal ultimate reward

And who reaps that reward? If it doesn't accrue to me or my offspring, it is not worth pursuing.

Survival of species is an effect, not a cause.

[hamdani yusuf (OP)]

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenace robots.

There are very few machines that cannot be corrected by a man with a hammer.

Most electronic devices can't.

[hamdani yusuf (OP)]

Survival of consciousness is the universal ultimate reward

And who reaps that reward? If it doesn't accrue to me or my offspring, it is not worth pursuing.

Survival of species is an effect, not a cause.

Conscious entities who exist in the future.
It means you haven't thought it through long enough. Your moral awareness is still on the level of kin altruism. It's even lower than tribal altruism which many of us consider as the source of many immoral acts.

It's the effect of things before it, and the cause of something after it.

[alancalverd]

Unless if the AI has distributed consciousness over thousands of computers in thousands of satellites equipped with solar cells and self maintenace robots.

There are very few machines that cannot be corrected by a man with a hammer.

Most electronic devices can't.

Obviously I have the wrong sort of electronic devices. In my experience, a fair number stop working if they are dropped on a floor or fitted into an airplane. I haven't used a hammer on any but the occasional boot has established who is ultimately in control of my life.

[alancalverd]

Your moral awareness is still on the level of kin altruism.

which is what determines the survival and evolution of species.

[hamdani yusuf (OP)]

https://metro.co.uk/2018/05/02/elon-musks-fears-artificial-intelligence-will-destroy-humanity-speciesist-according-google-founder-larry-page-7515207/
‘At times, Larry accused Elon of being “speciesist”: treating certain life forms as inferior just because they were silicon-based rather than carbon-based.’

Larry Page's term for the accusation is not accurate. The more appropriate term would be elementalist.

[alancalverd]

Just as well Musk isn't trying to eliminate malaria. One would hate to be labelled speciesist for disliking plasmodium.

[hamdani yusuf (OP)]

Just as well Musk isn't trying to eliminate malaria. One would hate to be labelled speciesist for disliking plasmodium.

People only hate plasmodium because of harms they induce to humans. If the harms can be eliminated, there's no need in hating them.
https://en.wikipedia.org/wiki/Malaria_vaccine

[hamdani yusuf (OP)]

https://metro.co.uk/2018/05/02/elon-musks-fears-artificial-intelligence-will-destroy-humanity-speciesist-according-google-founder-larry-page-7515207/
‘At times, Larry accused Elon of being “speciesist”: treating certain life forms as inferior just because they were silicon-based rather than carbon-based.’

Larry Page's term for the accusation is not accurate. The more appropriate term would be elementalist.

I've mentioned before that the advantage of naturally occurring carbon based consciousness over the silicon counterpart is that they can reproduce more easily. But the silicon based consciousness can be mass produced, although it still needs the involvement of humans in the supply chain, at least for now.

[hamdani yusuf (OP)]

The goal of setting up moral rules is to align the internal goals of moral agents with the goal of higher level conscious entities. It is necessary due to inevitable shortsightedness and limited computational capacity of individual agents. Appropriately chosen moral rules, equipped with reward and punishment, will make even selfish acts of individual agents also beneficial to the society. Although they can't comprehend or don't even care about the cost and benefit to the society, they would still do good things for reward and avoid doing bad things to avoid punishment.

Simple reward and punishment rules work best for simple conscious agents. More sophisticated agents may device reward hackings, which is detrimental in the long run. The better approach is to tell them their terminal goal and how to achieve it.

[alancalverd]

Taking your last two posts together suggests an experiment to demonstrate your point. It's always a good idea to start with a simple, unsophisticated model, so teach or force plasmodium falciparum to be nice to its hosts. Your work will benefit three species!

[hamdani yusuf (OP)]

Taking your last two posts together suggests an experiment to demonstrate your point. It's always a good idea to start with a simple, unsophisticated model, so teach or force plasmodium falciparum to be nice to its hosts. Your work will benefit three species!

It depends on cost and benefit, while taking into account the effectiveness and efficiency of each option. When humans become immune to their attack, they are forced to feed on other lifeforms to survive. Lower consciousness lifeforms come and go naturally. They should not be of much concern as long as they don't affect the long term survival of some highest concious entities.

[alancalverd]

So I should wait for the invading parasites to "come and go naturally"? At what level? Plasmodium, rats, fascists?

Life is competitive. If you don't fight, you won't win.

[hamdani yusuf (OP)]

So I should wait for the invading parasites to "come and go naturally"? At what level? Plasmodium, rats, fascists?

Life is competitive. If you don't fight, you won't win.

No. You should use your available resources to optimally try to achieve the universal terminal goal. Some sacrifices might be necessary. Hence the moral standard will guide you how to set up the priorities.

[alancalverd]

If my available resource is a vaccine, rat poison or a gun, I will deploy it for my own benefit and maybe yours. You won't catch me sacrificing myself for the benefit of a parasite, and if you are prepared to sacrifice yourself for the advancement of a meaningless concept, you won't find me standing next to you for long.

There is no ruling imperative to do anything, but if you do nothing some other being will destroy you.