[hamdani yusuf (OP)]

In emergency cases, where time is severely constrained, the details could be more relaxed.

To get best result, most likely emergency scenarios must be considered, along with planned counter measures which are written down into standard procedures. Emergency drills must be scheduled so the procedures can be effectively executed when needed.

[hamdani yusuf (OP)]

Golden rule is inherently subjective. Its effectiveness relies on the fact that most conscious agents want to preserve themselves, or at least part of them or their duplicates. When those are not the case, it's no longer effective.
Some examples are shown in this video.

[alancalverd]

Nothing to do with generalised "conscious agents". Morality is purely human because we have no way of knowing what any other species thinks. And I'm happy that I have shown it to give consistent results.

Interesting video. Note how nobody who preached the virtue of martyrdom had actually killed themselves or their families. There will always be parasites, perverts  and teenagers, but the parasites and perverts failed test 1, from which the teenagers should have deduced that their preaching was immoral.

[alancalverd]

What makes you think that your standards for morality are better than theirs?

Take a simple case. Imagine you are a pervert charged with deciding who can drive in your country. Because you have a penis problem, you determine that women may not drive. Now apply Test 1. If a woman said you may not drive because you don't have a vagina,  would you be happy?

[alancalverd]

there are some things you judge as immoral, yet still pass those tests.

I don't recall one. Please remind me.

[hamdani yusuf (OP)]

Nothing to do with generalised "conscious agents". Morality is purely human because we have no way of knowing what any other species thinks. And I'm happy that I have shown it to give consistent results.

How do you know other humans think? What if human colonizers of Mars evolved to adapt there so they are no longer interbreed with earthbound humans? Do they stop having morality?

[hamdani yusuf (OP)]

there are some things you judge as immoral, yet still pass those tests.

I don't recall one. Please remind me.

Circumcision is occasionally medically indicated and has preventive value. Ritual of any sort is stupid and ritual that hurts others is wrong. Oddly, ritual circumcision apparently passes both of my tests if you ask a devout Muslim or Jew, but if it's going to hurt, it should be done only under informed consent and therefore restricted to adults. Or I could argue that it fails Test 1 because you wouldn't like it if I cut off your ear to satisfy my religious convictions that you do not share. So it's wrong under the general provisions of slavery law.

[hamdani yusuf (OP)]

What makes you think that your standards for morality are better than theirs?

Take a simple case. Imagine you are a pervert charged with deciding who can drive in your country. Because you have a penis problem, you determine that women may not drive. Now apply Test 1. If a woman said you may not drive because you don't have a vagina,  would you be happy?

Replace penis with age, and woman with kid. What would be the difference?

https://en.wikipedia.org/wiki/Appeal_to_emotion

Appeal to emotion or argumentum ad passiones ("argument from passion") is a logical fallacy characterized by the manipulation of the recipient's emotions in order to win an argument, especially in the absence of factual evidence.[1] This kind of appeal to emotion is a type of red herring and encompasses several logical fallacies, including appeal to consequences, appeal to fear, appeal to flattery, appeal to pity, appeal to ridicule, appeal to spite, and wishful thinking.

The appeal to emotion is only fallacious when the emotions that are elicited are irrelevant to evaluating the truth of the conclusion and serve to distract from rational consideration of relevant premises or information. For instance, if a student says "If I fail this paper I will lose my scholarship. It's not plagiarized" the emotions elicited by the first statement are not relevant to establishing whether the paper was plagiarized. On the other hand, "Look at the suffering children. We must do more for refugees." is not uncontroversially fallacious, because the suffering of the children and our emotional perception of the badness of suffering may be relevant to the conclusion. To be sure, the proper role for emotion in moral reasoning is a contested issue in ethics, but the charge of "appeal to emotion" often cannot be made without begging the question against theories of moral cognition that reserve a role for emotion in moral reasoning.

Appeals to emotion are intended to draw inward feelings such as fear, pity, and joy from the recipient of the information with the end goal of convincing them that the statements being presented in the fallacious argument are true or false, resp.

[hamdani yusuf (OP)]

Soon enough, machines would drive safer than human drivers.

It's just a matter of time until AGIs write better laws and regulations than human lawmakers. They just need the input of correct terminal goal to achieve, and they will make the most effective and efficient instrumental goals (i.e. laws and regulations) necessary to achieve that terminal goal.

[alancalverd]

Replace penis with age, and woman with kid. What would be the difference?

Sadly, the first condition seems inevitable but the difference is significant.

You can't hold an infant legally liable for the harm he does to others, so you draw an arbitrary line (which can be moved in the light of experience) below which you cannot take charge of a lethal weapon.  All the evidence from civilised countries shows that at any age, women are actually safer drivers than men.

[alancalverd]

You seem to have missed the point on circumcision. Ritual infant circumcision is immoral because it falls into the category of  mutilation for the benefit or pleasure of the person doing it, whereas voluntary adult circumcision is an initiation rite undertaken with consent.

Whether it is a Big Deal is another matter. As I argued elsewhere, morality is binary but the impact of an immoral act, and therefore the extent to which it should be punished, lies on a continuum.

[alancalverd]

How do you know other humans think?

52% of Americans seem capable of making a rational decision, which suggests some inherent capability similar to mine.

What if human colonizers of Mars evolved to adapt there so they are no longer interbreed with earthbound humans? Do they stop having morality?

Most social animals seem to have a code of conduct that involves mututal respect intraspecies, but not necessarily interspecies. I do not treat mosquitoes and humans alike.

[alancalverd]

It's just a matter of time until AGIs write better laws and regulations than human lawmakers.

A semiliterate  chimpanzee couldn't write worse laws than the European Union, but you'd need to define a "better" law to make your point.

[hamdani yusuf (OP)]

Whether it is a Big Deal is another matter. As I argued elsewhere, morality is binary but the impact of an immoral act, and therefore the extent to which it should be punished, lies on a continuum.

You can also say that black and white are binary. But then you'll have trouble describing continuous shades of grey, or explaining some optical illusions.
Punishment is a method to modify preferences of conscious agents to be more aligned with those of the punishers. Its effectiveness relies on assumptions that most moral agents prefer to avoid that type of punishment. But that's the best that our ancestors can come up with without the ability to modify people's minds. 

[alancalverd]

Black is the absence of visible light, white is a spectrum approximating to sunlight. Grey is what we perceive when the level of white light is above zero and insufficient to saturate the retinal rods. Optical illusions are just that - misconstrued data. 

[hamdani yusuf (OP)]

It's just a matter of time until AGIs write better laws and regulations than human lawmakers. They just need the input of correct terminal goal to achieve, and they will make the most effective and efficient instrumental goals (i.e. laws and regulations) necessary to achieve that terminal goal.

This is an example where lawmakers produce suboptimum laws.

[alancalverd]

Like I said, a semiliterate chimpanzee. But what would an AI system do? What is the goal of the law in this case?

[hamdani yusuf (OP)]

Whether it is a Big Deal is another matter. As I argued elsewhere, morality is binary but the impact of an immoral act, and therefore the extent to which it should be punished, lies on a continuum.

It seems like you are choosing deontology instead of consequentialism. Do you think that lying is immoral? Is there a situation where it's different?

[alancalverd]

That one was dealt with by Maimonides. A lie is permissible

1. To save a life
2. To comfort the dying
3. To avert a greater wrong.

I think these exceptions pass the moral tests.

[hamdani yusuf (OP)]

That one was dealt with by Maimonides. A lie is permissible

1. To save a life
2. To comfort the dying
3. To avert a greater wrong.

I think these exceptions pass the moral tests.

Those exceptions make lying not inherently immoral.
It's not surprising that our inconsistencies in defining morality create widespread apathy among people to discuss morality scientifically.