[ukmicky]

Every variation would be murder or manslaughter and anyone who kills an innocent person is an idiot.

[ukmicky]

Here is another plot twist I just thought of. You have inadvertently switched the track when you learn about the situation, so the train is heading toward one person. Will you switch it back to its original track, which will kill 5 people, instead of letting your mistake inadvertently kill 1?
In this scenario, both decisions involve you action. Hence there is no excuse of being a passive bystander.

Another variation of the plot twist. You didn't pull the lever inadvertently. You pulled it because you weren't aware of the one person on the second track.
When you become aware of that person, will you switch back to the original track, which will eventually kill 5 people?

hmmmm

[hamdani yusuf (OP)]

hmmmm

It seems like you haven't thought much about this problem. There are many variations of it out there. They are tools to check the consistency of your decision making process. Sanity check for our moral standards, which we need to figure out urgently more than ever, as full self driving vehicles are becoming reality. The understanding of the universal moral standard becomes a necessity when we are engaging AGI and genetic engineering. Economic competition is forcing us to get there sooner, rather than later.

[alancalverd]

The understanding of the universal moral standard becomes a necessity when we are engaging AGI and genetic engineering. Economic competition is forcing us to get there sooner, rather than later.

No. What we need to do is to impress a human  moral standard on  semiautonomous machines.

A classic example of not doing so is the use of "altitude hold" in a simple autopilot. Most of the time this just saves you having to make continuous minor adjustments of power and trim as you burn fuel, fly into a different weather system, or the passengers start walking around. But there is a dangerous temptation to let "George" fly the plane in strong turbulence, because his reactions are quicker and he doesn't get tired. This can  be fatal. If you hit a strong downdraft, George will point the nose of the plane upwards to regain altitude and you may stall. Unfortunately the strong downdrafts are found in dense cloud, so the plane will quite suddenly flip, spin, or do a dozen ballistic things all at once and topple the gyro horizon, making recovery to aerodynamic flight quite a conundrum and in some cases impossible.  The proper thing to do is to fly by hand in strong turbulence, accepting that you will not maintain constant speed or altitude, but simply keep the wings generating lift even when the meal trays hit the ceiling, in a compromise with the forces of nature. Given the choice between the unpalatable and the unacceptable, you must accept the unpalatable to survive.

Which takes us back to the "immoral but right" decisions of conflict, which cannot be universal because the rest of the universe is at best indifferent to human life, and at worst, opposed to it. 

[hamdani yusuf (OP)]

No. What we need to do is to impress a human  moral standard on  semiautonomous machines.

To accept your suggestion, you need to define what human is, and why it is necessary to be followed by non-human conscious agents.

[hamdani yusuf (OP)]

A classic example of not doing so is the use of "altitude hold" in a simple autopilot. Most of the time this just saves you having to make continuous minor adjustments of power and trim as you burn fuel, fly into a different weather system, or the passengers start walking around. But there is a dangerous temptation to let "George" fly the plane in strong turbulence, because his reactions are quicker and he doesn't get tired. This can  be fatal. If you hit a strong downdraft, George will point the nose of the plane upwards to regain altitude and you may stall. Unfortunately the strong downdrafts are found in dense cloud, so the plane will quite suddenly flip, spin, or do a dozen ballistic things all at once and topple the gyro horizon, making recovery to aerodynamic flight quite a conundrum and in some cases impossible.  The proper thing to do is to fly by hand in strong turbulence, accepting that you will not maintain constant speed or altitude, but simply keep the wings generating lift even when the meal trays hit the ceiling, in a compromise with the forces of nature. Given the choice between the unpalatable and the unacceptable, you must accept the unpalatable to survive.

You can put the handwritten codes of known practical rules into the machines' algorithm. Alternatively, you can train the machine using reinforced learning, given that you can provide a virtual environment accurate enough to represent parts of the real world which is considered relevant to the application. Tesla's Dojo clearly takes the second route.

[hamdani yusuf (OP)]

Which takes us back to the "immoral but right" decisions of conflict, which cannot be universal because the rest of the universe is at best indifferent to human life, and at worst, opposed to it. 

What you do is a better representation of your morality than what you say. In your case, I'll call them fake morality and real morality.

[alancalverd]

Alternatively, you can train the machine using reinforced learning, given that you can provide a virtual environment accurate enough to represent parts of the real world which is considered relevant to the application.

And look what happened to the 737 Max. One jammed sensor and several hundred dead. But apparently cheaper than putting a warning in the pilot's notes and spending an extra hour on type training.

Back in the days of Tridents, a friend was flying a Trident 3 from Belfast to London when he noticed "it felt a bit heavier than usual". Looked at the fuel ticket and saw that he had been given the fuel for a much thirstier T2, so he called back to Belfast who called the next guy who had just departed in a T2 for Paris, checked his fuel ticket, and diverted to Birmingham rather than ditch in the Channel. Not sure how you could program that into anything except three humans, never mind getting Birmingham Control to reorganise all their traffic. 

[hamdani yusuf (OP)]

And look what happened to the 737 Max. One jammed sensor and several hundred dead. But apparently cheaper than putting a warning in the pilot's notes and spending an extra hour on type training.

It was human error in design phase.

[hamdani yusuf (OP)]

Not sure how you could program that into anything except three humans, never mind getting Birmingham Control to reorganise all their traffic. 

Perhaps the plane engine shouldn't be able to start with incorrect fuel amount or type for the planned journey. It would be easier to fix.

[hamdani yusuf (OP)]

Non-universal moral standards can have many forms. Here is an example.

WOTM: In-Group Morality

[hamdani yusuf (OP)]

PragerFU: Where Do Good and Evil Come From?

[alancalverd]

And look what happened to the 737 Max. One jammed sensor and several hundred dead. But apparently cheaper than putting a warning in the pilot's notes and spending an extra hour on type training.

It was human error in design phase.

Not at all. The design was perfect and simply introduced an interesting new characteristic in the flight envelope. The wrong decision was to substitute weak automation for an hour's training. 

Almost every time you get to fly a new type or even a significant variant of a familiar type, you fly a simulator or "live" with an instructor, to learn  where the new bits are and how it handles differently from the old one, then you get a formal "type rating" added to your licence. Boeing management, not engineering, decided that this could be avoided by automating the pitch control, despite several previous problems with auto pitch control  mechanisms on various jets including some famous fatalities.

The sudden pitch up of a 737MAX after rotation was known (other aircraft have similar characteristics) and entirely simulable or teachable, and indeed the European Aviation Safety Authority, despite its appalling record of shambolic maladministration, now insists that pilots must have an examined rating for the MAX including how to switch the bloody system off!

In each of the fatal cases the pilots knew something was wrong and tried but couldn't override the servo.

[alancalverd]

Not sure how you could program that into anything except three humans, never mind getting Birmingham Control to reorganise all their traffic.

Perhaps the plane engine shouldn't be able to start with incorrect fuel amount or type for the planned journey. It would be easier to fix.

Yes, one more thing to go wrong. Suppose we have planned Belfast- Heathrow but a passenger gets sick so we divert to Birmingham. Will the machine say "fuel overload - switch off engine"? Or Heathrow is fogged so we make a late divert to Southend. Is that another 30 miles (insufficient fuel - switch off engine) or had we planned to approach Heathrow from the east anyway, plus a go-around plus a divert plus extra taxi time?

Fueller overloads the plane, or you have a late change of destination. Offloading fuel is a hell of a faff, but flying another 30 minutes to get below landing weight is no big deal.

And will the machine talk to Heathrow/Birmingham/Southend/Airways Control and request priority? What will it do if Birmingham says "negative, go to Manchester"?

The great thing about flying is that you have dozens of options until the fuel runs out, so why not leave it to a human to negotiate with other humans and decide what to do next? I have been rightly criticised by an examiner for rushing towards a preplanned diversion instead of reviewing the situation with some arithmetic and a radio chat. He said "Yes, Southend is nearest but there's no direct train to Cambridge - can we reach Norwich?"   

The Trident story just illustrates the importance of checking that third parties have indeed done what they should have, and recognising when things have changed. Humans are generally good at that. One neat gadget that deserves to be resurrected is the electromechanical final checklist. This is a row of switches labelled "fuel" "flaps" etc which you have to flip upwards before takeoff and downwards before landing, and when they are all in the same orientation you get a green light. It just reminds you to do all the stuff in the handbook, but it doesn't interrupt the fuel flow to the engines, or attempt to take control of the flight surfaces.

Anyway it's a long way from morality. Apologies.

[hamdani yusuf (OP)]

WOTM: Objective Morality

[hamdani yusuf (OP)]

Not at all. The design was perfect and simply introduced an interesting new characteristic in the flight envelope. The wrong decision was to substitute weak automation for an hour's training.

Not providing redundancy for critical components which can cause a single point failure is a sign of bad design.
How did they fix the problem? By changing the design.

[hamdani yusuf (OP)]

Yes, one more thing to go wrong. Suppose we have planned Belfast- Heathrow but a passenger gets sick so we divert to Birmingham. Will the machine say "fuel overload - switch off engine"? Or Heathrow is fogged so we make a late divert to Southend. Is that another 30 miles (insufficient fuel - switch off engine) or had we planned to approach Heathrow from the east anyway, plus a go-around plus a divert plus extra taxi time?

The machine should tell before the engine even started.

[alancalverd]

So it knows before we take off that the Heathrow runway will be blocked by a broken-down fire truck an hour later? That's pretty clever by any standards.

Human pilots just allow for the possibility by uploading enough fuel for at least 40 minutes holding, two approaches, and a nominated divert, plus 30 minutes fuel remaining after all that.

Our Masters at EASA even went so far as to say that you may not make an approach with less than 40 minutes fuel remaining. That isn't what they meant, but it was what they said, and AFAIK it is still the law!   

[alancalverd]

Not at all. The design was perfect and simply introduced an interesting new characteristic in the flight envelope. The wrong decision was to substitute weak automation for an hour's training.

Not providing redundancy for critical components which can cause a single point failure is a sign of bad design.
How did they fix the problem? By changing the design.

There were no critical components in the original design. The flying characteristics of any aircraft change in different phases of flight, but rather than teach people to fly the MAX throughout the modified takeoff envelope they installed two wholly unnecessary critical components (yes, the system incorporated redundancy) and software that made things worse if one failed.

[hamdani yusuf (OP)]

There were no critical components in the original design. The flying characteristics of any aircraft change in different phases of flight, but rather than teach people to fly the MAX throughout the modified takeoff envelope they installed two wholly unnecessary critical components (yes, the system incorporated redundancy) and software that made things worse if one failed.

I've read the summary of the investigation report. It was the system design flaw which allows a single sensor failure to cause disastrous consequence.
Having a secondary component standby as a backup is not necessarily means you have effective redundancy, if the backup can't take over the functionality of the primary component when it fails.