
Yudkowsky’s whole argument is that it seems like it is nearly impossible to do it right the first time and that there is a good possibility we don’t get two tries. Not because it is malevolent, but because it is really hard to put into numbers what we care about.










I think it is still too hard. The hurdles, safety and difficulty prevent humans adequately for the moment.