Look Hard, Then Steer Slightly by Robin Hanson

Okay, page 1 is all about arguing that attempts to steer the future are futile. We just have to surrender to whatever technology, that autonomous force, is bringing us.

Page 2 introduces the idea that actually, we can do a reasonable job of foreseeing what technology, like it or not, is bringing us.

Then we get to the fun part: Ems. They're like Fisher-Price little people, only smarter and less physical. There's going to be this whole economy and society of ems, or uploads in conventional parlance. Why? Because otherwise AI is going to fail, but uploading will succeed.

The essay, and apparently the book, is full of confident assertions about ems, what they will be like, how they will fit into our world. And I mean confident. There's not the slightest doubt - Ems are going to do all the work, and they're going to have a class hierarchy, money, retirement....

Call me skeptical. I don't see how you get to whole-brain scanning at the level of detail necessary to make an upload work, even assuming you have the hardware to run it on. If you did have that level of technology, I don't see why you wouldn't have an understanding of the brain's circuits and learning mechanisms and more generally, how to make AI of human and superhuman capability that is tailored to what you want it to do, rather than being inherently egoist and potentially dangerous. Claims of expertise in AI research from more than a decade ago don't impress me that much, I'm afraid. Anyone who says there's not been much progress since then isn't paying attention.

I may agree that we're heading into the rapids, but that suggests maybe we'd best paddle to the shore, or look for a rock to wash up on or a branch to grab, i.e. STOP. At least until we can figure out what we do and don't want to do. I don't think the smart move would be to start planning for the age of the elves, I mean, ems.

You are talking about maps of gross pathways or small sections of microscopic detail. If you are going to directly "upload" a fully entrained brain and have it work you need whole-brain data at sub-cellular, perhaps even molecular levels of detail. The whole thing all at once, or it will be damaged, and a large fraction of it at least, or it won't work at all. We're nowhere near that. Also, these maps of axons/dendrites don't reveal the synapse strengths or the cell-level learning mechanisms; we really don't know what is going on below all that spiking activity. If we had that kind of data, we'd probably know enough about how it works that we could design other systems based on the same mechanisms.

I didn't say everything has changed in the last decade but a lot has in terms of the performance level of systems in use and under development today. This has been driven not only by theoretical advances but to a large extent by hardware and data sets. We're still nowhere near brain levels of complexity but if we had the hardware that could run uploads it could as well run brain-derived and hybrid algorithms that would probably be more useful, safer, and ethical than the idea of creating human-like minds to use as slaves or indentured servants - or worse, to replace humanity wholesale.

Destructive scanning is almost certainly necessary, which of course means death of the human being. I don't know what experts believe you need only connection data; nearly everyone would agree you need synapse type and strength and it may well turn out that there is important processing going on inside the cell, i.e. the neuron may be a fairly complex cellular automaton. I expect this is essential to learning as well as emotional and likely even cognitive states, i.e. the axonal output is not a strict function of dendritic input but also depends on internal states which respond to chemical modulators and history. Plus of course there are some huge number of different neuron types, different synapse types, and your scan has to identify each one.

Perhaps if you have only gross connection data that will be enough to pull out something that sort of works, like a person recovering from alcohol coma or some huge pervasive brain trauma, and then your em, while not a faithful upload, can retrain and adapt. But if we know enough to facilitate such self-repair we probably know enough to engineer AI.

Really, the weakest part of your argument (apart from the elf economics) is the notion that we will be able to make uploading work before we can make AI work. Both are very much dependent on having the hardware and for uploads you also need the scans. The assumption that we can make uploads work without understanding how the brain works in detail seems especially unrealistic. But also, neuromorphic circuits are evidently the way to go for brainlike computing, and they are not going to run scan data, because they will be neuromorphic, not neuron simulations.

We will continue to study the brain and use what we learn for medicine and to gain insight which will be used to engineer hybrid neuromorphic and algorithmic AI systems, which hopefully we will use as powerful tools rather than something to replace human beings.

Dear Professor Hanson,

Let me get this straight. According to some credentialed scientists, it has taken thousands of years of evolution for nature to produce the human brain. There appears to be about 7 billion human brains presently active on earth. Each of these human brains had to commence in a baby that was then born to a woman. You are saying that that was a waste of time for women. Here you are, you can fabricate a brain that is far superior to a natural born human one. The mighty Hanson brain only operates to Hanson's concepts of superior scientific knowledge. Hanson knows better than God what a brain should do.

Joe Fisher

No, Joe, you have got it wrong. Most artificial intelligence researchers want to create intelligence artificially, from scratch. Robin Hanson wants to read the soul, God's work, from actual human brains. If he can do it, God's work is not wasted. Indeed some speculate that God is using us to create heaven. The environment inside a computer would look a lot like what heaven is supposed be. The souls living there would live forever, or at least as long as computers could be kept running, perhaps until the heat death of the universe in 101000 years. "When we've been there ten thousand years; Bright shining as the sun; We've no less days to sing God's praise; Than when we'd first begun."

I really enjoyed your essay, Robin. I agree with your overall point that we have a limited ability to influence the future, so our interventions will have to be well thought out. I also agree that seeing the future--to an extent--is not impossible.

But as a fellow social scientist I'm skeptical about our ability to see the future in as much detail as you seem to. Your vision of an em future is plausible, but still seems to me to assume too many things that we can't know. I don't see how, for example, we can possibly what ems--who will in a way be quite alien to us--will feel over hundreds of years about members of their clan. It likewise strikes me as extremely unlikely that ems will stay recognizably the same for long while information technology is developing rapidly. And although the rise of ems would be an enormously important development, I think there are other important developments that could radically alter the trajectory of the future.

You're right that some of the predictions people have made in the past were pretty good. But when enough people make predictions some are bound to be right even if they're just throwing darts. So I don't think we should be too confident in our predictions when we're making plans for the future. We certainly should prepare and plan for something like the scenario you outline. But I also think it is just one of many possible futures we need consider.

Best,

Robert de Neufville

I think we are all over the place about speed because we are making unspoken assumptions about performance over cost. Here are a couple of real life examples. If an em runs on hardware with the same kind of profile as the HD 4870 (first graph), the economically optimal choice will be to run as fast as possible; if it's more like the GeForce GTX 260 (second graph) it will prefer to work at the lower "sweet spot" rate.

As long as we don't know what the hardware is like, we can imagine all sorts of profiles, including weird, monotonically decreasing ones. How much any of that will matter depends on how steep the graph is, i.e. on marginal cost. Larger slopes will cause more herding at the optimal rate. If I had to bet, I would bet on that kind of tight range, with ems down-clocking only if idle and up-clocking (if at all possible) only to cope with temporary spikes in workload too small to warrant delegation. But, who knows what the real profile will be like.

I dont't have a strong opinion (yet...) on the feasibility of scanning, but I think we should at least be aware of what the guy says who currently seems closest to actually doing it: "The key insights that allowed us to bring the HBP [Human Brain Project] to fruition were the realisation that it is just not possible to map the brain experimentally [...]". :/

Robin,

What if there were a far larger and immediate problem such as a financial medium designed to siphon value out of virtually the rest of the economy and store it as notational promises and this system was going parabolic, resulting in ever more social and environmental resources being consumed to power it. Wouldn't this be a problem for the actual generation involved to deal with?

Wouldn't both social engineers and even mechanical engineers view this as the elephant in the room? Yet most entries seem obsessed with what amounts to science fiction.

Regards,

John Merryman

I certainly don't want to end the conversation, Robin. I would love to see the draft of your book. My point was not that we can't know the future and shouldn't bother speculating. In fact, I think it is very important to speculate in the way you do. My point was rather that we need to recognize that our speculation is speculation. We have a strong cognitive tendency to imagine that the plausible scenarios we invent are necessary futures. But because of our limited ability to see the future, we need to plan and prepare for a wide range of contingencies.

So honestly, you are surprised to find that there are websites which do not exclusively focus on your favorite issue?

Robin,

That was the most fascinating essay so far. Thank you. I certainly agree with your log scenario too.

But one question; Do you think our real human brains are well enough developed to focus on ..cloning? from them, rather than focus on improving the way we use them, so improve their capability?

I think I perhaps see a long term plan hatching, 100 years to evolve our intellect more by better training us how to think, (to steer that log better), then when we're ready we'll have a much more useful em's!

I'll definitely look out for your book. I can't score at present for some reason, but look out for a boost when my powers return.

I see you have a physics masters so hope you'll read mine, a combination of.... well, you'll see I hope.

Well done and I hope your essay rises during the impending malestrom.

Peter Jackson

(full name given as though the system tells me I'm logged in I don't think AI's quite truthful enough yet!)

Robin,

True, but I propose we could be far more useful if we also learned to use our on-board computers properly, including by thinking outside the Earth-centric frame. You queried if steering to a 'quantum leap' in understanding of nature (unification of classical and quantum physics) answered the question (on my blog) I responded as below;

Robin,

I'm an enabler. I 'implement' near impossible projects, in energy, defence etc. See my post on Sabine's essay. I've learned that most of what mankind does, if not just theorising, is to treat symptoms. Unintended and reverse outcomes are common because we don't think deep enough or think through implications.

Many of the essays here either consider symptoms, or don't actually propose how to move ahead at all. Just saying; 'we must do this or that' is useless. That's why we stumble from crisis to crisis, one of which may be our end.

Uniting classical and quantum physics is now almost 100 years overdue. It will have the most fundamental effects on all scientific understanding, so also technology of any other discovery or advancement; certainly QG, and I'm afraid also AI, because it will enable major leaps in both. We will then understand exactly which of ALL the u isseus facing us must be addressed AND how to address them.

Also our most fundamental understanding is advanced. The same model informs cosmology at the widest level. Have read of this papers, just accepted (but not in a major journal), if you're interested in the first evolutionary sequence of galaxy types ever produced, and a credible re-interpretation of the so called 'big bang' with pre-'BB' condition logically implicit. The whole construct in empirical and coherent, resolving dozens of anomalies. Preprint; HJ Vol.6 2014.

Now this can all be 'action today!' But none of the 'discrete field dynamics' model will be accepted by mainstream science in the near future because physicists neural networks are imprinted with the present paradigm so reject alternatives despite evidence and logic. Of course that's no evidence that it's wrong. It is in fact self evident. So I (we) must find a clear cut 'way in' to get people used to the different concepts. Success would give a clear direction, impetus and certain advancement. Now how many other essays can do that? Despite many good ones, very very few. (please point any out I've missed so far.)

Best wishes

Peter

Hi Robin,

Great essay! I enjoyed your river analogy. I agree with you, the first step in solving humanity's problems is identifying those problems, and foreseeing their effect. In my essay , I touch upon a similar idea.

best regards,

Mohammed