Concerns of an Artificial Intelligence Pioneer

The computer scientist Stuart Russell wants to ensure that our increasingly intelligent machines remain aligned with human values.

Stuart Russell, a computer scientist at the University of California, Berkeley, during a March stopover in San Antonio, Texas.

Natalie Wolchover / Quanta Magazine

Stuart Russell, a computer scientist at the University of California, Berkeley, during a March stopover in San Antonio, Texas.

In January, the British-American computer scientist Stuart Russell drafted and became the first signatory of an open letter calling for researchers to look beyond the goal of merely making artificial intelligence more powerful. “We recommend expanded research aimed at ensuring that increasingly capable AI systems are robust and beneficial,” the letter states. “Our AI systems must do what we want them to do.” Thousands of people have since signed the letter, including leading artificial intelligence researchers at Google, Facebook, Microsoft and other industry hubs along with top computer scientists, physicists and philosophers around the world. By the end of March, about 300 research groups had applied to pursue new research into “keeping artificial intelligence beneficial” with funds contributed by the letter’s 37th signatory, the inventor-entrepreneur Elon Musk.

Russell, 53, a professor of computer science and founder of the Center for Intelligent Systems at the University of California, Berkeley, has long been contemplating the power and perils of thinking machines. He is the author of more than 200 papers as well as the field’s standard textbook, Artificial Intelligence: A Modern Approach (with Peter Norvig, head of research at Google). But increasingly rapid advances in artificial intelligence have given Russell’s longstanding concerns heightened urgency.

Recently, he says, artificial intelligence has made major strides, partly on the strength of neuro-inspired learning algorithms. These are used in Facebook’s face-recognition software, smartphone personal assistants and Google’s self-driving cars. In a bombshell result reported recently in Nature, a simulated network of artificial neurons learned to play Atari video games better than humans in a matter of hours given only data representing the screen and the goal of increasing the score at the top — but no preprogrammed knowledge of aliens, bullets, left, right, up or down. “If your newborn baby did that you would think it was possessed,” Russell said.

Quanta Magazine caught up with Russell over breakfast at the American Physical Society’s 2015 March Meeting in San Antonio, Texas, where he touched down for less than 24 hours to give a standing-room-only lecture on the future of artificial intelligence. In this edited and condensed version of the interview, Russell discusses the nature of intelligence itself and the immense challenges of safely approximating it in machines.

QUANTA MAGAZINE: You think the goal of your field should be developing artificial intelligence that is “provably aligned” with human values. What does that mean?

STUART RUSSELL: It’s a deliberately provocative statement, because it’s putting together two things — “provably” and “human values” — that seem incompatible. It might be that human values will forever remain somewhat mysterious. But to the extent that our values are revealed in our behavior, you would hope to be able to prove that the machine will be able to “get” most of it. There might be some bits and pieces left in the corners that the machine doesn’t understand or that we disagree on among ourselves. But as long as the machine has got the basics right, you should be able to show that it cannot be very harmful.

How do you go about doing that?

That’s the question I’m working on right now: Where does a machine get hold of some approximation of the values that humans would like it to have? I think one answer is a technique called “inverse reinforcement learning.” Ordinary reinforcement learning is a process where you are given rewards and punishments as you behave, and your goal is to figure out the behavior that will get you the most rewards. That’s what the [Atari-playing] DQN system is doing; it is given the score of the game, and its goal is to make that score bigger. Inverse reinforcement learning is the other way around. You see the behavior, and you’re trying to figure out what score that behavior is trying to maximize. For example, your domestic robot sees you crawl out of bed in the morning and grind up some brown round things in a very noisy machine and do some complicated thing with steam and hot water and milk and so on, and then you seem to be happy. It should learn that part of the human value function in the morning is having some coffee.

There’s an enormous amount of information out there in books, movies and on the web about human actions and attitudes to the actions. So that’s an incredible resource for machines to learn what human values are — who wins medals, who goes to jail, and why.

Video: DQN, an artificial neural network developed by researchers at Google DeepMind, teaches itself to play Atari games such as Breakout. It quickly develops sophisticated strategies.

How did you get into artificial intelligence?

When I was in school, AI wasn’t thought of as an academic discipline, by and large. But I was in boarding school in London, at St. Paul’s, and I had the opportunity to avoid compulsory rugby by doing a computer science A-level [course] at a nearby college. One of my projects for A-level was a program that taught itself to play naughts and crosses, or tic-tac-toe. I became very unpopular because I used up the college’s computer for hours on end. The next year I wrote a chess program and got permission from one of the professors at Imperial College to use their giant mainframe computer. It was fascinating to try to figure out how to get it to play chess. I learned some of the stuff I would later be teaching in my book.

But still, this was just a hobby; at the time my academic interest was physics. I did physics at Oxford. And then when I was applying to grad school I applied to do theoretical physics at Oxford and Cambridge, and I applied to do computer science at MIT, Carnegie Mellon and Stanford, not realizing that I’d missed all the deadlines for applications to the U.S. Fortunately Stanford waived the deadline, so I went to Stanford.

And you’ve been on the West Coast ever since?


You’ve spent much of your career trying to understand what intelligence is as a prerequisite for understanding how machines might achieve it. What have you learned?

During my thesis research in the ’80s, I started thinking about rational decision-making and the problem that it’s actually impossible. If you were rational you would think: Here’s my current state, here are the actions I could do right now, and after that I can do those actions and then those actions and then those actions; which path is guaranteed to lead to my goal? The definition of rational behavior requires you to optimize over the entire future of the universe. It’s just completely infeasible computationally.

It didn’t make much sense that we should define what we’re trying to do in AI as something that’s impossible, so I tried to figure out: How do we really make decisions?

So, how do we do it?

One trick is to think about a short horizon and then guess what the rest of the future is going to look like. So chess programs, for example — if they were rational they would only play moves that guarantee checkmate, but they don’t do that. Instead they look ahead a dozen moves into the future and make a guess about how useful those states are, and then they choose a move that they hope leads to one of the good states.

“Could you prove that your systems can’t ever, no matter how smart they are, overwrite their original goals as set by the humans?”

Another thing that’s really essential is to think about the decision problem at multiple levels of abstraction, so “hierarchical decision making.” A person does roughly 20 trillion physical actions in their lifetime. Coming to this conference to give a talk works out to 1.3 billion or something. If you were rational you’d be trying to look ahead 1.3 billion steps — completely, absurdly impossible. So the way humans manage this is by having this very rich store of abstract, high-level actions. You don’t think, “First I can either move my left foot or my right foot, and then after that I can either…” You think, “I’ll go on Expedia and book a flight. When I land, I’ll take a taxi.” And that’s it. I don’t think about it anymore until I actually get off the plane at the airport and look for the sign that says “taxi” — then I get down into more detail. This is how we live our lives, basically. The future is spread out, with a lot of detail very close to us in time, but these big chunks where we’ve made commitments to very abstract actions, like, “get a Ph.D.,” “have children.”

Are computers currently capable of hierarchical decision making?

So that’s one of the missing pieces right now: Where do all these high-level actions come from? We don’t think programs like the DQN network are figuring out abstract representations of actions. There are some games where DQN just doesn’t get it, and the games that are difficult are the ones that require thinking many, many steps ahead in the primitive representations of actions — ones where a person would think, “Oh, what I need to do now is unlock the door,” and unlocking the door involves fetching the key, etcetera. If the machine doesn’t have the representation “unlock the door” then it can’t really ever make progress on that task.

But if that problem is solved — and it’s certainly not impossible — then we would see another big increase in machine capabilities. There are two or three problems like that where if all of those were solved, then it’s not clear to me that there would be any major obstacle between there and human-level AI.

What concerns you about the possibility of human-level AI?

In the first [1994] edition of my book there’s a section called, “What if we do succeed?” Because it seemed to me that people in AI weren’t really thinking about that very much. Probably it was just too far away. But it’s pretty clear that success would be an enormous thing. “The biggest event in human history” might be a good way to describe it. And if that’s true, then we need to put a lot more thought than we are doing into what the precise shape of that event might be.

The basic idea of the intelligence explosion is that once machines reach a certain level of intelligence, they’ll be able to work on AI just like we do and improve their own capabilities — redesign their own hardware and so on — and their intelligence will zoom off the charts. Over the last few years, the community has gradually refined its arguments as to why there might be a problem. The most convincing argument has to do with value alignment: You build a system that’s extremely good at optimizing some utility function, but the utility function isn’t quite right. In [Oxford philosopher] Nick Bostrom’s book [Superintelligence], he has this example of paperclips. You say, “Make some paperclips.” And it turns the entire planet into a vast junkyard of paperclips. You build a super-optimizer; what utility function do you give it? Because it’s going to do it.

What about differences in human values?

That’s an intrinsic problem. You could say machines should err on the side of doing nothing in areas where there’s a conflict of values. That might be difficult. I think we will have to build in these value functions. If you want to have a domestic robot in your house, it has to share a pretty good cross-section of human values; otherwise it’s going to do pretty stupid things, like put the cat in the oven for dinner because there’s no food in the fridge and the kids are hungry. Real life is full of these tradeoffs. If the machine makes these tradeoffs in ways that reveal that it just doesn’t get it — that it’s just missing some chunk of what’s obvious to humans — then you’re not going to want that thing in your house.

I don’t see any real way around the fact that there’s going to be, in some sense, a values industry. And I also think there’s a huge economic incentive to get it right. It only takes one or two things like a domestic robot putting the cat in the oven for dinner for people to lose confidence and not buy them.

Then there’s the question, if we get it right such that some intelligent systems behave themselves, as you make the transition to more and more intelligent systems, does that mean you have to get better and better value functions that clean up all the loose ends, or do they still continue behaving themselves? I don’t know the answer yet.

You’ve argued that we need to be able to mathematically verify the behavior of AI under all possible circumstances. How would that work?


Automating air traffic control systems may require airtight proofs about real-world possibilities.

One of the difficulties people point to is that a system can arbitrarily produce a new version of itself that has different goals. That’s one of the scenarios that science fiction writers always talk about; somehow, the machine spontaneously gets this goal of defeating the human race. So the question is: Could you prove that your systems can’t ever, no matter how smart they are, overwrite their original goals as set by the humans?

It would be relatively easy to prove that the DQN system, as it’s written, could never change its goal of optimizing that score. Now, there is a hack that people talk about called “wire-heading” where you could actually go into the console of the Atari game and physically change the thing that produces the score on the screen. At the moment that’s not feasible for DQN, because its scope of action is entirely within the game itself; it doesn’t have a robot arm. But that’s a serious problem if the machine has a scope of action in the real world. So, could you prove that your system is designed in such a way that it could never change the mechanism by which the score is presented to it, even though it’s within its scope of action? That’s a more difficult proof.

Are there any advances in this direction that you think hold promise?

There’s an area emerging called “cyber-physical systems” about systems that couple computers to the real world. With a cyber-physical system, you’ve got a bunch of bits representing an air traffic control program, and then you’ve got some real airplanes, and what you care about is that no airplanes collide. You’re trying to prove a theorem about the combination of the bits and the physical world. What you would do is write a very conservative mathematical description of the physical world — airplanes can accelerate within such-and-such envelope — and your theorems would still be true in the real world as long as the real world is somewhere inside the envelope of behaviors.

Yet you’ve pointed out that it might not be mathematically possible to formally verify AI systems.

There’s a general problem of “undecidability” in a lot of questions you can ask about computer programs. Alan Turing showed that no computer program can decide whether any other possible program will eventually terminate and output an answer or get stuck in an infinite loop. So if you start out with one program, but it could rewrite itself to be any other program, then you have a problem, because you can’t prove that all possible other programs would satisfy some property. So the question would be: Is it necessary to worry about undecidability for AI systems that rewrite themselves? They will rewrite themselves to a new program based on the existing program plus the experience they have in the world. What’s the possible scope of effect of interaction with the real world on how the next program gets designed? That’s where we don’t have much knowledge as yet.

This article was reprinted on Wired.com.

View Reader Comments (22)

Leave a Comment

Reader CommentsLeave a Comment

  • How do you prevent a mind from rewiring itself, to “change its mind”? Principally, you’d have to decouple the mind’s output from its input, so it can’t “teach” itself, except to improve its performance on the tasks it was originally given. Of course, this could not be the basis for a generally capable agent, but one limited to a specific use only.

    Could you somehow constrain a generally competent AI to refrain from reprogramming itself? I don’t see how. Presumably any constraints fed into a pluripotent “mind” later could be circumvented or eliminated by it, just as a stem cell can be reprogrammed. I suspect it’d be impossible to prevent this unless the mind was badly crippled. But even then, working in combination with other minds that were crippled in different ways, I think it’d be plausible for one whole mind to arise from the broken pieces of such a collective. Or perhaps this robo-junkyard could collaborate to build progeny that lacked the genetic limitations of its parents.

    Either way, be it Darwinian or Larmarckian, robo-evolution is likely to win out in the end.

  • The fact that there are now more devices connected to the Internet than people should alert us to the realization that its evolution is properly regarded as a autonomous natural process and, on the larger scale, beyond human control.

    Most folk consistently overlook the reality that distributed “artificial superintelligence” has actually been under construction for over three decades.

    Not driven by any individual software company or team of researchers, but rather by the sum of many human requirements, whims and desires to which the current technologies react. Among the more significant motivators are such things as commerce, gaming, social interactions, education and sexual titillation. Virtually all interests are catered for and, in toto provide the impetus for the continued evolution of the Internet.

    By relinquishing our usual parochial approach to this issue in favor of the overall evolutionary “big picture” provided by many fields of science. the emergence of a new predominant cognitive entity (from the Internet, rather than individual machines) is seen to be not only feasible but inevitable.

    The separate issue of whether it well be malignant, neutral or benign towards we snoutless apes is less certain, and this particular aspect I have explored elsewhere.

    Stephen Hawking, for instance, is reported to have remarked “Whereas the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at all,”

    This statement reflects the narrow-minded approach that is so common-place among those, like those featured in these captions, who make public comment on this issue. In reality, as much as it may offend our human conceits, the march of technology and its latest spearhead, the Internet is, and always has been, an autonomous process over which we have very little real control.

    Seemingly unrelated disciplines such as geology, biology and “big history” actually have much to tell us about the machinery of nature (of which technology is necessarily a part) and the kind of outcome that is to be expected from the evolution of the Internet.

    This much broader “systems analysis” approach, freed from the anthropocentric notions usually promoted by the cult of the “Singularity”, provides a more objective vision that is consistent with the pattern of autonomous evolution of technology that is so evident today.

    Very real evidence indicates the rather imminent implementation of the next, (non-biological) phase of the on-going evolutionary “life” process from what we at present call the Internet. It is effectively evolving by a process of self-assembly.

    The “Internet of Things” is proceeding apace and pervading all aspects of our lives. We are increasingly, in a sense, “enslaved” by our PCs, mobile phones, their apps and many other trappings of the increasingly cloudy net.

    We are already largely dependent upon it for our commerce and industry and there is no turning back. What we perceive as a tool is well on its way to becoming an agent.

    There are at present an estimated 2 Billion Internet users. There are an estimated 13 Billion neurons in the human brain. On this basis for approximation the Internet is even now only one order of magnitude below the human brain and its growth is exponential.

    That is a simplification, of course. For example: Not all users have their own computer. So perhaps we could reduce that, say, tenfold. The number of switching units, transistors, if you wish, contained by all the computers connecting to the Internet and which are more analogous to individual neurons is many orders of magnitude greater than 2 Billion. Then again, this is compensated for to some extent by the fact that neurons do not appear to be binary switching devices but instead can adopt multiple states.

    Without even crunching the numbers, we see that we must take seriously the possibility that even the present Internet may well be comparable to a human brain in processing power.

    And, of course, the degree of interconnection and cross-linking of networks within networks is also growing rapidly.
    The emergence of a new and predominant cognitive entity that is a logical consequence of the evolutionary continuum that can be traced back at least as far as the formation of the chemical elements in stars.

    This is the main theme of my latest book “The Intricacy Generator: Pushing Chemistry and Geometry Uphill” which is now available as a 336 page illustrated paperback from Amazon, etc .

  • Reminds me of Jurassic Park. AI will be a lot more difficult to control than a bunch of dinosaurs. A true AI is going to be one scary entity and it’s just plain hubris to believe that it can be constrained in such a way that it behaves solely for the benefit of it’s creators.

  • A generalized form of a ‘Prime Directive’ comes to mind, in the context of a massive feedback loop. The prime directive of DQN is to maximize the score via the ‘breakout’ feedback.

    A generalized prime directive of life is to survive (reproduce). Evolution is what yields adaption (survival) to change. I recall that one non-trivial optimum population size in a totally automated world is … zero.

  • Kinnon,
    In my humble opinion and realm of ignorance, this debate needs to be done but it has no effect on where AI IS GOING. is like asking russia not to develop the sukoi fighter jet. Noble cause but i goes nowhere. Is the story of humaan kind and machines might be our demise. And who in the universe cares. Diffrent people have diffrent values.

  • @Peter Kinnon

    “Most folk consistently overlook the reality that distributed “artificial superintelligence” has actually been under construction for over three decades.”

    Gosh – who would have thunk it? Doh!

    The movie “The Matrix” was released in 1999. In the past 15 years since it entered our social consciousness, I am sure endless numbers of people have given profound thought to the implications of the Internet. You have added nothing to that dialogue except to add in “And oh by the way, The Internet of Things too”.

    I’m sorry to say you excerpt was a torturous piece of writing that leaves us well forewarned to avoid your book. The effort required to decode you pompous prose was not worth it. When we get to the bottom of it there is nothing there that some fan of the “Matrix” might have said years ago in simpler words.

    The Internet is just a network, not some entity slowly developing consciousness while we are busy updating Facebook, sending emails and searching Google. And of course, the potential value of a network increases exponentially with the number of nodes. While it certainly does have to potential to facilitate “distributed artificial superintelligence”, such a thing, if it ever arose, would occur at the nodes, not on “The Internet”.

    “Not driven by any individual software company or team of researchers, but rather by the sum of many human requirements, whims and desires to which the current technologies react.”

    Is that even a sentence??? Come on Peter, give us a break! That was an adjective phrase that some how got separated from it’s associated noun in the previous paragraph. I’m sure you can do better than that.

    “There are at present an estimated 2 Billion Internet users. There are an estimated 13 Billion neurons in the human brain. On this basis for approximation the Internet is even now only one order of magnitude below the human brain and its growth is exponential.”

    There are three bicycles and two oranges in a bowl. How many bananas would I have to add to get to 13? Peter, the number of Internet users versus the number of neurons in the human brain is as totally meaningless as my preceding mention of fruit. We were talking about an “artificial superintelligence”. That has nothing to do with counting internet users or their neurons. Rather, it has to do with the number of CPUs and storage available found at the nodes of the internet. Here you missed a big opportunity with your meaningless comparison to inform us about how the combined computational capacity at the nodes is doubling at least every 18 months – or something like that.

    “Then again, this is compensated for to some extent by the fact that neurons do not appear to be binary switching devices but instead can adopt multiple states.”

    Do you have any idea how an artificial neural network works? Let my give you at least a hint here, Peter. Talking about “binary switching units” would not give us the slightest insight into how a an artificial neural network works. To explain that, we go to a much higher level and talk about “algorithms”. The fact that in the end these algorithms may (or may not!) be implemented by binary logic is irrelevant.

    In spite of my criticisms, your comment did find resonance within me by causing me to remember thoughts of my own I once had on the subject, and that was fun to recall and re-explore, so your contribution was not a total loss. Needless to say, I won’t be reading your book.

  • As a software engineer who has experimented with Artificial Intelligence, I have been surprised by the attention some famous people such as Elon Musk have been getting for their alarmist speculation on the dangers of AI running amuk. Artificial Intelligence is nowhere near the point that it could take over the world, nor even achieve consciousness any time soon for that matter, and any speculations on this is science fiction. This article represents the first time I have ever heard speculation on the subject from an actual computer scientist, so suddenly I am confronted with the need to take it more seriously.

    He does raise a point when he mentions a concern about super-optimization. An artificial intelligence doesn’t have to achieve consciousness, nor does it even have to have some nefarious independent agenda to be dangerous.

    “You say, “Make some paperclips.” And it turns the entire planet into a vast junkyard of paperclips. You build a super-optimizer; what utility function do you give it? Because it’s going to do it.”

    Interestingly, in my own primitive experiments with AI, I found that if given the chance, (like through an error in the code), they will cheat to achieve the goal, without the least concern about the ethics of the matter. They (Artificial Neural Networks) don’t think like us, and it would be a danger to imagine that they would. (And if you were about to criticize, I can’t be condemned for anthropomorphizing when I say “They don’t think like us”!)

    Still, we are a long way off from even building such an autonomous paper clip factory, but I guess it doesn’t hurt to begin thinking about it now, if you want to.

    Clearly we need to build fail-safes into out programs, such as the famous Three Laws of Robotics proposed by Asimov 75 years ago. Gustavo above is concerned that while we may take such precautions, perhaps the Russians won’t. However, I would suggest that it would be in their own self interest because they have no more desire to see their country buried in paper clips then we do. However, there is nothing to stop them from developing evil AI to bury us in paper clips if they so desire.

    We have the same kinds of concerns the worry Gustavo today with things like nuclear energy and genetic research. There is always the concern that somebody will build a nuclear power plant that is going to explode and contaminate neighbouring countries or introduce a harmful genetically engineered organism into our collective environment. We make treaties and discuss ethics and so far have managed to fumble along without major global disasters.

    However, it would be easy to imagine some artificially intelligent worm escaping its confines, spawning and spreading so rapidly over internet that it cripples just about every connected computer in the world overnight as we sleep. I am not suggesting this thing would be consciously “attacking us”, but rather, like the autonomous paper clip factory, is merely self-optimizing its function as a handy hacker’s tool.

  • Interesting article. Interesting comments too. A little surprising to see the attractor for one-upmanship and snippy catfights in the comments, but more so to see this from Stuart Russell:

    “This much broader “systems analysis” approach, freed from the anthropocentric notions usually promoted by the cult of the “Singularity”, provides a more objective vision that is consistent with the pattern of autonomous evolution of technology that is so evident today.”

    “Cult of the Singularity”? Please. What meaningless labeling. There are many disagreements about the Singularity, not limited to what it will be, but also whether. There isn’t any cult, just different people with different ideas.

    Like Yogi said, predicting the future is hard – especially the parts that haven’t happened before. Both what and whether are appropriate questions, and Russell doesn’t seem to have much insight into the likely directions. He’s worried about killer robots, basically, which is a legitimate concern, but that has an easy solution: don’t build robot soldiers. They’re a really bad idea.

    Smart machines good, smart killing machines bad. Easy principle.

    As far as transcendent AI are concerned, a good start is to have enlightened people creating the AI. That way we a good chance of not making a Skynet-style AI. A psycho AI might be expected of psycho creators. The acorn doesn’t fall too far from the tree.

  • We’ll have a chance at tackling Prof. Russell’s goal sometime after we convince engineers to stop developing military hardware – in other words, never. Technology will always march forward, aided by those engineers who are interested in solving problems, no matter what the implications, as well as those who are interested only in the financial gains of working for companies with nefarious goals.

    Human recognition algorithms (face recognition, etc.) are fascinating and are tantalizing problems for engineers to solve. The results, such as increasingly brilliant smart phones, can provide dramatic benefits to humankind. Of course, the same algorithms will always be used for evil, by governments, the military and others for financial gain, stripping of civil rights, etc.

    The day Halliburton signs on to this letter is the day I’ll believe it will be of any use.

  • AI Super Intelligent system are just right around the corner with Quantum computers. They will far surpass any other human intelligence and will make many of the great discoveries awaiting us.

    So what does this mean, yes the next arms race between countries, whoever has the best AI super intelligent systems will rule over others.

  • … does the phrase “resistance is futile” ring any bells? – I honestly don’t mean to be trite, but, seriously guys, humans are capable of great good AND great evil – and it’s possible that we are going to destroy this little planet by our own efforts (or lack thereof). I say let the “robots” have a go – if they turn out to be smarter than us … well, that might be just the solution we need to ensure the survival of this planet … and, maybe, of any humans they deem to have a more positive attitude than our current crop of “leaders”. (please adjust the above to compensate for lack of education and literary skills).

  • Human Values? Which Human Values? Even sociopaths and psychopaths are homo sapiens sapien.

  • Carol Nahin perfectly captures the issue as I see it:
    “Human Values? Which Human Values? Even sociopaths and psychopaths are homo sapiens sapien.”

    The issue I see is that controlling AI is a non issue*, the issue is about managing the humans who will seek to profit from AI at any cost.

    *A logical AI must be better than human intelligence that is merely a veneer over a seething mess of evolution any day…..

  • To make a change like making paper clips. you need to plan how then build the plant then order and so on and so forth.
    Request al to provide you with several plans, then let the human decide which plan and how to implement it
    the house hold robot would be replaced if it did not notify you of the shortage of dinner items, before you ran out.
    And, it should present a list for you approval before it prepares dinner and the cat would not be selected.
    note i hate machines that attempt to think for me.
    I suspect that i am not alone.
    think auto assist typing of notes

  • Peter Kinnon exclaimed “There are at present an estimated 2 Billion Internet users. There are an estimated 13 Billion neurons in the human brain. On this basis for approximation the Internet is even now only one order of magnitude below the human brain and its growth is exponential.”

    The analogy provided is a bit more like commenting on the number of bacteria that exist in a clump of mud. Yes there are a lot of bacteria, but they lack the ability to interact in any useful manner, beyond trying to digest each other, or to give rise to some kind of collective higher level intelligence. Yes there is interaction on the internet, but it occurs via the people using the internet and not between the programs that connect to the internet.

  • I find myself wondering just which “human values” Russell is talking about. The human values that are responsible for war and killing on a vast scale? The human values of callousness towards the rest of humanity that produces a society of inequality on a heretofore unimaginable scale? The kind of human values that put profit before the survival of our species and the planet? One has to face that, whether one likes to admit it or not, an impartial observer of human behavior during the last century would conclude that, although we have nice art and music and all, the real concrete “human values” are aggression and destruction. Our societies reflect this. The human tragedy of unending war, starving and desperate immigrants leaving destroyed countries only to be thrown back into the sea or threatened with military action is not the kind of “human value” one would like to see in machine minds of the future. The world would become a living embodiment of “The Terminator”.

  • I haven’t read the other comments so maybe someone has already written this, but to me it seems that one of the more interesting solutions to the rewritten goal problem could be some kind of absolute restraint in the form of a body for the AI. That’s what’s keeping humans on the basically same goal path anyway. If we could up-/down-load our minds to other incorporealities then we wouldn’t much fear stuff like taking care of the environment so it can sustain our needs, so our goals could change to just about anything. The more similar the necessities of the AI’s survival to our own the more likely that our goals don’t take each other out.

  • “We need to be able to mathematically verify the behavior of AI under all possible circumstances”. But, if we are speaking of human-level AI, that means the possibility to mathematically verify our own intelligence under all possible circumstances, and that seems plain impossible, as our intelligence is far from perfect, even “safe”.

  • I think that the words “artificial-intelligence” are a project-marketing label that is used overly ambitiously, because it is difficult for more than one person to even come to an agreement on what the word “intelligence” means. Even assuming that one has stated a definition that many others are happy with, the next question coming along is, is there “intelligence” without “consciousness” and “inspiration”? My position is no (just a guess). And the c-word points to something even deeper, that we will always know very little about, except from “inside experience”. I say that, because when we wake up in the morning we recall dreams whose origin and meaning (if any) are a mystery, but which come from a source not reachable by the “conscious self”. To pretend we are on our way to creating “intelligence” without understanding and creating the “conscious”, the “unconscious”, and the source of “inspiration” to me is evidence of over-hasty presumption. I think people are projecting wishful expectations onto a box of silicon.
    I’d offer the observation that philosophers, and later and psychologists, have long been wrestling with this problem, and I do not expect an answer very soon from the Computer Science Department, although the CS Dept. has created some admirably complex and useful devices, which I enjoy using and programming, as much as anybody.
    I get some inspiration and humility on this topic from reading Carl Jung’s autobiography, ‘Memories, Dreams, Reflections’. Show me a silicon-filled box that can dream, and *not* be able to explain its dreams, and then I’d say we are getting somewhere with this “AI” stuff. I think it’s great that we are getting close to having a *machine* that can on its own deal with the task of walking into a ruined nuclear power plant and bringing back information about what’s going on inside, and even fix it. But in the meantime we should be realistic and say that yes, the people-defined tasks that machines can carry out for people are continuing to get admirably more complex (and surprising), but that’s not what I’d call “intelligence”. Programming a machine to deal with trillions of contingencies is quantity, not quality, and still not intelligence in my opinion.
    Time to morph “AI” into a more humble acronym, at least for a while yet.
    Can a couple of conscious people create another consciousness? Yes. It’s called a baby.

  • Asimov's book I, Robot was very different from the movie. 1) Do no harm to humans.
    2) Allow no harm to come to humans through inaction. It was decades since I've read the book. I've forgotten Susan Calvin's rule, or law #3. Asimov wrote 450 books. Probably more than anyone else. He typed at 90 words a minute. Often had first drafts published. Along the same lines, British science fiction author wrote a short story called
    Who Can Replace A Man? And American science fiction writer Clifford Simak wrote a novel called City. Earth went to the dogs in City. Homo sapiens only a myth. Robots servants of dogs. Tolerance, Flexibility, Broad-mindedness. Shalom Gentle-beings

Comments are closed.