In the discussion over the nature of mathematics, one interesting side-argument that recently started was over the idea of "understanding." What does it take to really understand something? The answer is not as simple as you'd think. In fact it is a hotly debated topic in artificial intelligence (AI) circles. J.R. Searle gave the following argument - called the Chinese room argument - against what is called Strong AI. I wrote a post on my personal blog about this two years ago and some of the following is excerpted from there.

Imagine a monolingual English speaker/reader in a room. The person has on a table an instruction booklet, pen, and paper. Notes written in Chinese are then passed into the room. The instruction booklet tells the person things like, "If you see Chinese character X on a slip of paper and Chinese character Y on another slip of paper, write Chinese character Z on your pad." Chinese speakers outside the room label the slips going in 'stories' and 'questions' and the slips coming out 'answers to questions.' The instruction manual can be as sophisticated as you'd like. The question is, does our English speaker/reader - who only speaks and reads English - understand the Chinese, i.e. the details of the story and the associated questions and answers? Searle says no. To Searle, the room and the English speaker/reader is a computer and you can run as sophisticated a program as you'd like, but that it cannot understand Chinese regardless of the program. As such, Searle claims no program can be constitutive of understanding.

There have been critiques to Searle's argument and the first one that comes to mind is adaptability. In a sense, for example, one might say that a spam filter "learns" what is spam and what is not and thus "understands" spam. Likewise, linguistic programs can "learn" language. Is there not an element of understanding inherent in learning?

Now, one point to remember here is that Chinese is a non-phonetic language and should remain completely indecipherable to a native English speaker without formal training. Another take on it is the systems approach where one states that while the man is not intelligent, the system (i.e the entire room) is. Take for example the human brain, one could argue that none of our neurons are intelligent however when they are combined together within the human brain , they represent an intelligent system (i.e the human).

The only problem with the systems approach in this particular case is that the room isn't necessarily an interconnected system. The person and the papers are, but the actual processing boils down to neurons in the guy's brain in the end. So I'm not sure I buy that argument, particularly since it implies that a highly complex robot would be intelligent and suggests we likely would already have created AI then. It does bring up a very interesting question, though. If it is true that a systems approach is the answer, at what point does a system become complex enough to exhibit true intelligence? This is an interesting question related to emergence. But where does emergence occur?

For our English speaker to truly understand the ideas behind the Chinese symbols they need to be translated into English (or someone needs to explain them to him, which is essentially the same thing). So, does translation then produce understanding? Again, no. It is still possible for the English speaker to execute the program and he/she at least understands more of the symbols, but it is akin to recoding the program into the native language of the operating system (the English speaker). But there are plenty of things written in English that I don't understand. I may understand the individual words, but I don't understand the way in which they are combined. For a rudimentary example of this, imagine I'm a spy and intercepted the code "The jackal screams at midnight." I know what all of those words mean individually, but I have absolutely no idea of their context. It seems to me, then, that true understanding requires context. As it turns out, Aerts, Broekaert, and Gabora considered this about ten years ago. More recently, Svozil has made in-roads into this exact process by proposing a context translation principle.

So where does that leave us? It seems that there is an important connection between the notion of true "understanding" and emergence & complexity.

    • [deleted]

    Whatever "understanding" is, it cannot be achieved via thought, as there is always a further "why" to deal with... ad infinitum.

    • [deleted]

    Isn't this simply a reforumalation of Icompletenes, in the spirit of Godel? To understad something, you need to form relationhips among the members and the rules which associte them. In the cause of language, you need defintions, which are essentialy axiomatic assignments that we use to create the mental imagery in our thoughts that corresponds to the objects we are experiencing through the senses. You can never simply take the relations we form among the members and use them to derive the existence and structre of the defintions themselves.

    Maybe, but I think there's more to it than simply Gödelian incompleteness. There is clearly a link to emergence and potentially to consciousness here and I fail to see how Gödel's theorems shed any light on that.

    • [deleted]

    I dissagree. This is a tour-de-force example of the incompleteness of any axiomatic system to account for itself. Ultimately, the ecamples you presented are an isssue of association. Asking the person to understand Chinese from the relations given is the same as asking a computer to reproduce the fundmanetal axioms that it was supplied by using only those axioms.

    • [deleted]

    Lets start from the beginning. A human being feels the absolute mysteriousness of his own existence, and of the world in which he exists. Immanuel Kant said "I feel awe at two things, the starry skies above me and the moral universe within me.". Even though one's existential situation is not clear, one has, somehow, a moral sense allowing one to navigate the world populated with others. Somehow, there is the ability to recognize what one ought to do in this or that situation. What would it mean to understand one's situation? I suppose that it would mean to have some sort of narrative in one's mind about one's origin, how one is constituted and perhaps one's end. Even if one cannot obtain that in a way that is satisfying, it is anyway, not necessary in order to be a morally upright person. This is the context in which "understanding" is interesting to me. Many narratives have been created, and some of these have led to either the strengthening or the weakening of the moral sense, fostering either a helping or fearing/attacking of others.

    So, we build a narrative about self and world, wanting it to be full of veracity, not contradicting any facts which can be established. We attempt to build a 'Scientia' of 'Physis', a knowledge of nature, in which we are embedded. What are the limits to this endeavour? How far can our narrative be sketched out? How far can thought go? Many, many facts can and have been discovered, but the big picture involves piecing these facts together into a grand narrative. Can the narrative ever be something other than Mythos, though filled with facts?

    5 days later
    • [deleted]

    Dear Ian,

    "...If it is true that a systems approach is the answer, at what point does a system become complex enough to exhibit true intelligence? This is an interesting question related to emergence. But where does emergence occur? ..."

    "...It seems that there is an important connection between the notion of true "understanding" and emergence & complexity. ..."

    I will reply to the idea of 'understanding' in a follow-up message. I am replying to the ideas of emergence and complexity with regard to how they relate to intelligence. I think that intelligence cannot 'emerge'. The concept of emergence, I think, is a substitute to avoid admitting that we do not know the fundamental properties from which intelligence is generated. By generated, I mean that those fundamental properties must pertain directly to intelligence itself or they can never lead us to an understanding of intelligence.

    The notion of emergence merely represents the point at which we are forced to abandon our mechanical notions of the universe and admit that we and intelligence are the most important properties that this universe has given birth to,and, that we do not know how or why. It is the point where our theoretical ideas clearly fail us. I think that this point of being forced to abandon our mechanical ideas is synonymous with the probable fact that it means our mechanical ideas are artificial right from the start.

    Complexity is a building process. However, it can only form structures that are already implied in the fundamental properties that that complexity is built upon. Theoretical physics has nothing to offer us with regard to the property of intelligence. Artificialo intelligence is just what its name implies. It is totally artificial. It is our pretense or false claim to have the ability to mechanically produce the property of intelligence. Any theoretical effort to imply that intelligence can emerge magically from fundamental properties that represent pure dumbness should be summarily rejected as a logical fallacy.

    James

    James,

    I think I have to disagree with you on the point that theoretical physics has nothing to offer us in relation to this.

    First, I think it is important that we recognize that there is a difference between intelligence and consciousness. Where "understanding" comes into this, is not entirely clear. But, at any rate, it does seem to be the case that the more complex an organism is, the more intelligence intelligence it seems to possess. Note, however, that this seems to be restricted to a certain class of "organisms," i.e. there seems to be something special about biology. Until we understand what that is, we won't really be able to make much progress toward the "Holy Grail" of AI.

    Ian

      • [deleted]

      Analysis is diabolical; that is, it divides the whole into parts, and they remain unjoined until the the symbolical joining process is undertaken. Scientific knowledge has become so specialized that in different corners different persons work on very specific intellectual pursuits, while very, very few have enough breadth of knowledge to even attempt to rejoin the parts into a whole.

      In one corner of science, there is the problem of understanding of how mind arises from matter (brain). In another corner, there is the problem of understanding how mind and matter form an inseparable whole (QM). Not for nothing did Bohr say "Those who look at QM and are not shocked have not understood it." Feynman said "This is the only mystery." This is a schizoid situation that we are in, and it is crazy making. Its not trivial to say that what overall picture science gives us determines our future. There are numerous other examples of how much we have broken the unity of self/nature. We are creating fragmented pictures of ourselves, other creatures and existence as a whole.

      The Shamanic cultures and persons who daily experience the unity of self/nature are often branded "pre-scientific" as a pronouncement of the superiority of the scientific cultures. But from the outside, it is we who look fragmented, tormented and violent.

      In one corner, people are trying to establish what linear progression in time has led to the present (Evolution). In another corner, other people have already established that time does not enjoy an ontological status, that it is a relative metric not to be understood as self-existent (Relativity). In yet another corner, space and time have been shown to not even have a noumenal status (QM).

      In one corner of human intellectual activity, it is considered that survival is the overarching dynamic of existence, because... survival happens. In another corner, people have established that there is a hierarchy of dynamics with survival being only one of the lot... other dynamics elucidated include the establishing of relationships which even trump one's own survival; further, there is also expression, creativity and transcendent self-knowledge which have been established as dynamics every bit as important as survival and competition.

      How shall we a construct a overall narrative that somehow encompasses all of this, and more? If we perform poorly in the Symbolical, joining endeavour, we will only deepen and extend our own suffering, as well as harming those nearby that we share the world with.

      • [deleted]

      Ian,

      Mind and matter are an inseparable whole (QM), so how can one exist without or before the other? Therefore, how can one emerge from the other? Mind and life are always discerned together; where there is life, there is mind also. So, what applies to the emergence of mind also applies to the emergence of life. Thus, how could life emerge from matter, if matter and mind are inseparable?

      Are we at an intractable impasse?

      7 days later
      • [deleted]

      Another issue that is i.m.o. far more interesting than the Chinese Room Argument is the following. If you have an intelligent machine, you can arrange it to be run in a fully determinstic environment. Eg., just simulate its brain inside a virtual environment.

      Now, you can then argue that the physical states it evolves through can be mapped one to one to those of a simple system that runs trough the same number of states, like a clock. But how can a clock have the same consciousness as a (digital) brain?

      One possible answer is that the digital brain is really running a nontrivial program. If you had fed it counterfactual input, it would have had given you the corresponding counterfactual output. In contrast, the clock can only run in one particular way. But then, one can object to this answer by asking how on Earth counterfactuals that, by definition, refer to things that did not happen, can be relevant at all?

      My own take on this is that we should think of the consciousness generated by a brain as an effective Hamiltonian that describes its time evolution, instead of a physical state. In principle, a real brain is described by some microscopic Hamiltonian that describes everything that goes on in the brain. What we call consciousness should then be obtained (in principle, of course) by integrating out all the microscopic details.

      If you touch an hot object then the microscopic Hamiltonian will describe exactly all the electric activity in your nerves leading to your arm being pulled back. The effective Hamiltonian describes the same process but now in terms of coarse grained variables (e.g. your fingers, your arm, but not individual nerve cells).

      Now, not all of these coarse grained variables can refer to the "trivial" macroscopically observable quantities of the system. E.g. if we apply this procedure to balls that can collide with each other by integrating out the atomic degrees of freedom, in order to obtain a description in terms of only the center of masses of the balls and total momenta, we would be forced to include a few extra variables such as internal energy or temperature, entropy etc. otherwise you cannot get a closed macroscopic description.

      So, you could well imagine that the effective Hamiltonian will describe the system in terms of arms, legs and other body parts, but also new variables will enter and they could correspond to the subjective things we can feel such as pain etc.

      This then addresses the issue of how counterfactuals can matter. If what we really are is an Hamiltonian, the effective version which, loosely speaking, describes only the things we are conscious of, then many different counterfactual states at the micro level will be mapped to the exact same macro state. The way all these states evolve under the miscoscopic Hamiltonian completely fixes the effective Hamiltonian, even how it operates for counterfactual macrostates.

      To illustrate that last point, suppose we look at watermolecules as the can be found in a cup of water at some temperature. The macro description only describes the thermodynamic phase the water is in (ice, water , steam). By studying the water molecules inside the cup of water, you can in principle reconstruct the full Hamiltonian of the interacting water molecules and that then fixes the properties of ice and steam also.

      One consequence of this picture is that conscious observers cannot be thought of as living in the a single parallel universe in the Many Worlds Interpreation of QM. A precise microstate does not define an Hamiltonian. An observer having a definite conscious experience is an Hamiltonian of the form:

      [math]H = \sum_{r,s}h_{r,s}|r\rangle\langle s|[/math]

      where the microstates r and s that are summed over are consistent with the conscious experience. Restricting r and s to be one particular state would not yield a proper Hamiltonian.

        6 days later

        Now that's *really* interesting. I will have to think some more about it (I'm what is called a "processor" rather than a "reactor" I guess) but, at first glance, it seems like an idea worth thinking about in greater depth.

        • [deleted]

        Ian,

        I think I should move slowly, selecting one point at a time. You said:

        "...Note, however, that this seems to be restricted to a certain class of "organisms," i.e. there seems to be something special about biology. ..."

        I think the word 'restricted' is the most revealing part of this statement. I think it is an important part of the evidence that theoretical physics is not helpful and is instead detrimental to solving the problem of the existence of intelligence. In my opinion, it is theoretical physics that is the cause of the appearance of this restriction and separation of biological properties from the rest of the universe.

        I think that empirical physics and any mathematical equations that properly model the patterns seen in empirical evidence is capable of helping us remove the problem of 'restricting' our understanding of the origin and evolution of intelligence.

        I think the basic problem is that when theoretical physics is included, it brings with it an interpretation of the properties of the universe that are 'restricted' to dumbness. Neither intelligence nor awareness, and any other suggestion that the fundamental properties of the universe are anything other than dumb and purposeless without intelligent meaning, is permitted for consideration due only to the imposition of a philosophical preference that is introduced, not for scientific reasons, but for serving humanistic attitudes.

        I do not say this to promote religion. I say this to enable us to see that the fundamentals of the universe are responsible for the intelligence necessary to make decisions such as prefering one philosophical preference over another. If we disregard human emotional needs and concentrate on the evolution of the universe, then I think we should recognize that all effects, including all intelligent effects, must have been provided for in potential form since the beginning of the universe.

        James

        • [deleted]

        Ian,

        Following on my previous message, I think that the universe does not have the logical breaks that we tend to see in its operation. I think that all such divisions are artificial and are of our own making. Life, intelligence, and the rest of the universe shared their origin.

        "...Where "understanding" comes into this, is not entirely clear. ..."

        I would say that 'understanding' is not clear at all. The first problem encountered when studying the development of understanding is the recognition that we receive a storm of tiny, multitudinous numbers of truncated almost random signs of events that have occurred. We call these tiny signs 'photons'. There is no way to find meaning in that storm unless we already know what to look for and already contain the means to understand it. The means for understanding must exist first. That 'means' had the same origin as did everything else.

        Understanding in its macroscopic form is attached to complex forms of life. However, it is also attached to complex arrangements of particles of matter. It arises from the cooperative properties of that 'matter'. We have no problem seeing that all other types of results arise from the cooperative properties of matter. However, we do not know why any of these results arise. Theoretical physics offers us a way to view the mechanical type results. However, the existence of intelligent life demonstrates that matter cannot be what theoretical physics tells us it is.

        James

        • [deleted]

        Ian,

        "First, I think it is important that we recognize that there is a difference between intelligence and consciousness. ..."

        I think between the two of these consciousness is almost like an afterthought. We absorb an incredible amount of information from photons. We select, by intelligent means, patterns from a hodgepodge of patterns. We may have selected correctly, and, we may have selected incorectly. Of these two possibilities, the first gives us the data for reality and the second gives us the data for illusion. Once the patterns are chosen, we attach meaning, by unknown means, to these patterns.

        The combination of meaning and patterns is used to create an image. The image is formed within our minds. We do not experience distance, we imagine distance. We imagine distance because that is how we attach meaning to some of the patterns in photonic data. We visualize distance, but do not directly experience it ourselves. We only experience photonic information locally within our minds.

        Out subconscious mind performs all of the work necessary to draw a conclusion about the meaning of selected photonic information. We do not consciously experience this highly complex decoding. After our subconscious mind reaches a conclusion, its answer is transferred to our conscious mind. Our conscious mind only knows the conclusion and not the process that produced it. The processs is flawed and chancey. Our subconscious mind may be delivering a strong conclusion or a weak conclusion. It helps us out by sending along with its conclusion an emotional feeling. It is the emotional feeling that tells us if our subconscious mind is confident about its conclusion or is iffy about it.

        The subconscious mind knows only what it has received in terms of patterns of information. It can conclude that it is certain about its conclusion even when its conclusion is clearly in error. This is why optical illusions work. We can be made aware that we are seeing curved lines when in fact we are seeing parallel lines. We can be made aware that we see motion when in fact we are seeing a sequence of still pictures.

        A great deal of understanding is risky because, we may be understanding or we may only conclude that we are understanding. However, there are some real, permanent ways to determine at least some real understanding. I will give an example of real understanding, passed on to us by our DNA, in a followup message.

        James

        James,

        You said:

        "... I think that the universe does not have the logical breaks that we tend to see in its operation. I think that all such divisions are artificial and are of our own making. Life, intelligence, and the rest of the universe shared their origin."

        I completely agree with you. This is one of the views of what is called "QBism" or quantum Bayesianism. You may find this paper of interest.

        Ian

        • [deleted]

        Observers as Hamiltonians can perhaps also be motivated directly from the MWI of quantum mechanics. It has been argued that the way the classical world effectively emerges is due to decoherence. Of course, this picture has to be true at a certain level, decoherence certainly exists and unless quantum mechanics as we know it is somehow wrong, I don't see what could be wrong with this picture.

        But on a deeper level there seems to be something not quite right. If we consider the entire multiverse, then the Schrödinger equation for that can be argued to reduce to H|psi) = 0. Time can only really exist at a local level, at least that is the most natural picture considering relativity. But if the entire multiverse is a state |psi) which is an eigenvector of the full Hamiltonian, then the information about the excited states of H is not present in |psi) itself.

        This means that the observations of local observers cannot be extracted from

        |psi) itself. I think that this fact is recognized by the researchers in quantum gravity; they seem to invoke some extra structure in the multiverse to make individual universes with nontrivial dynamics well defined.

        Interesting points. Note, however, that H|psi) = 0 (which is the Wheeler-DeWitt equation), can also be used to model closed time-like curves. I have a preprint floating around out there (which will hopefully get cleaned up and published at some point) that notes that it is possible to have temporal evolution with such a Hamiltonian. It merely needs to be cyclic in nature.

        I don't think MWI is necessarily the answer, though, nor do I think decoherence is the only explanation. There's a statistical argument for it to: the more complex a system is, the less likely it is to be perfectly reversible (some of the details behind this idea go back to Eddington in the 1920s).

        8 days later
        • [deleted]

        Ian,

        Here is my example of real understanding with the means by which we may know it:

        Photons signify something to us causing us to search inside our being for the store of true knowledge that is already ours.We must decide within our subconscious minds what possible relationships the photon data has to our inborn intelligence. When this evaluation process is completed, the conclusion becomes a conscious thought. A physical feeling is communicated along with the thought. If it were not for this feeling, we wouldn't know the difference between a good thought and a poor one.

        This emotional feeling may be the result of either programmed behavior or learned behavior. The programmed behavior may be our response to feeling happiness. We can learn to experience happiness from possession of material things. We may smile, laugh or even shout triumphantly. However, our programmed response is very different and much more important.

        Here, I am reaching for the emotion of happiness that is unique and has permanence. I will refer to it as real joy. Real joy is the feeling we experience while witnessing the birth of a loved baby. It is the uniqueness of this emotion that I wish to clearly identify.

        The emotion of real joy does not cause us to laugh. It causes us to cry. It does not cause us to squeal with delight. It causes us to become mute, even possibly temporarily losing our ability to speak. It does not cause us to leap into the air. It is more likely to cause us to feel weak in the knees. It does not cause us to act proud and haughty. It reduces us to a posture of humility and humbleness. This is the kind of emotion that lets us know that this is real happiness. The kind of happines we are programmed to know as permanently real.

        This is an example of our being naturally directed to fundamental truth. There is a general process by which natural truths are revealed to our conscious minds. We still must learn how to use it to its fullest extent. We are not programmed to always cry when a fundamental truth is being revealed to us. The process is more complex and varied than that. However, there must be a preset physical response, if we will recognize it, for confirming truth. If we allow our thoughts to be guided along this natural path of intelligent thought, then we can understand everything real that we are capable of knowing.

        James

        Hmmm... I guess I'm a bit more of a rationalist but I'll have to think about that.