Intention is Physical by Natesh Ganesh

Lee Bloomquist

Natesh, I can't find an equation that defines "dissipation" in the essay. Is it in the references? I can find "the lower bound on dissipation in this system as it undergoes a state transition..." But that seems specific to finite state machines, which are not equivalent in power to Turing machines. Is it just "delta E"? where E is energy lost from something like a thermodynamic engine?

Natesh Ganesh

Dear Bloomquist,

The dissipation by the system S into the bath B is captured by the \Delta expression of the bath. This would be the change in the average energy of the bath and since only S can exchange energy with B, the increase in energy of the environment would be due to the dissipation by S due to a state transition. A much detailed treatment of the methodology I use is available in the reference as cited in the submission.

Addressing your comment about finite state automata (FSA) vs Turing machines, you are right that Turing machines have greater power but the Markov FSA as I have defined in the essay is still very general and will allow for a wide range of scenarios. Furthermore I have heard arguments that biological organisms need not be Turing machines and capable of computing all functions. Having said that I recognize that the model prescribed here can be vastly improved. I am currently working on something a little more general than the FSA I have prescribed here (still not a Turing machine yet) that will still allow for some insightful takeaways.

I want to note that I substitute the dissipation bound for the actual dissipation, since for the biological processes we would be interested in the bounds are good approximations of the actual dissipation. Furthermore I want to add that, the dissipation bounds though might have a Landauer-like essence to it, it is more rigorously derived and overcomes some of the objections that critics have raised against the hand wavy like calculations in Landauer's original paper. You can view it as energy lost by a very specialized type of thermodynamic engine.

Thank you again for the comments. Please let me know if I missed in address anything else you have brought up.

Natesh

Natesh Ganesh

Dear Bloomquist,

Thanks for your comments and very interesting links. I will have to look through them in detail as soon as I can. I am hoping we can find many more examples of biological systems that satisfy the idea

presented in this essay. Here are some of my thoughts on your comments-

I have not thought about the behavior of a larger system comprising of many small minimally dissipative parts in detail but if I have to venture a guess, I would think some sort of cooperative behaviour would emerge. Also with respect to the fish example, the fish is a system that can act on its environment and thus its behavior is a tradeoff between 'exploration vs exploitation' under this idea and would not just be a form of predictive learning that we would see in a system that

cannot act on its environment.

While there may be many languages, some providing a more detailed and useful definition of purpose, the language in this paper would explain the emergence of what these other languages describe better.

Thanks again for your comments.

Natesh

Natesh Ganesh

Dear Jim,

I am glad you find it interesting. Yes, while England's ideas have been a big step forward in the right direction, there are some caveats in his hypothesis and I illustrate those points and present a way to unify individual learning and evolutionary processes under the single fluctuation theorems.

"but humanly done with purpose and goals?" I am sorry but I fail to understand your question. Can you help me out here?

I might have used finite state automata/machine which are popular in computer engineering and and being one I am very familiar with them. Their popularity in computer engineering does not reduce their general applicability.

Thanks for your comments. Let me know if there are other things I can clarify if you get a chance to view it in detail.

Natesh

Natesh Ganesh

Dear Jeff,

Thank you for your encouraging comments and kind rating. Gives me greater confidence to carry on and to work harder. Yes, it was tough but after several edits I think I managed to find a good balance of math vs no-math. And the language of math is always beautiful and adds so much to the discussion, wouldnt you agree. I am hoping more people will read this essay.

Natesh

Natesh Ganesh

Dear Joe,

Thank you for your very kind and encouraging comments. Inspires me to work harder. I am glad that I managed to communicate the ideas in the essay coherently to you. Yes, the topic of this essay is at a very unique intersection of so many different fields. I wish I wasn't right at the word limit and had more room to discuss a bunch of other things. There is a much needed discussion of semantics, consciousness and the implications of the ideas presented here on the philosophy of the mind that I would have loved to delve into.

The title of your essay is very intriguing. I am caught up at a conference for the next two days but I will definitely read your essay in detail over the weekend and get back to you with questions/comments. I look forward to reading your thoughts on this problem. Thanks a lot again for your encouragement.

Natesh

[deleted]

Thank you for your patience with me, Natesh. I see you are at a nanotechnology lab, so I imagine that "I squared R" dissipates a lot of heat that is of concern. How much dissipation in your hypothesis is from I^(2) R ?

From my work experience in software, implementing a FSM by an array with two indexes, one for current state, one for current input signal, storing at those indexes next state and output signal, is much quicker and easier to debug than leaving the FSM in a lot of "if-then" statements-- I.e. using data instead of code increases the speed of execution and reduces debugging significantly. Then if the rest of the code is needed for the full Turing machine, most of the speed loss and complexity in my coding experience would come from the full Turing machine, not the FSM. That's why I ask. And, in your line of work, it seems to me that the vonNeumann architecture would be more relevant for I^(2) R dissipation than more abstract models like co-algebras, streams, or Chu spaces for modeling computing. For example, as in Samson Abramsky's Big Toy Models.

In engineering terms, apoptosis is a quality control in animals (it never occurs in plants) that seems like a way to minimize energy loss and therefore may be relevant to your dissipation hypothesis. In apoptosis, defective cells that would be energy-wise inefficient are destroyed and their components re-used to build other cells, thus holding onto the potential energy in the sub-assemblies and again reducing energy loss. But I don't see how I^(2) R heat loss plays a role in apoptosis.

If you had an equation for dissipation rather than a verbal explanation, It might give me something more general than I^(2) R to think about-- especially regarding apoptosis as an example of minimizing dissipation of energy. What are your thoughts?

Regarding your references-- I will try to find them online. Are they in arxiv? The closest big universities where I could photocopy these papers are hours away from me.

Lee Bloomquist

Sorry, the previous post was me. Don't know how that happened.

Lee Bloomquist

Natesh Ganesh

Hi Lee,

I am happy to answer all questions. I want to point out again that I use the dissipation bound as a (good) approximation of the actual dissipation in the processes that we are interested in. The dissipation bound expression as entropy \delta S and mutual information terms I. The dissipation bound is fundamental and relates to dissipation associated with irreversible information loss. It is implementation independent. When you talk about I^2*R expression for dissipation, you are thinking about wires and charges moving through those wires. So that expression will not hold for spin based architectures. However irrespective of implementation, if there is information loss, then there will be fundamental dissipation associated with it characterized by entropy and information terms. Hence their presence in the expression. For a long time, these bounds from fundamental law (informally called Landauer bounds) were many many orders of magnitude lower than the I^2*R and leakage dissipation and were not significant. But decades later and thanks to Moores law, we will hit such Landauer limits in a decade or so. At the most fundamental level, we define a broad physical implementation of finite state machine that is architecture or technology (like CMOS) independent, and the bound in the paper is the bound of that nature and only depends upon just definition achieved physically. We can always add the dissipation of the architecture and circuit on top of this. Unfortunately the paper in the references that can make this so much clearer is not on arxiv. If you can access them on a college campus somewhere, here are some other papers that will make a lot clearer and what exactly I am talking about, especially the second one.

Anderson, Neal G. "On the physical implementation of logical transformations: Generalized L-machines." Theoretical Computer Science 411.48 (2010): 4179-4199.

Anderson, Neal G., Ilke Ercan, and Natesh Ganesh. "Toward nanoprocessor thermodynamics." IEEE Transactions on Nanotechnology 12.6 (2013): 902-909.

Thanks.

Natesh

Lee Bloomquist

Natesh. All I can get on the second paper is the abstract and a bit of the intro. The rest is behind a paywall.

You wrote, "When you talk about I^2*R expression for dissipation, you are thinking about wires and charges moving through those wires. So that expression will not hold for spin based architectures." It's Joule heating, as in this article:

http://poplab.stanford.edu/pdfs/Grosse-GrapheneContactSJEM-nnano11.pdf

And, you wrote about "entropy":

"I want to point out again that I use the dissipation bound as a (good) approximation of the actual dissipation in the processes that we are interested in. The dissipation bound expression as entropy \delta S and mutual information terms I."

But the abstract talks about "energy dissipation":

****

Abstract:

A hierarchical methodology for the determination of fundamental lower bounds on energy dissipation in nanoprocessors is described. The methodology aims to bridge computational description of nanoprocessors at the instruction-set-architecture level to their physical description at the level of dynamical laws and entropic inequalities. The ultimate objective is hierarchical sets of energy dissipation bounds for nanoprocessors that have the character and predictive force of thermodynamic laws and can be used to understand and evaluate the ultimate performance limits and resource requirements of future nanocomputing systems. The methodology is applied to a simple processor to demonstrate instruction- and architecture-level dissipation analyses.

...I. Introduction

Heat dissipation threatens to limit performance gains achievable from post-CMOS nanocomputing technologies, regardless of future success in nanofabrication. Simple analyses suggest that the component of dissipation resulting solely from logical irreversibility, inherent in most computing paradigms, may be sufficient to challenge heat removal capabilities at the circuit densities and computational throughputs that will be required to supersede ultimate CMOS.1

For 1010devices/cm3 each switching at 1013sтИ'1 and dissipating at the Landauer limit EminтЙИkBT have Pdiss=414 W/cm2 at T=300K Comprehensive lower bounds on this fundamental component of heat dissipation, if obtainable for specified nanocomputer implementations in concrete nanocomputing paradigms, will thus be useful for determination of the ultimate performance capabilities of nanocomputing systems under various assumptions regarding circuit density and heat removal capabilities.

****

Are you saying the heat dissipation in post-CMOS devices is just due to spin, and none of that heat is due to Joule heating?

Best Regards,

Lee

Lee Bloomquist

Natesh, thank you for your reply! You wrote:

"Also with respect to the fish example, the fish is a system that can act on its environment and thus its behavior is a tradeoff between 'exploration vs exploitation' under this idea and would not just be a form of predictive learning that we would see in a system that cannot act on its environment."

Is what you wrote, above, about probability-learning foraging fish implied by the definitions of terms in your hypothesis? That is, do the definitions of the terms in your hypothesis (like "open," "constraints," etc.) imply what you have written, above?

Your hypothesis: "Open physical systems with constraints on their finite complexity, that dissipate minimally when driven by external fields, will necessarily exhibit learning and inference dynamics."

Or, is more than this required to understand your hypothesis-- more than just the above statement of your hypothesis together with definitions of the terms used in your hypothesis?

sridattadev kancharla

Dear Ganesh,

I wish you all the best with your in depth analysis of how intentions govern reality. I welcome you to read there are no goals as such in which I propose that consciousness is the fundamental basis of existence and that intent is the only true content of reality. Also that we can quantify consciousness using Riemann sphere and achieve artificial consciousness as per the article Representation of qdits on Riemann Sphere. I saw that you are also arriving at study of consciousness in physical systems in the conclusion of your essay. Also please see all the diagrams I have attached in my essay.

Love,

Natesh Ganesh

Hi,

Thanks for reading my essay and your comments. I have read your work and will politely disagree with you on your premise and conclusion though. I do think that starting from consciousness as a fundamental basis of existence is not the right approach. I would argue that consciousness is an emergent property in input mapping that occur in certain systems due to thermodynamic constraints. I think we will just have to agree to disagree. Also you should check out the integrated information theory of consciousness by Tononi and Koch. I think you will enjoy their work. Please rate my work if you enjoyed reading it. Thanks and good luck in the contest.

Cheers

Natesh

Satyavarapu Gupta

Dear Ganesh,

Thank you for the good essay on "Intension is Physical"

You are observations are excellent, like... "fundamental limits on energy efficiency in new computing paradigms using physical information theory as part of my dissertation"

I have some questions here, you mean to say for every external input, our Brain predicts and takes a correction. It wont be acting on directly by its own self....

Probably if we make energy efficient computer, it will become super intelligent, Probably we may require some software also....

Even though my essay (Distances, Locations, Ages and Reproduction of Galaxies in our Dynamic Universe) is not related to Brain functions, It is on COSMOLOGY....

With axioms like... No Isotropy; No Homogeneity; No Space-time continuum; Non-uniform density of matter(Universe is lumpy); No singularities; No collisions between bodies; No Blackholes; No warm holes; No Bigbang; No repulsion between distant Galaxies; Non-empty Universe; No imaginary or negative time axis; No imaginary X, Y, Z axes; No differential and Integral Equations mathematically; No General Relativity and Model does not reduce to General Relativity on any condition; No Creation of matter like Bigbang or steady-state models; No many mini Bigbangs; No Missing Mass; No Dark matter; No Dark energy; No Bigbang generated CMB detected; No Multi-verses etc.

Dynamic Universe Model gave many results otherwise difficult to explain

Have a look at my essay on Dynamic Universe Model and its blog also...

http://vaksdynamicuniversemodel.blogspot.in/

Best wishes................

=snp. gupta

Natesh Ganesh

Hi,

Thanks for your comments. Here are my answers to some of the questions you have brought up.

"I have some questions here, you mean to say for every external input, our Brain predicts and takes a correction. It wont be acting on directly by its own self...."

--> If you think of the brain as an hierarchical predictive machine, everything you experience is a result of the continuous prediction-error correction mechanism it is executing. And using terms like "self" is a little loaded and misleading. The sense of self is the result of a physical system that I have described in the essay just evolving under physical law.

"Probably if we make energy efficient computer, it will become super intelligent, Probably we may require some software also...."

--> Yes, it is a very new idea in computing called 'thermodynamic computing'. There is very little work right now but it is gaining momentum. And the point is that there is no fixed algorithm or software. The hardware is set to evolve under larger thermodynamic constraints (as stated in the minimal dissipation hypothesis) for a large set of different environments and it will learn.

I will take a look at your essay but I am an engineer by training and my knowledge in cosmology is very limited, so please forgive me if I cant fully grasp the ideas you express in them. Good luck in the essay contest and please rate if you have enjoyed the work.

Cheers

Natesh

Willy K

Your work is highly mathematical. Although my math skills are not good enough to understand the details, I think it is a tremendous work. I would love to see your work on consciousness, qualia and meaning, which you hinted at in the essay, primarily because I had considered those areas well-nigh impervious to mathematics. But I am pretty sure you will talk even of those areas using math. Rather impressive, I must say.

I had wanted to evaluate my work on whether it satisfied the mathematical theorems of Ashby and Conant. I am speaking here of the Law of Requisite Variety (Ashby) and the Good Regulator Theorem (Conant). But I could not because my math skills are not good enough for the job. My guess is you would want to evaluate your work as well on its alignment with the basics of those works. Ashby's work is considered a classic in understanding the functioning of all systems, but on glancing through his works, it is clear that he did his work primarily with the human brain in mind.

Natesh Ganesh

Hi Willy,

Thank you for the kind comments. Yes, I did not have space to treat questions like qualia and meaning in detail but I intend to in the near future. I am an engineer and my immediate focus is to leverage the ideas presented here into actual computing systems. The math was a necessary evil and I will work on making the explanations are lot clearer going forward. I find the ability to address such topics with math very exciting.

I like that you brought up both Ashby and Conant, two people whose work has been very influential on me. The minimal dissipation hypothesis is connected to the Good Regulator Theorem in a straight forward manner, and is something I have thought about and worked on. I will have to think a little bit more about rigorously showing the connection to the law of requisite variety but my initial feeling is that it is not impossible.

Thanks again for your comments. Please rate if you enjoyed the essay. It looks like I could use the help.

Cheers

Natesh

Natesh Ganesh

Hi Lee,

Apologies for the late reply. Have been away at a conference to talk about some of the ideas here and how to relate it to computing.

"Or, is more than this required to understand your hypothesis-- more than just the above statement of your hypothesis together with definitions of the terms used in your hypothesis?"

--> All that is needed to understand my hypothesis is that statement. I have provided as many definitions as I can there in the essay but due to space limitations I have had to reference some of the other definitions in former papers.

"Is what you wrote, above, about probability-learning foraging fish implied by the definitions of terms in your hypothesis? That is, do the definitions of the terms in your hypothesis (like "open," "constraints," etc.) imply what you have written, above?"

--> Yes. It does.

pjackson

Natesh,

Great essay, well informed and organised analysis of a very interesting hypothesis.

I also like most and agree with much, certainly with the addage that; 'our aims and goals are shaped by our history', and the importance of; efficiency, neuronal avalanches, branching parameters, critical regions and that a 'hierarchical predictive coding' model is possible.

I'm not yet convinced that minimal dissipation itself is a precondition of learning and 'inference dynamics' and didn't feel you proved that. Do you not think it might be more a ubiquitous characteristic than a driver - in a similar way to 'sum over paths'? If 3 lifeguards head off to save a drowning girl 100m down the beach, one heads straight for her, one to the shore point opposite her position, and one at the right angle to allow for swimming slower than running; Of course the 3rd gets there with least energy, but I feel that underlying that may be a still greater and more useful truth and meaning.

Also might the error/feedback mechanism be better described as iterative value judgement comparisons. Perhaps no 'right or wrong' just different outcomes, which we can't value till compared with previous runs.

Say the first 'run' of consequences is imaginative drawn from input history. Say Do I want a PhD Y/N? gives an 'AIM' (Y1). We run a scenario to imagine it and implications. We then keep running it as more data comes in. Subsequent lower level/consequential Y/N decisions are the same, taking Y1 into the loop, an on hierarchically. If it turns out not to be as envisaged, or we win millions and want to be a playboy instead, we change the aim to N and form new ones.

May it be that you're a little too enamored by the formula and suggest conclusions from those rather than the deeper meaning they're abstracted from.

i.e. Might Lifeguard 3 have said 'how do I get there fastest' rather than '..by using least energy'? (or just done that due to it's successful outcome in past re-runs of the scenario).

Lastly on 'Agency', which I see as a semantic 'shroud'. If all subsequent Y/N decisions in a cascade are simply consequential on the first Y/N decision, and the branches lead to a mechanical motor neuron or chemical response, all repeated in iterative loops, might the concept 'agency' not fade away with most of the mystery?

All 'leading edge' questions I think and thank you for a brilliant essay leading to them. Top mark coming.

Best

Peter

Natesh Ganesh

Dear Peter,

Thank you for your kind comments. Much appreciated.

"I also like most and agree with much, certainly with the addage that; 'our aims and goals are shaped by our history', and the importance of; efficiency, neuronal avalanches, branching parameters, critical regions and that a 'hierarchical predictive coding' model is possible."

--> Agreed.

"I'm not yet convinced that minimal dissipation itself is a precondition of learning and 'inference dynamics' and didn't feel you proved that. Do you not think it might be more a ubiquitous characteristic than a driver - in a similar way to 'sum over paths'?"

--> I started by calling it the minimal dissipation hypothesis because of the optimization principles I was using to minimize the dissipation terms. This is an evolving idea, and after the feedback I have received I am wondering if I should instead talked about learning as a manifestation of dissipation-complexity trade offs and not just minimizing dissipation, since it is a little confusing when stated that way. I will have to think about this in more detail and will get back to you with an answer.

"Also might the error/feedback mechanism be better described as iterative value judgement comparisons. Perhaps no 'right or wrong' just different outcomes, which we can't value till compared with previous runs."

--> I had not thought about it that way but it sounds very interesting. Let me mull over it.

"Lastly on 'Agency', which I see as a semantic 'shroud'. If all subsequent Y/N decisions in a cascade are simply consequential on the first Y/N decision, and the branches lead to a mechanical motor neuron or chemical response, all repeated in iterative loops, might the concept 'agency' not fade away with most of the mystery?"

--> To a certain extent, I think it does but there is also significant role noise will play in affecting decisions, as well symmetry breaking at critical bifurcation points. I do think that 'agency' in a very a traditional sense is an illusion, something I would have talked about more in detail if I had the space. I would recommend reading Alan Kadin's submission "No ghost in the machine", in which he goes into detail about some of these ideas about agency being an 'illusion', that I did not have the chance to. It is a very interesting read.

Thanks again for your comments. I will get back to you on some of these once I have thought about a detailed response.

Natesh