SyNAPSE Project AI Neuromorphic Chip

IBM’s Aug 18th Press Release announced another significant milestone for the DARPA SyNAPSE project, the world’s best funded and arguably the “most likely to succeed” approach to creating a general artificial intelligence.

The release notes that the new chips represent a departure from traditional models of computing:

…. cognitive computers are expected to learn through experiences, find correlations, create hypotheses, and remember – and learn from – the outcomes, mimicking the brains structural and synaptic plasticity.

To do this, IBM is combining principles from nanoscience, neuroscience and supercomputing as part of a multi-year cognitive computing initiative. The company and its university collaborators also announced they have been awarded approximately $21 million in new funding from the Defense Advanced Research Projects Agency (DARPA) for Phase 2 of the Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE) project.

As we’ve noted here many times, another remarkable project is the Blue Brain Project in Europe spearheaded by Dr. Henry Markram.     That team has joined with many others and is in the process of applying to the European Union for substantial funding – perhaps as much as 1.6 billion dollars.    Although Blue Brain tends to shy away from stating that their objective is a general artificial intelligence,  I would argue that they should have that goal and also that they are much more likely to be funded by stating that goal in no uncertain terms.

Unfortunately there remain many both in and outside of technology circles who believe the search for a general artificial intelligence is either dangerous or a waste of time and money.   Both these scenarios are possible but unlikely.   Sure, intelligence can be dangerous but given human history compared to technology history it seems odd to argue that we are more likely to create a Frankenstein than a helpful machine process.    Computers don’t kill people, people kill people.

In terms of a waste of time and money, clearly we humans have overrated our intelligence for some time – probably since the beginning of self-awareness.   There are few rational reasons to reject the idea that we cannot duplicate processes that are similar to our own thinking in a machine.   The advantages of machine based intelligence are likely to be  substantial – probably on the order of a new human age with vastly improved resource efficiency, poverty reduction, and more.  Thus the costs – currently measured in the low tens of millions – pale in comparison to almost all other government projects – many with massively dubious and negative ROIs.

http://cacm.acm.org/magazines/2011/8/114944-cognitive-computing/fulltext

One of the paper’s most notable items asserts that within a decade the project expects to have the computational scale needed for human level modelling, though it also notes that this is not the same as creating a model of the human brain – this may require computational structures yet to be invented.    However on balance it would seem the SyNAPSE project continues to build on their core assumptions, taking us ever closer to the holy grail of technology – a general artificial intelligence.

More at Dr. Modha’s blog , where we learn more about the new approaches the SyNAPSE team at IBM will take in an effort to achieve human quality cognition in a machine:

18 Aug 2011: Today, IBM (NYSE: IBM) researchers unveiled a new generation of experimental computer chips designed to emulate the brain’s abilities for perception, action and cognition. The technology could yield many orders of magnitude less power consumption and space than used in today’s computers.

In a sharp departure from traditional concepts in designing and building computers, IBM’s first neurosynaptic computing chips recreate the phenomena between spiking neurons and synapses in biological systems, such as the brain, through advanced algorithms and silicon circuitry. Its first two prototype chips have already been fabricated and are currently undergoing testing.

Called cognitive computers, systems built with these chips won’t be programmed the same way traditional computers are today. Rather, cognitive computers are expected to learn through experiences, find correlations, create hypotheses, and remember – and learn from – the outcomes, mimicking the brains structural and synaptic plasticity.

How hard is it to build an intelligent machine? I don’t think it’s so hard ….   The basic idea I promote is that you mustn’t look for a magic bullet. You mustn’t look for one wonderful way to solve all problems. Instead you want to look for 20 or 30 ways to solve different kinds of problems. And to build some kind of higher administrative device that figures out what kind of problem you have and what method to use.

Now, if you take any particular researcher today, it’s very unlikely that that researcher is going to work on this architectural level of what the thinking machine should be like. Instead a typical researcher says, “I have a new way to use statistics to solve all problems.” Or: “I have a new way to make a system that imitates evolution. It does trials and finds the things that work and remembers the things that don’t and gets better that way.” And another one says, “It’s going to use formal logic and reasoning of a certain kind, and it will figure out everything.” So each researcher today is likely to have one particular idea, and that researcher is trying to show that he or she can make a machine that will solve all problems in that way.

I think this is a disease that has spread through my profession. Each practitioner thinks there’s one magic way to get a machine to be smart, and so they’re all wasting their time in a sense.

I was surprised to see his lack of optimism in the face of so much progress in areas I’d argue are very generalized indeed.     The DARPA  SyNAPSE project we’ve discussed several times here at Technology Report remains the best funded AI research to date, and lead researchers seem to feel optimistic that progress there could lead to a human scale general intelligence within several years rather than several decades that Minsky implies may be required given the new approaches.

Simply put, DARPA SyNAPSE  is creating a computing infrastructure to rival the human brain in terms of connectivity, and counting on the possibility that we are dealing mostly with *quantity of connections* rather than *quality of connections* when we talk about human level intelligence.

The other very promising project for generalized AI is somewhat at odds with the DARPA SyNAPSE view.    The Blue Brain project is also a promising development ground for general artificial intelligence, but the approach is very different as described by Dr. Henry Markram, the project manager at Blue Brain.      The Blue Brain team is focusing more on “reverse engineering” animal brains and eventually a human brain.

Given the new level of enthusiasm and funding from DARPA, it seems likely that progress will continue at a faster pace that at anytime in the past.

Ironically I think Minsky’s early optimism in the 1950s  was more justified than his current pessimism, though his observation that academics are working in too much isolation is certainly true.    I’m often surprised how many technologists don’t seem to understand many simple aspects of human biology and evolution and vica versa.     Human intelligence, though intriguing, continues to be overrated as an phenomenon of exceptional quality.    We’re a somewhat arrogant creature by evolutionary design, but that does not justify our self importance.    Machines surpass most of us in most compartmentalized aspects of intelligence and many aspects of creativity  (mathematics / translation and language / game playing / music / information retrival, etc, etc).    It seems reasonable that what we call “consciousness” may only require massive connectivity – perhaps something as simple as creating a fast, multitasked conversation between different parts of an artificial brain.

This is the article, about a new chip called a “Memristor”  that seemed to spawn a lot of discussion.   It’s a new approach under development as part of the DARPA SyNAPSE project.  [thanks to my good pal Roy K who always forwards me very interesting stuff!]

It’s encouraging to see the debate over computer consciousness take such a serious tone as this topic is arguably one of the most intriguing in all of human history.    Many experts believe we’ll see machines become self-aware within 10-20 years.    Given the massive computational superiority computers already enjoy over humans, one can make a strong case that a conscious computer – or more likely a computer + human brain hybrid – will  clone its intellect and improve that intellect over and over in a very short time, leading to levels of intelligence far beyond that of “normal” humans.

Also encouraging that pioneers in the field like the article’s authors:  Sean Lorenz, Heather Ames & Massimiliano Versace are willing to discuss this topic rather than shelve it as so many in computer research have done.    I think early inflated optimism about artificial intelligence led to so much disappointment in the computer community that the “old guard” programmers are being too stubborn now, especially in the face of very significant advances in the understanding of human neurobiology and in computational speeds and memory capacities.

From the post “ Moneta_And_The_C_Word”

Conclusion
Although Searle has argued for biological embodiment as a necessity for causation of consciousness, this paper puts forth the argument that biological embodiment is not the only embodiment that can produce consciousness. Instead, we argue that the brain is an optimal form of embodiment giving rise to consciousness because it can produce observable reports, oral reports, and observed and measured activity. The first two qualifications of consciousness can be replicated with computer simulations as discussed by the proponents of WBE. However, the third qualification requires a unique stipulation for embodiment that is able to self organize and generate unique global and local patterns of activities within its constituent elements. At this point in time, this is only achievable within brain tissue. However, with the advancement of neural chip development, we would argue that embodiment necessary for consciousness would be achievable in a new medium, the neuromorphic chip.

The project leader of Blue Brain, Dr. Henry Markram, has been very vocal and very critical of claims by  the IBM team leader Dr. Dharmendra S. Modha.    Markram’s concerns are expressed here in his Technology Report guest post about the IBM project claims.

We asked Dr. Modha for a response but didn’t hear back, so I’d like to refer folks to the Modha blog here, especially to the post called ‘The Cat is Out of the bag and BlueMatter”, which details progress in the SyNAPSE project and explains the claims made that they are simulating brain activity that is roughly equivalent to that we’d see from a cat.  Here’s an excerpt from that post:

Towards this end, we are announcing two major milestones.

First, using Dawn Blue Gene / P supercomputer at Lawrence Livermore National Lab with 147,456 processors and 144 TB of main memory, we achieved a simulation with 1 billion spiking neurons and 10 trillion individual learning synapses. This is equivalent to 1,000 cognitive computing chips each with 1 million neurons and 10 billion synapses, and exceeds the scale of cat cerebral cortex. The simulation ran 100 to 1,000 times slower than real-time.

Second, we have developed a new algorithm, BlueMatter, that exploits the Blue Gene supercomputing architecture to noninvasively measure and map the connections between all cortical and sub-cortical locations within the human brain using magnetic resonance diffusion weighted imaging. Mapping the wiring diagram of the brain is crucial to untangling its vast communication network and understanding how it represents and processes information.

Finally, here is an excellent presentation by Dr. Modha that outlines in simple terms what they are trying to do with DARPA SyNAPSE, which is build a human scale brain by 2018.

——   Guest Post by Dr. Henry Markram of the Blue Brain Project —-

IBM’s claim is a HOAX.

This is a mega public relations stunt – a clear case of scientific deception of the public. These simulations do not even come close to the complexity of an ant, let alone that of a cat. IBM allows Mohda to mislead the public into believing that they have simulated a brain with the complexity of a cat – sheer nonsense.

Here are the scientific reasons why this is a hoax and misleading PR stunt:

How complex is their model?
They claim to have simulated over a billion neurons interacting. Their so called “neurons” are the tiniest of points you can imagine, a microscopic dot. Over 98% of the volume of a neuron is branches (like a tree). They just cut off all the branches and roots and took a point in the middle of the trunk to represent a entire neuron. In real life, each segment of the branches of a neuron contains dozens of ion channels that powerfully controls the information processing in a neuron. They have none of that. Neurons contain 10′s of thousands of proteins that form a network with 10′s of millions of interactions. These interactions are incredibly complex and will require solving millions of differential equations. They have none of that. Neurons contain around 20’000 genes that produce products called mRNA, which builds the proteins. The way neurons build proteins and transport them to all the corners of the neuron where they are needed is an even more complex process which also controls what a neuron is, its memories and how it will process information. They have none of that. They use an alpha function (up fast down slow) to simulate a synaptic event. This is a completely inaccurate representation of a synapse. There are at least 6 types of synapses that are highly non-linear in their transmission (i.e. that transform inputs and not only transmit inputs). In fact you would need a 10′s of thousands of differential equations to simulate one synapse. Synapses are also extremely complex molecular machines that would themselves require thousands of differential equations to simulate just one. They simulated none of this. There are complex differential equations that must be solved to simulate the ionic flow in the branches, to simulate the ion channels biophysics, the protein-protein interactions, as well as the complete biochemical and genetic machinery as well as the synaptic transmission between neurons. 100′s of thousands of more differential equations. They have none of this. Then there are glia – 10 times more than neurons..And the blood supply…and more and more. These “points” they simulated and the synapses that they use for communication are literally millions of times simpler than a real cat brain. So they have not even simulated a cat’s brain at more than one millionth of it’s complexity.

Is it nonetheless the biggest point neuron simulation ever run?
No. These people simulated 1 billion points interacting. They used a formulation to model the summing up and threshold spiking of the “points” called the Izhikevik Formulation (an extremely simple equation). Eugene Izhikevik himself already in 2005 ran a simulation with 100 billion such points interacting just for the fun of it: (over 60 times larger than Modha’s simulation). This simulation ran on a cluster of desktop PCs and which every graduate student can run This is no technical achievement and certainly not even a record number of point neurons. That model exhibited oscillations, but that always happens so even simulating 100 Billion such points interacting is light years away from a brain.
see: http://www.izhikevich.org/human_brain_simulation/Blue_Brain.htm#Simulation%20of%20Large-Scale%20Brain%20Models

Is the simulator they built a big step?
Not even close. There are numerous proprietary and peer-reviewed neurosimulators (e.g., NCS, pNEURON, SPLIT, NEST) out there that can handle very large parallel models that are essentially only bound by the available memory. The bigger the machine you have available, the more neurons you can simulate. All these simulators apply optimizations for the particular platform in order to make optimal use of the available hardware. Without any comparison to existing simulators, their publication is a non-peer reviewed claim.

Did they learn anything about the brain?
They got very excited because they saw oscillations. Oscillations are an obligatory artifact that one always gets when many points interact. These findings that they claim on the neuroscience side may excite engineers, but not neuroscientists.

Why did they get the Gordon Bell Prize?
They submitted a non-peer reviewed paper to the Gordon Bell Committee and were awarded the prize almost instantly after they made their press release. They seem to have been very successful in influencing the committee with their claim, which technically is not peer-reviewed by the respective community and is neuroscientifically outrageous.

But is there any innovation here?
The only innovation here is that IBM has built a large supercomputer – which is irrelevant to the press release.

Why did IBM let Mohda make such a deceptive claim to the public?
I don’t know. Perhaps this is a publicity stunt to promote their supercompter. The supercomputer industry is suffering from the financial crisis and they probably are desperate to boost their sales. It is so disappointing to see this truly great company allow the deception of the public on such a grand scale.

But have you not said you can simulate the Human brain in 10 years?
I am a biologist and neuroscientist that has studied the brain for 30 years.  I know how complex it is. I believe that with the right resources and the right strategy it is possible. We have so far only simulated a small part of the brain at the cellular level of a rodent and I have always been clear about that.

Would other neuroscientists agree with you?
There is no neuroscientist on earth that would agree that they came even close to simulating the cat’s brain – or any brain.

But did Mohda not collaborate with neuroscientists?
I would be very surprised if any neuroscientists that he may have had in his DARPA consortium realized he was going to make such an outrages claim. I can’t imagine that that the San Fransisco neuroscientists knew he was going to make such a stupid claim. Modha himself is a software engineer with no knowledge of the brain.

But did you not collaborate with IBM?
I was collaborating with IBM on the Blue Brain Project at the very beginning because they had the best available technology to faithfully allow us to integrate the diversity and complexity found in brain tissue into a model. This for me is a major endeavor to advance our insights into the brain and drug development. Two years ago, when the same Dharmendra Mhoda claimed the “mouse-scale simulations”, I cut all neuroscience collaboration with IBM because this is an unethical claim and it deceives the public.

What IBM allowed Modha to do here is not only wrong, but outrageous. They deceived millions of people.

Henry Markram
Blue Brain Project

At the Supercomputing Conference SC09 in Portland Oregon IBM has announced a spectacular advance in our ability to mechanically simulate cognitive activity with machines – they have developed a brain simulation that approximates a cat brain in complexity.

We have profiled the SyNAPSE project here at Technology Report thanks to a guest post by one of those working there. This new development is a remarkable advance given that SyNAPSE has been going strong for under one year. With a cat brain complexity under its belt it appears only a matter of a few more years before the project is likely to have modeled interactions at the scale of human brain complexity.

The most provocative idea about brain modelling is that these models will at some point attain human-like consciousness along with the ability to communicate with humans and (hopefully) cooperate with us in problem solving. No longer just a science fiction topic, this potential “explosion of intelligence” relates to one of the hottest topics in technology – the Singularity.

More on the IBM Blue Matter project from:

Forbes
Popular Mechanics

It is somewhat tempting to think like a conspiracy theory buff and suggest that the Biologically-Inspired Cognitive Architectures project “BICA” – a major effort to create artificial intelligence – has succeeded and gone off the record rather than been cancelled by the US Government.

However the idea that BICA has simply been “cancelled” in favor of newer approaches seems far more likely, especially given the new focus of the DARPA SyNAPSE project we’ve discussed here at Technology Report several times before.    It appears that the more general and decentralized approach of BICA has been replaced with a more collaborative and engineered approach taken in the DARPA SyNAPSE project.

The Defense Advanced Research Projects Agency (DARPA) is one of the world’s best funded advanced technology research groups. DARPA’s most impressive accomplishment to date has been to fund the prizes that have inspired several university groups to create fully autonomous vehicles that can navigate both city traffic and complex off the road tracks without any human controls.

BICA Project:
http://www.darpa.mil/IPTO/programs/bica/bica_phase1.asp

DARPA SyNAPSE Project: ]]> http://technology-report.com/2009/08/us-military-killed-the-biologically-inspired-cognitive-architectures-bica-project-without-explanation/feed/ 0 http://technology-report.com/2009/07/darpa-synapse-project-summary/ http://technology-report.com/2009/07/darpa-synapse-project-summary/#comments Fri, 24 Jul 2009 04:13:06 +0000 Joe Hunkins http://technology-report.com/?p=1294 Today we have a guest post with permission from Max over at the “Neurdons” blog, written by a group working on the DARPA SyNAPSE project we have discussed here before.   SyNAPSE seeks to create a fully functional artificial intelligence.

This piece was written by Ben Chandler, an AI researcher with the SyNAPSE project:

About SyNAPSE