File 025960

Images

Text

From: Joscha Bach
Sent: 7/24/2016 5:23:46 AM
To: jeffrey E. [jeeyacation@gmail.com]
Subject: Re: Mechanisms for learning
Importance: High
Average humans are probably not fully able to learn how to program etc. Fraction of people that reshape the
world are at the right end of the Bell curves, so difference in variation means fewer women. We tend to be more
interested in the outliers. Have people explicitly evolved to be not to be too smart on average, or is general
intelligence so hard to get from human brains?
Am 24.07.2016 um 00:20 schrieb jeffrey E. <jeevacation@gmail.com>:
the average is the only thing that matters
On Sat, Jul 23, 2016 at 6:02 PM, Joscha Bach <
> wrote:
Am 23.07.2016 um 04:41 schrieb jeffrey E. <jeevacation@gmail.com>:
Cells seem to be mostly indistinguishable (except for )04 I thought?
Size is different on average, but males with small brains are still male.
Variation of all traits is higher for males, so there are more high IQ men (wider bell-curve). Given average
brain size difference, the small difference in average IQ is surprising, I think
gender differenece. unlikely motivational, every cell is different. size is didfferent. .
the more radical idea is that brains are the nucleus of the human" cell: the body the membrane, the
organizm the colletcions and interactions of brains, wisdom of crowds . mirroring. etc. . money used
as signals. as is spoken language. . what does social behavior mean, it is the cooridiation of indiv
brains. =
I think that mathematics based thinking does not allow for censorship. money people concepts if they are
all math , then poliiical correctness and their structures are only subsets of a larger group of thoughts. its
not emotinal for me.
most people have censorship built in at a deep level, social reasoning gets to different results than equivalent
abstract reasoning
On Sat, Jul 23, 2016 at 3:02 AM, Joscha Bach < H- wrote:
Some thoughts I meant to send back for a long time:
HOUSE OVERSIGHT 025960
no worry, if i understand correctly you are suggsting there are layers 1 through N. lets cal them L 1 -
Ln. there are times T 1- Tn / and then conjecture that changing the time . correlations ( by genetic swtch
or other method. , you might be able to make blacks smarter by changing the time for motor layer
development and changing the time for other layers. . ). like telemeres for the cell , are their equivalents for
the layers. . as you talked about culling the unused neurons in each layer, each neuron in each layer would
get dfferent ( kill yourself if you are not being used instructions ).
Exactly. I looked up the statistics, black kids in the US have slower cognitive development (and never catch
up), which the study of course attributed to social factors without any evidence, and they had faster motor
development! I suspect this means their brains are slower at learning high-level concepts, because the low-
level structures are optimized for a shorter time. But they will keep the lead in motor development, because it
is easier to learn, and they have more time and attention to practice once they get the structures in place.
It could also be that they have an additional set of learning directives in place that adapts them better to a
more hunting/running style of life, whereas the Europeans had to adapt for identifying long-term seasonal
patterns, delayed gratification for agriculture etc.
and CONCEPTS could be layer to layer communicaiton. . are gender differences also a matter of time,
and structure of interactions. .
I suspect gender differences are mostly motivational, i.e. we have a reward system for all the different social
and cognitive needs, which makes us receive different kinds of pleasure and pain, thereby pay attention and
learn. You cannot learn what does not attract your attention. Women tend to find abstract systems, conflicts
and mechanisms intrinsically boring. Most women in computer science do not write programs because they
enjoy solving puzzles, but because they want to help people, get approval etc. There are almost no women in
math, because it does not help people or yield social attention.
Men tend to find elaborate social relations boring. If there is no pleasure in observing and empathizing with
people, one will not have good social cognition.
IQ is not the only meaningful difference. Chinese pay an inordinate amount of attention to authority. I suspect
historically., the authorities tended to kill them a lot if they did not. Jews tend be intellectually independent
and anti-authoritarian, which might make them creative and inventive in ways that are hard to find in Asia.
At the moment, I think speed/quality of brain development plus motivational system are the key to
understanding both mind and individual differences. Important part of human language might be the result in
a motivational need to discover/invent grammatical structure, which as a side effect makes us interested in
music.
It would mean that Chomsky's life long hypothesis, that people have a special circuit for grammatical
language, is wrong. They might be GENERAL learners, just like Gorillas, but with a strong motivational urge
to build grammatical structure, which the brain simply invents (there are only a few ways in which a natural
language can work). That is much easier to wire into a brain than a specific circuit.
i my self believe that african music, westrn music , chinese music are a window into the the structure of
those layers. . western rigid, african primitve , chinese nature based, . I belive that synphoic movments
best describe your layering, they open with a basic simple melody„ after it is learned, it is repeated and"
developed" , inverted, distorted, related, etc. the development stage take a long time. then there is a
HOUSE OVERSIGHT 025961
recapitlaltion of the whole. and its inter and intra actions. . african musci has lots of beats. and little
development. - no accident, it mirrors their learning process.
Interesting question if music is somehow indicative of genetically defined prefs, but I am not sure. It could be
path effects, starting in culture.
re taboo
, maybe climate change is a good way of dealing with overpopulation. . the earths forest fire. potentailly
a good thing for the species
Making having children expensive in terms of upbringing and missed opportunity (like in the west) is a more
humane way. Environmental stress while leaving near rock bottom tends to lead people to have more
children, because there is no missed opportunity, and high mortality requires more attempts at procreation.
Humans are a hardy species, outside of focused famine events and wars only small fractions of any given
population die.
I suspect that strong reductions in population will come from large-scale failure of agriculture. The climate
change itself with result in migration and wars, but most people will probably survive that. But who knows, I
might be wrong.
too many people, so many mass executions of the elderly and infirm make sense
is the fundamental fact that everyone dies at some time .make it imporrisbole to ask so why not earilier. if
the brain discards unused neurons, why shold socieity keep their equivalent
The radical idea of treating individuals in a society as cells and the society itself as a well-organized organism
is fascism, or course. Probably the most efficient and rationally stringent way of governance, if someone
could pull it off in a sustainable way; and if it is aggressive and expansive, its efficiency makes it a virus that
everybody will want to stomp out. Fascism makes romantic doo-gooders like me very uncomfortable (I
visited KZ Buchenwald five times and it had a profound influence on me; we East Germans inoculated
ourselves very thoroughly against fascism), and the general public will not be willing to consider it.
I rather like the treatment Fascism gets in the Amazon Series "The Man in the High Castle", which explores
what would have happened if the Germans and Japanese had won the war: A society that tries to function as a
brutal and ruthlessly efficient machine, eliminating all social and evolutionary slack. It is very dark, but not a
flat caricature of pointless evil for its own sake. Heinlein's late book "Starship Troopers" explores fascism,
too, but unlike Philipp K Dick he does not see it as a form of insanity, but as the most desirable order.
I find your "political incorrectness" very fascinating. In the beginning, I thought it is a form of costly
signaling, but now I think you are simply entirely unconstrained in your thoughts. How did you manage in
your youth? Did you get in trouble, or did you keep your thoughts to yourself? I wonder what kind of person
you want to transform into.
It was interesting to notice that at the Forbidden Research conference, nobody managed to say anything
remotely out-of line. One large discussion group wanted to address the question of whether "democracy still
works", and mostly expressed their disagreement with Trump. Ideology is like halitosis: easy to see in others,
hard in oneself A speaker felt that the media "stifle all criticism of Trump", another wanted to remove "men
and Elon Musk from government", and everybody strongly agreed that we need more diversity everywhere.
I noticed some time ago that Joi has remarkable public communication skills. He picks controversial, insight-
laden topics, but sanitizes them by carefully replacing the parts of content that would divide his audience with
HOUSE OVERSIGHT 025962
symbolic messages that everybody can fill with their own content in a way that resonates with them. The non-
controversial parts will still be insightful. He manages to come across as very subversive, while rarely
offending anyone (except the hard scientists, that miss hard substance).
He also asks influential people and smart students or faculty to write parts of his essays and speeches for him.
This invests them in his success, especially because he is going to reward and acknowledge them. Very few of
his ideas are original, instead he is good at identifying and testing thoughts he reads or hears from others.
I am still beset by the ruinous instinct that the goal of communication ought to be mutual understanding. Joi is
right. Public communication is about reaching one's goals.
Bests,
Joscha
On Sun, Jul 10, 2016 at 12:42 AM, Joscha Bach <______________________H wrote:
Dear Jeffrey,
thank you for your support and encouragement, even where I fail.
Sorry for being such an embarrassment today. I will spell out today's argument a bit better and cohesive
when I get to it. Also, I should have recognized that the main point I tried to make would trigger Noam (who
was as always very generous, patient, kind and humble on the personal level, even though he did not feel like
conceding anything on the conceptual one). Almost all of Noam's work focused on the idea that humans have
very specific circuits or modules (even when most people in his field began to have other ideas), and his
frustration is that it is so hard to find or explain them.
I found Noam's hypothesis very compelling in the past. I still think that the idea that language is somehow a
cultural or social invention of our species is wrong. But I think that there is a chance (we don't know that, but
it seems to most promising hypothesis IMHO) that the difference between humans and apes is not a very
intricate special circuit, but genetically simple developmental switches. The bootstrapping of cognition
works layer by layer during the first 20 years of our life. Each layer takes between a few months and a few
years to train in humans. While a layer is learned, there is not much going on in the higher layers yet, and
after the low level learning is finished, it does not change very much. This leads to the characteristic bursts in
child development, that have famously been described by Piaget.
The first few layers are simple perceptual stuff, the last ones learn social structure and self-in-society. The
switching works with something like a genetic clock, very slowly in humans, but much more quickly in other
apes, and very fast in small mammals. As a result, human children take nine months before their brains are
mature enough to crawl, and more than a year before they can walk. Many African populations are quite a bit
faster. In the US, black children outperform white children in motor development, even in very poor and
socially disadvantaged households, but they lag behind (and never catch up) in cognitive development even
after controlling for family income.
Gorillas can crawl after 2 months, and build their own nests after 2.5 years. They leave their mothers at 3-4
years. Human children are pretty much useless during the first 10-12 years, but during each phase, their
brains have the opportunity to encounter many times as much training data as a gorilla brain. Humans are
HOUSE OVERSIGHT 025963
literally smarter on every level, and because the abilities of the higher levels depend on those of the lower
levels, they can perform abstractions that mature gorillas will never learn, no matter how much we try to
train them.
The second set of mechanisms is in the motivational system. Motivation tells the brain what to pay attention
to, by giving reward and punishment. If a brain does not get much reward for solving puzzles, the individual
will find mathematics very boring and won't learn much of it. If a brain gets lots of rewards for discovering
other people's intentions, it will learn a lot of social cognition.
Language might be the result of three things that are different in humans:
- extended training periods per layer (after the respective layer is done, it is difficult to learn a new set of
phonemes or the first language)
- more layers
- different internal rewards. Perhaps the reward for learning grammatical structure is the same that makes us
like music. Our brains may enjoy learning compositional regular structure, and they enjoy making
themselves understood, and everything else is something the universal cortical learning figures out on its
own.
This is a hypothesis that is shared by a growing number of people these days. In humans, it is reflected for
instance by the fact that races with faster motor development have lower IQ. (In individuals of the same
group, slower development often indicates defects, of course.)
Another support comes from machine learning: we find that the same learning functions can learn visual and
auditory pattern recognition, and even end-to-end-learning. Google has built automatic image recognition
into their current photo app:
http://blogs.wsj . com/digits/2015/07/01/go o gle-mistakenly-tags-black-p eop le-as-gorillas-showing-limits-o f-
algorithms/
The state of the art in research can do better than that: it can begin to "imagine" things. I.e. when the
experimenter asks the system to "dream" what a certain object looks like, the system can produce a
somewhat compelling image, which indicates that it is indeed learning visual structure. This stuff is
something nobody could do a few months ago:
http://www.creativeai.net/posts/Mv4WG6rdzAerZF7ch/synthesizing-preferred-inputs-via-deep-generator-
networks
A machine learning program that can learn how to play an Atari game without any human supervision or
hand-crafted engineering (the feat that gave DeepMind 500M from Google) now just takes about 130 lines of
Python code.
These models do not have interesting motivational systems, and a relatively simple architecture. They
currently seem to mimic some of the stuff that goes on in the first few layers of the cortex. They learn object
features, visual styles, lighting and rotation in 3d, and simple action policies. Almost everything else is
missing. But there is a lot of enthusiasm that the field might be on the right track, and that we can learn
motor simulations and intuitive physics soon. (The majority of the people in Al do not work on this,
however. They try to improve the performance for the current benchmarks.)
Noam's criticism of machine translation mostly applies to the Latent Semantic Analysis models that Google
and others have been using for many years. These models map linguistic symbols to concepts, and relate
concepts to each other, but they do not relate the concepts to "proper" mental representations of what objects
and processes look like and how they interact. Concepts are probably one of the top layers of the learning
hierarchy, i.e. they are acquired *after* we learn to simulate a mental world, not before. Classical linguists
ignored the simulation of a mental world entirely.
HOUSE OVERSIGHT 025964
It seems miraculous that purely conceptual machine translation works at all, but that is because concepts are
shared between speakers, so the structure of the conceptual space can be inferred from the statistics of
language use. But the statistics of language use have too little information to infer what objects look like and
how they interact.
My own original ideas concern a few parts of the emerging understanding of what the brain does. The
"request-confirmation networks" that I have introduced at a NIPS workshop in last the December are an
attempt at modeling how the higher layers might self-organize into cognitive programs.
Cheers!
Joscha
please note
The information contained in this communication is
confidential, may be attorney-client privileged, may
constitute inside information, and is intended only for
the use of the addressee. It is the property of
JEE
Unauthorized use, disclosure or copying of this
communication or any part thereof is strictly prohibited
and may be unlawful. If you have received this
communication in error, please notify us immediately by
return e-mail or by e-mail to jeevacation@gmail.com, and
destroy this communication and all copies thereof,
including all attachments. copyright -all rights reserved
please note
The information contained in this communication is
confidential, may be attorney-client privileged, may
constitute inside information, and is intended only for
the use of the addressee. It is the property of
JEE
Unauthorized use, disclosure or copying of this
communication or any part thereof is strictly prohibited
and may be unlawful. If you have received this
communication in error, please notify us immediately by
return e-mail or by e-mail to jeevacation@gmail.com, and
destroy this communication and all copies thereof,
including all attachments. copyright -all rights reserved
HOUSE OVERSIGHT 025965
please note
The information contained in this communication is
confidential, may be attorney-client privileged, may
constitute inside information, and is intended only for
the use of the addressee. It is the property of
JEE
Unauthorized use, disclosure or copying of this
communication or any part thereof is strictly prohibited
and may be unlawful. If you have received this
communication in error, please notify us immediately by
return e-mail or by e-mail to jeevacation@gmail.com, and
destroy this communication and all copies thereof,
including all attachments. copyright -all rights reserved
HOUSE OVERSIGHT 025966