Skip to main content

Full text of "Theory of self-reproducing automata"

See other formats


DUCING 
MATA 




Commerce 



NOTICE: Return or renew all Library Materials! The Minimum Fee for 
each Lost Book is $50.00. 

The person charging this material is responsible for 
its return to the library from which it was withdrawn 
on or before the Latest Date stamped below. 

Theft mutilation, and underlining of books are reasons for discipli- 
nary action and may result in dismissal from the University. 
To renew call Telephone Center, 333-8400 

UNIVERSITY OF ILLINOIS LIBRARY AT URBANA-CHAMPAIGN 



4 

"y. 

JUL 0 7 ?93 



TO re: 

CALL: 



ME 




0 5 2009 



W BOOKS 
333-8400 



L161— O-1096 



Theory of 

Self-Reproducing Automata 



Theory of 

Self-Reproducing Automata 

JOHN VON NEUMANN 

edited and completed by Arthur W. Burks 



University of Illinois Press 
URBANA AND LONDON 1966 



© 1966 by the Board of Trustees of the University of Illinois. 
Manufactured in the United States of America. Library of 
Congress Catalog Card No. 63-7246. 



5 l o , 



CONTENTS 



Preface xv 

EDITOR'S INTRODUCTION 

Von Neumann's Work on Computers 1 

Von Neumann the Mathematician 2 

Von Neumann and Computing 2 

Logical Design of Computers 6 

Programming and Flow Diagrams 12 

Computer Circuits 15 

Von Neumann's Theory of Automata 17 

Introduction 17 

Natural and Artificial Automata 21 

Mathematics of Automata Theory 25 

PART ONE 

THEORY AND ORGANIZATION OF COMPLICATED 
AUTOMATA 

First Lecture: Computing Machines in General 31 

Second Lecture: Rigorous Theories of Control and Information. . 42 

Third Lecture: Statistical Theories of Information 57 

Fourth Lecture: The Role of High and of Extremely High 

Complication 64 

Fifth Lecture: Re-evaluation of the Problems of Complicated 

Automata — Problems of Hierarchy and Evolution 74 



PART TWO 

THE THEORY OF AUTOMATA: CONSTRUCTION, 
REPRODUCTION, HOMOGENEITY 



CHAPTER 1 

GENERAL CONSIDERATIONS 

1.1 Introduction 91 

1.1.1.1 The theory of automata 91 

1.1.1.2 The constructive method and its limitations 91 

1.1.2.1 The main questions: (A)-(E) 92 

1.1.2.2 The nature of the answers to be obtained 92 



V 



Vi THEORY OF SELF-REPRODUCING AUTOMATA 

[1.1.2.3 Von Neumann's models of self -reproduction] 93 

1.2 The Role of Logics— Question (A) 99 

1.2.1 The logical operations — neurons 99 

1.2.2 Neural vs. muscular functions 101 

1.3 The Basic Problems of Construction — Question (B) 101 

1.3.1.1 The immediate treatment, involving geometry, 

kinematics, etc 101 

1.3.1.2 The non-geometrical treatment — structure of the 

vacuum 102 

1.3.2 Stationarity — quiescent vs. active states 103 

1.3.3.1 Discrete vs. continuous framework 103 

1.3.3.2 Homogeneity: discrete (crystalline) and continuous 

(Euclidean) 103 

1.3.3.3 Questions of structure: (P)-(R) 104 

1.3.3.4 Nature of results, crystalline vs. Euclidean: state- 

ments (X)-(Z) 105 

[1.3.3.5 Homogeneity, quiescence, and self -reproduction] . . 106 

1.3.4.1 Simplification of the problems of construction by 

the treatment according to Section 1.3.1.2 108 

1.3.4.2 Quiescence vs. activity; excitability vs. unexcita- 

bility; ordinary and special stimuli 109 

1.3.4.3 Critique of the distinctions of Section 1.3.4.2. . . . 110 

1.4 General Construction Schemes — Question (B) Continued. . Ill 

1.4.1.1 Construction of cell aggregates — the built-in plan. . Ill 

1.4.1.2 The three schemes for building in multiple plans — 

the parametric form 112 

1.4.2.1 The descriptive statement L for numerical param- 

eters 112 

1.4.2.2 Applications of L 113 

1.4.2.3 Use of L as an unlimited memory for (A) 113 

1.4.2.4 Use of base two for L 114 

[1.4.2.5 The linear array L] 114 

1.5 Universal Construction Schemes — Question (C) 116 

1.5.1 Use of L for non-numerical (universal) parametri- 

zation 116 

1.5.2 The universal type of plan 116 

1.6 Self-Reproduction — Question (D) 118 

1.6.1.1 The apparent difficulty of using L in the case of 

self -reproduction 118 

1.6.1.2 Circumvention of the difficulty — the types E 

andE F 118 

1.6.2.1 First remark: shape of L 119 



CONTENTS vii 

1.6.2.2 Second remark: avoidance of collision in a single 

reproduction 120 

1.6.2.3 Third remark: analysis of the method for overcom- 

ing the difficulty of Section 1.6.1.1 — the role 

of L. . 121 

1.6.3.1 Copying: use of descriptions vs. originals 122 

[1.6.3.2 The Richard paradox and Turing machines] 123 

1.7 Various Problems of External Construction Intermediate 

Between Questions (D) and (E) 126 

1.7.1 Positioning of primary, secondary, ternary, etc 126 

1.7.2.1 Constructed automata: initial state and starting 

stimulus 127 

1.7.2.2 Single-action vs. sequential self-reproduction 128 

1.7.3 Construction, position, conflict 129 

1.7.4.1 E F and the gene-function 130 

1.7.4.2 E F and the mutation — types of mutation 130 

1.8 Evolution— Question (E) 131 

CHAPTER 2 

A SYSTEM OF 29 STATES WITH A GENERAL 
TRANSITION RULE 

2.1 Introduction 132 

2.1.1 The model: states and the transition rule 132 

2.1.2 Formalization of the spatial and the temporal rela- 

tions 132 

2.1.3 Need for a pre-formalistic discussion of the states. . 134 

2.2 Logical Functions — Ordinary Transmission States 134 

2.2.1 Logical-neuronal functions 134 

2.2.2.1 Transmission states — connecting lines 135 

2.2.2.2 Delays, corners, and turns in connecting lines. . . . 136 

2.3 Neurons— Confluent States 136 

2.3.1 The + neuron 136 

2.3.2 Confluent states: the • neuron 136 

2.3.3 The - neuron 138 

2.3.4 The split 138 

2.4 Growth Functions: Unexcitable State and Special Transmis- 

sion States 139 

2.4.1 Muscular or growth function — ordinary vs. special 

stimuli 139 

2.4.2 Unexcitable state 139 

2.4.3 Direct and reverse processes — special transmission 

states 140 



Vlll 



THEORY OF SELF-REPRODUCING AUTOMATA 



2.5 The Reverse Process 140 

2.5.1.1 The reverse process for the ordinary states 140 

2.5.1.2 The reverse process for the special states 141 

2.5.2 Origination of special stimuli 141 

2.6 The Direct Process — Sensitized States 142 

2.6.1 The direct process 142 

2.6.2.1 First remark: duality of ordinary and special 

states 142 

2.6.2.2 The need for the reverse process 143 

2.6.3.1 Second remark: the need for fixed stimulus se- 

quences to control the direct process 143 

2.6.3.2 Additional states required 145 

2.6.4 Sensitized states 145 

2.7 Even and Odd Delays 146 

2.7.1 Even delays by path differences 146 

2.7.2.1 Odd delays and single delays 146 

2.7.2.2 Single delays through the confluent states 147 

2.8 Summary 148 

2.8.1 Rigorous description of the states and of the transi- 

tion rule 148 

2.8.2 Verbal summary 150 

[2.8.3 Illustrations of the transition rule] 151 

chapter 3 
DESIGN OF SOME BASIC ORGANS 

3.1 Introduction 157 

3.1.1 Free and rigid timing, periodic repetition, and 

phase stop 157 

3.1.2 Construction of organs, simple and composite 158 

3.2 Pulsers 159 

3.2.1 The pulser: structure, dimensions, and timing. . . . 159 

3.2.2 The periodic pulser: structure, dimensions, timing, 

and the PP(T) form 162 

3.3 The Decoding Organ: Structure, Dimensions, and Timing. . 175 

3.4 The Tripl e-retur n Counter 179 

3.5 The I vs. 10101 Discriminator: Structure, Dimensions, and 

Timing 187 

3.6 The Coded Channel 190 

3.6.1 Structure, dimensions, and timing of the coded 

channel 190 

3.6.2 Cyclicity in the coded channel 198 



CONTENTS ix 
CHAPTER 4 

DESIGN OF A TAPE AND ITS CONTROL 

4.1 Introduction 201 

[4.1.1 Abstract] 201 

4.1.2 The linear array L 202 

4.1.3 The constructing unit CU and the memory con- 

trol MC 204 

4.1.4 Restatement of the postulates concerning the con- 

structing unit CU and the memory control MC . . 207 

4.1.5 The modus operandi of the memory control MC on 

the linear array L 207 

4.1.6 The connecting loop Ci 210 

4.1.7 The timing loop C 2 213 

4.2 Lengthening and Shortening Loops Ci and C 2 , and Writing 

in the Linear Array L 214 

4.2.1 Moving the connection on L 214 

4.2.2 Lengthening on L 216 

4.2.3 Shortening on L 220 

4.2.4 Altering x n in L 224 

4.3 The Memory Control MC 226 

[4.3.1 The organization and operation of MC] 226 

4.3.2 Detailed discussion of the functioning of MC. . . . 231 

4.3.3 The read-write-erase unit RWE 237 

4.3.4 The basic control organ CO in MC 243 

4.3.5 The read-write-erase control RWEC 246 

CHAPTER 5 

[AUTOMATA SELF-REPRODUCTION] 

[5.1 Completion of the Memory Control MC 251 

5.1.1 The rest of the manuscript 251 

5.1.2 Solution of the interference problem 259 

5.1.3 Logical universality of the cellular structure 265 

5.2 The Universal Constructor CU + (MC + L) 271 

5.2.1 The constructing arm 271 

5.2.2 Redesign of the memory control MC 277 

5.2.3 The constructing unit CU 279 

5.3 Conclusion 286 

5.3.1 Summary of the present work 286 

5.3.2 Self -reproducing automata] 294 



X 



THEORY OF SELF-REPRODUCING AUTOMATA 



Bibliography 297 

Figures 305 

Symbol Index 379 

Author and Subject Index 381 

TABLES 

[Table I. External characteristics of periodic pulsers] 173 

Table II. Summary of the pulse sequences sent into loops Ci 

and C 2 235 

Table III. Summary of excess delay requirements for the peri- 
odic pulsers stimulating loops Ci and C 2 236 

Table IV. The pulsers used to stimulate loops Ci and C 2 237 

[Table V. How the control organs of RWEC control the pul- 
sers and periodic pulsers of RWE] 248 

LIST OF FIGURES 

Second Lecture 

Fig. 1. Neural network in which dominance is not transitive . . . 305 

Fifth Lecture 

Fig. 2. A binary tape constructed from rigid elements 306 

Chapter 1 

Fig. 3. The basic neurons 306 

Chapter 2 

Fig. 4. The quadratic lattice 307 

(a) Nearest (O) and next nearest ( • ) neighbors of X 

(b) Unit vectors 

(c) Nearest (O) and next nearest ( • ) neighbors of X 

Fig. 5. Ordinary transmission states 308 

Fig. 6. Confluent states 310 

(a) Achievement of a • neuron by using confluent states 

(b) Achievement of a split line by using confluent states 

Fig. 7. Organization of an area 311 

Fig. 8. The need for both even and odd delays 311 

Fig. 9. The 29 states and their transition rule 312 



LIST OF FIGURES xi 

Fig. 10. Succession of states in the direct process 313 

Fig. 11. Illustrations of transmission and confluent states 314 

(a) Realization of f(i + 3) = [a(i) + b(t + 1)] 

(b) Realization of g(t + 3) = [c(t) • d(t)] 

(c) Conversion of ordinary stimuli into special stimuli by 

confluent states, and wire-branching 

Fig. 12. Illustration of the direct process 315 

Fig. 13. Examples of the reverse process 316 

(a) Special transmission state killing a confluent state 

(b) Special and ordinary transmission states killing each 

other 

(c) Killing dominates reception but not emission 

Fig. 14. Procedure for modifying a remote cell and returning the 

constructing path to its original unexcitable state. ... 317 

Chapter 8 

Fig. 15. Two pulsers 318 

Fig. 16. Design of a pulser 319 

Fig. 17. Two periodic pulsers 321 

(a) Periodic pulser PP(10010001) 

(b) Special periodic pulser PP(1) 

Fig. 18. Design of a periodic pulser: initial construction 322 

Fig. 19. Design of a periodic pulser: equalization of delays 325 

Fig. 20. Alternate periodic pulser PP( 1) 329 

Fig. 21. Decoding organ D(100100001) 329 

Fig. 22. Design of a decoder 330 

Fig. 23. Triple-return counter 3> 332 

Fig. 24. Design of triple-return counter <J> 333 

Fig. 25. 1 vs 10101 dis crimina tor ^ 335 

Fig. 26. Recognizer R(101001) 335 

Fig. 27. A coded channel 336 

Fig. 28. Arranging inputs and outputs of a coded channel 337 

Fig. 29. Pulsers and decoders of a coded channel 340 

Fig. 30. Cyclicity in a coded channel 341 

Chapter 4 

Fig. 31. The linear array L, the connecting loop Q, and the timing 

loopC 2 342 

Fig. 32. Lengthening the timing loop C2 345 

Fig. 33. Lengthening the connecting loop Q 346 

Fig. 34. Shortening the timing loop C2 348 



xii 



THEORY OF SELF- REPRODUCING AUTOMATA 



Fig. 35. Shortening the connecting loop d 350 

Fig. 36. Writing "zero" in cell x n of the linear array L when 

lengthening Ci 352 

Fig. 37. Tape unit with unlimited memory capacity 353 

Fig. 38. The logical structure of the procedure followed by the 

memory control MC 354 

Fig. 39. Read-write-erase unit RWE (Note: parts (a) and (b) 

constitute one continuous drawing 355 

(a) Upper part of RWE 

(b) Lower part of RWE 

Fig. 40. Control organ CO 357 

Fig. 41. Read-write-erase control RWEC (Note: parts (a)-(f) 

constitute one continuous drawing.) 358 



(a) Control organs for lengthening C 2 (COi and C0 2 ) and 

for lengthening the lower part of Ci (C0 3 and C0 4 ) 

(b) PP(1) to store the fact that a "zero" is to be written 

in x n , and control organs for writing a zero and length- 
ening the upper part of Ci 

(c) PP(1) to store the fact that a "one" is to be written in 

x n , and control organs for leaving a one in cell x n and 
lengthening the upper part of Ci 

(d) Control organs for shortening C 2 

(e) Control organs and PP(1) for shortening the lower part 

of Ci and writing in cell x n 

(f) Control organs for shortening the upper part of G 



Chapter 5 

Fig. 42. Crossing organ 364 

(a) Crossing organ 

(b) Initial state of clock 

Fig. 43. State organ SO of a finite automaton FA 365 

Fig. 44. Constructing arm 366 

Fig. 45. Horizontal advance of constructing arm 366 

Fig. 46. Vertical advance of constructing arm 367 

Fig. 47. Horizontal retreat of constructing arm with construction 

of 7 and 8 368 

Fig. 48. Vertical retreat of constructing arm with construction 

of 7 and 8 369 

Fig. 49. Injection of starting stimulus into the secondary autom- 
aton 370 

Fig. 50. Operation of the constructing arm 371 



LIST OF FIGURES xiii 

Fig. 51. New method of operating the linear array L 372 

Fig. 52. Writing "one" in cell x n and lengthening the reading 

loop 373 

Fig. 53. Static-dynamic converter 374 

Fig. 54. The universal constructor M c * 375 

Fig. 55. Self-reproduction 376 



Fig. 56. Self -reproduction of a universal computer-constructor. . 377 



PREFACE 



In the late 1940's John von Neumann began to develop a theory 
of automata. He envisaged a systematic theory which would be 
mathematical and logical in form and which would contribute in an 
essential way to our understanding of natural systems (natural auto- 
mata) as well as to our understanding of both analog and digital 
computers (artificial automata). 

To this end von Neumann produced five works, in the following 
order: 

(1) "The General and Logical Theory of Automata." Read at the 
Hixon Symposium in September, 1948; published in 1951. Col- 
lected Works 5.288-328. 1 

(2) "Theory and Organization of Complicated Automata." Five 
lectures delivered at the University of Illinois in December, 
1949. This is Part I of the present volume. 

(3) "Probabilistic Logics and the Synthesis of Reliable Organisms 
from Unreliable Components." Lectures given at the California 
Institute of Technology in January, 1952. Collected Works 5.329- 
378. 

(4) "The Theory of Automata: Construction, Reproduction, Ho- 
mogeneity." Von Neumann started this manuscript in the fall 
of 1952 and continued working on it for about a year. This is 
Part II of the present volume. 

(5) The Computer and the Brain. Written during 1955 and 1956; 
published in 1958. 

The second and fourth of these were left at his death in a manuscript 
form which required extensive editing. As edited they constitute the 
two parts of the present volume, which thus concludes von Neu- 
mann's work on the theory of automata. 

As a background for this editorial work I made a study of all of 
von Neumann's contributions on computers, including the theory of 
automata. I have summarized his contributions in the "Introduction" 
to the present volume. 

Von Neumann was especially interested in complicated automata, 
such as the human nervous system and the tremendously large com- 

1 Complete references are given in the bibliography. "Collected Works 5.288- 
328" refers to pp. 288-328 of Vol. V of von Neumann's Collected Works. 



xv 



Xvi THEORY OF SELF-REPRODUCING AUTOMATA 

puters he foresaw for the future. He wanted a theory of the logical 
organization of complicated systems of computing elements and be- 
lieved that such a theory was an essential prerequisite to constructing 
very large computers. The two problems in automata theory that 
von Neumann concentrated on are both intimately related to com- 
plexity. These are the problems of reliability and self-reproduction. 
The reliability of components limits the complexity of the automata 
we can build, and self -reproduction requires an automaton of con- 
siderable complexity. 

Von Neumann discussed reliability at length in his "Probabilistic 
Logics and the Synthesis of Reliable Organisms from Unreliable 
Components. " His work on self -reproducing automata is found chiefly 
in the present volume. Part II, which constitutes the bulk of the 
present volume, treats the logical design of a self-reproducing cellular 
automaton. Though the shorter Part I is devoted to complicated 
automata in general, its high point is the kinematic model of self- 
reproduction (Fifth Lecture). It therefore seemed appropriate to use 
the title "Theory of Self-Reproducing Automata" for the whole 
work. 

It is unfortunate that, because of his premature death, von Neu- 
mann was unable to put in final form any of the research he was 
doing in automata theory. The manuscripts for both parts of the 
present volume were unfinished; indeed, they were both, in a sense, 
first drafts. There is one compensation in this: one can see von Neu- 
mann's powerful mind at work. Early drafts by a thinker of von 
Neumann's ability are not often available. For this reason, I have 
tried hard to preserve the original flavor of von Neumann's manu- 
scripts, while yet rendering them easily readable. So that the reader 
will know what the raw manuscripts are like, I will describe them 
and the editorial changes I have made in them. 

Von Neumann agreed to write a book on automata theory in con- 
nection with his five lectures at the University of Illinois in Decem- 
ber, 1949. A tape recording of the lectures was made to aid him in 
writing the book. Unfortunately, the recording and the typescript of 
it turned out badly, with gaps in the text, unintelligible passages, and 
missing words. Von Neumann himself never edited this typescript, 
but instead planned to use the manuscript "The Theory of Auto- 
mata: Construction, Reproduction, Homogeneity" for the promised 
book. The recording itself is not extant. Despite these circumstances, 
the Illinois lectures deserve publication, and the recorded version, 
highly edited of necessity, constitutes Part I of this volume. 

Von Neumann prepared a detailed outline of the lectures in advance 



PREFACE 



xvii 



of delivery, and the content of the lectures corresponded roughly to 
this outline. The outline bore the title "Theory and Organization of 
Complicated Automata/' and began with the following three lines: 

The logical organization and limitations of high-speed digital 

computers. 

Comparison of these and other complicated automata, both 
artificial and natural. 

Inference from the comparison of the nervous systems 
found in nature. 

There followed the title of each lecture and a list of topics to be cov- 
ered in that lecture; these are reproduced verbatim at the beginning 
of each lecture below, even though the lecture materials do not cor- 
respond exactly to the list of topics. 

Because of the state of the manuscript it has been necessary to do 
much editing. The typescript of the recording is especially poor in the 
more formal portions of the lectures, where von Neumann used the 
blackboard. For these portions, particularly, I have found two sets 
of notes taken at the lectures to be helpful. I have preserved von 
Neumann's phraseology where feasible, but I have frequently found 
it necessary to use my own words. I have sometimes felt it best to 
summarize what von Neumann was saying rather than to attempt 
to reconstruct the text. Several of the points von Neumann made in 
the Illinois lectures also appear in his published writings, or are well 
known, and in these cases I have often summarized what von Neu- 
mann said or given references to his published works. 

Where the writing is strictly my own, it appears in brackets. The 
reconstructed edition of von Neumann's words is not bracketed, but 
it should be kept in mind that much of this unbracketed text is heavily 
edited. 

The manuscript "The Theory of Automata: Construction, Repro- 
duction, Homogeneity" was in a much better state. It seems to have 
been a first draft, with the exception that there is an earlier outline 
(with figures) of the procedure whereby the memory control MC 
lengthens and shortens the connecting loop Ci and the timing loop 
C 2 under the direction of the constructing unit CU (cf. Sees. 4.1 and 
4.2). Despite its being a first draft, the manuscript was publishable 
as it stood except for deficiencies of the following three types. 

(1) First, the manuscript lacked many of those simple mecha- 
nisms which make for easy reading. There were no figure titles. For- 
mulas, sections, and figures were referred to by number only, without 
explicit indication as to whether the item referred to is a formula, 
section, or figure. Section titles were listed on a separate sheet. Also 



xviii 



THEORY OF SELF-REPRODUCING AUTOMATA 



on a separate sheet, von Neumann gave only brief indications for the 
footnotes he planned. Organs were referred to by letter alone. For 
example, von Neumann merely used "A" and "B" to refer to the 
organs I have called the "constructing unit CU" and the "memory 
control MC," respectively. I have worked through the manuscript 
several times and each time I have been amazed at how von Neumann 
could keep track of what he was doing with so few mnemonic devices. 

In editing the manuscript I have endeavored to supply these de- 
vices. For example, where von Neumann wrote "CO" I often put 
"control organ CO." I have added titles to the figures and completed 
the footnote references. Von Neumann wrote some explanatory re- 
marks on the figures; these have been moved to the text. Similar 
and related changes have been made, all without any indication in 
the text. 

In addition, I have inserted footnotes, commentaries, explanations, 
and summaries at various places in the text, and have added a con- 
including chapter (Ch. 5). All such additions are in brackets. Von 
Neumann's brackets have been changed to braces, except for his 
usage "[0]" and "[1]" to refer to ordinary and special symbols. In 
connection with my bracketed additions, I have added Tables I and 
V and many figures. Figures 1-8, 16, 18, 19, 22, 24, 28-36, 38, 39, 
and 41 are von Neumann's; the remaining figures are mine. 

(2) Second, the manuscript "Theory of Automata: Construction, 
Reproduction, Homogeneity" contained many errors. These range 
from minor slips (which I have corrected without any specific indica- 
tion), through errors of medium significance (which I have corrected 
or commented on in bracketed passages), to major errors requiring 
considerable redesign (which I have discussed in Sections 5.1.1 and 
5.1.2). All of these errors are correctable, but because organs designed 
in the earlier parts of the manuscript are used in later parts, many of 
these errors propagate and "amplify." In this connection, it should 
be kept in mind that the manuscript was an early draft, and that 
von Neumann was working out the design as he proceeded, leaving 
many design parameters for later specification. 

(3) Third, the manuscript "Theory of Automata: Construction, 
Reproduction, Homogeneity" is incomplete. The construction stops 
before the tape unit is quite finished. In Chapter 5 I show how to 
complete the design of von Neumann's self -reproducing automaton. 

The technical development of the manuscript is extremely com- 
plicated and involved. The deficiencies just mentioned add to its 
difficulty. In some respects it would have been editorially easier not 
to edit the manuscript after Chapter 2 and instead work out the 



PREFACE 



xix 



design of von Neumann's self -reproducing automaton along the lines 
he last envisaged. But this was not a real alternative because of the 
historical importance of the manuscript, and the opportunity it gives 
to observe a powerful mind at work. I have therefore endeavored to 
make corrections and add comments so as to preserve the original 
style of the manuscript while making it relatively easy to read. 

I am indebted to a number of people for their assistance. The late 
Mrs. Klara von Neumann-Eckardt gave me information about her 
husband's manuscripts. Several people who worked with von Neu- 
mann on computers gave me firsthand information: Abraham Taub, 
Herman Goldstine, the late Adele Goldstine, and especially Julian 
Bigelow and Stan Ulam; von Neumann often discussed his work on 
automata theory with Bigelow and with Ulam. John Kemeny, Pierce 
Ketchum, E. F. Moore, and Claude Shannon heard lectures by or 
had discussions with von Neumann on automata theory. Kurt GodePs 
letter at the end of the Second Lecture of Part I is reproduced with 
his kind permission. Thanks go to many of my graduate students 
and research associates for technical assistance, particularly Michael 
Faiman, John Hanne, James Thatcher, Stephen Hedetniemi, Frederick 
Suppe, and Richard Laing. Alice Finney, Karen Brandt, Ann Jacobs, 
and Alice R. Burks have provided editorial assistance. M. Elizabeth 
Brandt drew the figures. My editorial work was supported by the 
National Science Foundation. None of these share any responsibility 
for the editing. 

Arthur W. Burks 
Ann Arbor, 1965 



EDITOR'S INTRODUCTION 



Von Neumann's Work on Computers 

John von Neumann was born on December 28, 1903 in Budapest, 
Hungary, and died in Washington, D.C., February 8, 1957. 1 He 
earned a doctorate in mathematics from the University of Budapest 
and an undergraduate chemistry degree from the Eidgenossische 
Technische Hochschule in Zurich, Switzerland. He became a Privat- 
docent at the University of Berlin in 1927 and a Privatdocent at the 
University of Hamburg in 1929. In 1930 he came to the United States 
as a visiting lecturer at Princeton University, where he was made full 
professor in 1931. In 1933 he joined the newly formed Institute for 
Advanced Study as a professor and retained that post for the rest of 
his life. 2 

In later life, while retaining his theoretical interests and produc- 
tivity, von Neumann developed strong interests in the applications 
of mathematics. During the Second World War he became heavily 
involved in scientific research on problems of defense. He played a 
major role in the development of the atomic bomb, contributing 
particularly to the method of implosion. He was a consultant to 
many government laboratories and organizations and a member of 
many important scientific advisory committees. After the war he 
continued these consulting and advisory activities. Altogether he 
was involved in such diverse fields as ordnance, submarine warfare, 
bombing objectives, nuclear weapons (including the hydrogen bomb), 
military strategy, weather prediction, intercontinental ballistic mis- 
siles, high-speed electronic digital computers, and computing methods. 
In October, 1954, the President of the United States appointed him 
to the United States Atomic Energy Commission, a position he held 
at the time of his death. He received many awards and honors during 
his lifetime, including membership in the National Academy of Sci- 
ences, two Presidential Awards, and the Enrico Fermi Award of the 
Atomic Energy Commission. The latter was given especially for his 

1 See Ulam, "John von Neumann," and Mrs. von Neumann's preface to The 
Computer and the Brain. 

2 See his Collected Works, edited by A. Taub. An excellent summary of von 
Neumann's accomplishments is presented in the Bulletin of the American 
Mathematical Society, Vol. 64, No. 3, Part 2, May, 1958. 



1 



2 



THEORY OF SELF-REPRODUCING AUTOMATA 



contributions to the development of electronic computers and their 
uses. 

Von Neumann the Mathematician. During the last years of his life 
John von Neumann devoted considerable effort to developing a 
theory of automata. The present volume, edited from two unfinished 
manuscripts, is his last work on this subject. Because of his premature 
death he was unable to finish a volume which would present a com- 
plete picture of what he wished to accomplish. It is therefore appro- 
priate to summarize here the main features of his projected theory 
of automata. Since his conception of automata theory arose out of 
his work in mathematics and computers, we will begin by describing 
that work. 

Von Neumann was a very great mathematician. He made many 
important contributions in a wide range of fields. Von Neumann 
himself thought his most important mathematical achievements , 
were in three areas: the mathematical foundations of quantum theory, 
the theory of operators, and ergodic theory. His contributions in other 
areas bear more directly on his computer work. In the late 1920 , s he 
wrote on symbolic logic, set theory, axiomatics, and proof theory. 
In the middle thirties he worked on lattice theory, continuous ge- 
ometry, and Boolean algebra. In a famous paper of 1928 and in a 
book of 1944 3 he founded the modern mathematical theory of games. 
Starting in the late thirties and continuing through and after the 
war he did much research in fluid dynamics, dynamics, problems in 
the mechanics of continua arising out of nuclear technology, and 
meteorology. During the war he became involved in computing 
and computers, and after the war this became his main interest. 

Von Neumann and Computing. Von Neumann was led into com- 
puting by his studies in fluid dynamics. Hydrodynamical phenomena 
are treated mathematically by means of non-linear partial differential 
equations. Von Neumann became especially interested in hydro- 
dynamical turbulence and the interaction of shock waves. He soon 
found that existing analytical methods were inadequate for obtaining 
even qualitative information about the solutions of non-linear partial 
differential equations in fluid dynamics. Moreover, this was so of 
non-linear partial differential equations generally. 

Von Neumann's response to this situation was to do computing. 4 
During the war he found computing necessary to obtain answers to 

3 "Zur Theorie der Gesellschaftsspiel." Theory of Games and Economic Be- 
havior, with Oskar Morgenstern. 

4 See Ulam, "John von Neumann," pp. 7-8, 28 ff., and Birkhoff, Hydrody- 
namics, pp. 5, 25. 



editor's introduction 



3 



problems in other fields, including nuclear technology. Hence, when 
the new high-speed electronic digital general-purpose computers were 
developed during and after the war, he was quick to recognize their 
potentialities for hydrodynamics as well as other fields. In this con- 
nection he developed a general method for using computers which is 
of very great importance because it is applicable to a wide variety 
of problems in pure and applied mathematics. 

The procedure which he pioneered and promoted is to employ 
computers to solve crucial cases numerically and to use the results 
as a heuristic guide to theorizing. Von Neumann believed experi- 
mentation and computing to have shown that there are physical 
and mathematical regularities in the phenomena of fluid dynamics 
and important statistical properties of families of solutions of the 
non-linear partial differential equations involved. These regularities 
and general properties could constitute the basis of a new theory of 
fluid dynamics and of the corresponding non-linear equations. Von 
Neumann believed that one could discover these regularities and 
general properties by solving many specific equations and generaliz- 
ing the results. From the special cases one would gain a feeling for 
such phenomena as turbulence and shock waves, and with this quali- 
tative orientation could pick out further critical cases to solve nu- 
merically, eventually developing a satisfactory theory. See the First 
Lecture of Part I of this volume. 

This particular method of using computers is so important and has 
so much in common with other, seemingly quite different, uses of 
computers that it deserves extended discussion. It is of the essence 
of this procedure that computer solutions are not sought for their 
own sake, but as an aid to discovering useful concepts, broad prin- 
ciples, and general theories. It is thus appropriate to refer to this as 
the heuristic use of computers. 5 

The heuristic use of computers is similar to and may be combined 
with the traditional hypothetical-deductive-experimental method of 
science. In that method one makes an hypothesis on the basis of the 
available information, derives consequences from it by means of 
mathematics, tests the consequences experimentally, and forms a 
new hypothesis on the basis of the findings; this sequence is iterated 
indefinitely. In using a computer heuristically one proceeds in the 
same way, with computation replacing or augmenting experimenta- 
tion. One makes an hypothesis about the equations under investiga- 

5 See also Ulam, A Collection of Mathematical Problems, Ch. 8, ' 'Computing 
Machines as a Heuristic Aid." 



4 



THEORY OF SELF-REPRODUCING AUTOMATA 



tion, attempts to pick out some crucial special cases, uses a computer 
to solve these cases, checks the hypothesis against the results, forms 
a new hypothesis, and iterates the cycle. 

The computations may also be compared with experimental data. 
When this is done the heuristic use of computers becomes simulation. 
Computation in itself can only provide answers to purely mathe- 
matical questions, so when no comparison is made with empirical 
fact the heuristic use of computers contributes to pure mathematics. 
Von Neumann thought that the main difficulties in fluid dynamics 
stemmed from inadequate mathematical knowledge of non-linear 
partial differential equations, and that the heuristic use of computers 
would help mathematicians to construct an adequate and useful 
theory for this subject. He pointed out that while much progress 
had been made by means of wind tunnels, since the equations govern- 
ing the phenomena were known, these wind tunnels were being used 
as analog computers rather than as experimental apparatus. 

. . . many branches of both pure and applied mathematics are in great need 
of computing instruments to break the present stalemate created by the 
failure of the purely analytical approach to non-linear problems. . . . really 
efficient high-speed computing devices may, in the field of non-linear partial 
differential equations as well as in many other fields, which are now difficult 
or entirely denied access, provide us with those heuristic hints which are 
needed in all parts of mathematics for genuine progress. 6 

Von Neumann's suggestion that powerful computers may provide 
the mathematician "with those heuristic hints which are needed in 
all parts of mathematics for genuine progress" is connected to his 
strong conviction that pure mathematics depends heavily on empirical 
science for its ideas and problems. . . the best inspirations of mod- 
ern mathematics . . . originated in the natural sciences. ,, 7 He recog- 
nized that mathematics is not an empirical science and held that the 
mathematician's criteria of selection of problems and of success are 
mainly aesthetical. 

I think that it is a relatively good approximation to truth — which is 
much too complicated to allow anything but approximations — that mathe- 
matical ideas originate in empirics, although the genealogy is sometimes long 
and obscure. But once they are so conceived, the subject begins to live a 
peculiar life of its own and is better compared to a creative one, governed by 
almost entirely aesthetical motivations, than to anything else and, in par- 

6 Von Neumann and Goldstine, "On the Principles of Large Scale Comput- 
ing Machines," Collected Works 5.4. 

7 "The Mathematician," Collected Works 1.2. The next quotation is from 
the same article, 1.9. 



editor's introduction 



5 



ticular, to an empirical science. There is, however, a further point which, I 
believe, needs stressing. ... at a great distance from its empirical source, or 
after much "abstract" inbreeding, a mathematical subject is in danger of 
degeneration. . . . whenever this stage is reached, the only remedy seems to 
me to be the rejuvenating return to the source: the reinjection of more or 
less directly empirical ideas. 

The role that empirical science plays in pure mathematics is a 
heuristic one: empirical science supplies problems to investigate and 
suggests concepts and principles for their solution. While von Neu- 
mann never said so, I think it likely that he thought the computations 
produced by the heuristic use of computers can play the same role 
in some areas of mathematics. In the First Lecture of Part I below 
he said that powerful methods in pure mathematics depend for their 
success on the mathematicians having an intuitive and heuristic 
understanding of them, and suggested that one can build this in- 
tuitive familiarity with non-linear differential equations by using 
computers heuristically. 8 

It should be noted that in the heuristic use of computers the hu- 
man, not the machine, is the main source of suggestions, hypotheses, 
heuristic hints, and new ideas. Von Neumann wished to make the 
machine as intelligent as possible, but he recognized that human 
powers of intuition, spatial imagery, originality, etc., are far superior 
to those of present or immediately foreseeable machines. He wished to 
augment the ability of a skilled, informed, creative human by the 
use of a digital computer as a tool. This procedure would involve 
considerable interaction between man and the machine and would 
be facilitated by automatic programming and by input-output equip- 
ment designed for direct human use. 

Once he became interested in computing, von Neumann made 
important contributions to all aspects of the subject and its tech- 
nology. The extant methods of computation had been developed for 
hand computation and punched card machines and hence were not 
well suited to the new electronic computers, which were several orders 
of magnitude faster than the old. New methods were needed, and von 
Neumann developed many of them. He contributed at all levels. He 
devised algorithms and wrote programs for computations ranging 
from the calculation of elementary functions to the integration of 
non-linear partial differential equations and the solutions of games. 

8 In view of von Neumann's emphasis on the role of intuition in mathemati- 
cal discovery it is of interest to note that von Neumann's own intuition was 
auditory and abstract rather than visual. See Ulam, "John von Neumann, 
1903-1957," pp. 12, 23, and 38-39. 



THEORY OF SELF-REPRODUCING AUTOMATA 



He worked on general techniques for numerical integration and in- 
verting matrices. He obtained results in the theory of numerical 
stability and the accumulation of round-off errors. He helped develop 
the Monte Carlo method for solving integro-differential equations, 
inverting matrices, and solving linear systems of equations by ran- 
dom sampling techniques. 9 In this method the problem to be solved 
is reduced to a statistical problem which is then solved by computing 
the results for a sufficiently large sample of instances. 

Von Neumann also made important contributions to the design 
and programming of computers, and to the theory thereof. We will 
survey his work in these areas next. 

Logical Design of Computers. With his strong interest in computing 
and his background in logic and physics it was natural for von Neu- 
mann to become involved in the development of high-speed electronic 
digital computers. The first such computer was the ENIAC, designed 
and built at the Moore School of Electrical Engineering of the Uni- 
versity of Pennsylvania during the period 1943 to 1946. 10 Von Neu- 
mann had some contacts with this machine, and so a few words about 
it are in order. 

The idea of constructing a general purpose high-speed computer of 
electronic components originated with John Mauchly, who suggested 
to H. H. Goldstine of the Ordnance Department that the United 
States Army support the development and construction of such a 
machine, to be used primarily for ballistics computations. This sup- 
port was given, the Army being impressed especially with the great 
speed with which an electronic computer could prepare firing tables. 
The ENIAC was designed and constructed by a number of people, 
including the writer, under the technical direction of Mauchly and 
J. P. Eckert. Von Neumann came to visit us while we were building 
the ENIAC, and he immediately became interested in it. By this time 
the design of the ENIAC was already fixed, but after the ENIAC was 
completed von Neumann showed how to modify it so that it was much 
simpler to program. In the meantime he developed the logical design 
for a radically new computer, which we will describe later. 

The ENIAC was, of course, radically different from any earlier 

9 Ulam, "John von Neumann,'' pp. 33-34. Von Neumann Collected Works 
5.751-764. The method is described in Metropolis and Ulam, "The Monte Carlo 
Method." 

10 See Burks, "Electronic Computing Circuits of the ENIAC" and "Super 
Electronic Computing Machine," Goldstine and Goldstine, "The Electronic 
Numerical Integrator and Computer (ENIAC)," and Brainerd and Sharpless, 
"The ENIAC." 



editor's introduction 



7 



computer, but interestingly enough, it was also quite different from 
its immediate successors. It differed from its immediate successors in 
two fundamental respects: the use of several semiautonomous com- 
puting units working simultaneously and semi-independently, and 
the exclusive reliance on vacuum tubes for high-speed storage. Both 
of these design features resulted from the electronic technology of the 
time. 

The basic pulse rate of the ENIAC circuits was 100,000 pulses per 
second. To obtain a high computation speed all 10 (or 20) decimal 
digits were processed in parallel, and, moreover, a large number of 
computing units were constructed, each with some local programming 
equipment, so that many computations could proceed simultaneously 
under the overall direction of a master programming unit. There were 
30 basic units in the ENIAC: 20 accumulators (each of which could 
store and add a 10-digit number), 1 multiplier, 1 divider and square- 
rooter, 3 function table units, an input unit, an output unit, a master 
programmer, and 2 other units concerned with control. All of these 
basic units could operate at the same time. 

At that time the vacuum tube was the only reliable high-speed stor- 
age device — acoustic delay lines, electrostatic storage systems, mag- 
netic cores, etc., all came later — and so of necessity vacuum tubes were 
used for high-speed storage as well as for arithmetic and for logical 
control. This entailed a severe limitation on the high-speed store, as 
vacuum tubes are an expensive and bulky storage medium — the 
ENIAC contained 18,000 vacuum tubes as it was, a sufficient number 
for the skeptics to predict that it would never operate properly. The 
limited high speed storage of 20 10-digit decimal numbers was aug- 
mented by large quantities of low-speed storage of various types: elec- 
tromagnetic relays for input and output, hand operated mechanical 
switches controlling resistor matrices in the function table units for 
the storage of arbitrary numerical functions and of program informa- 
tion, and hand-operated mechanical switches and flexible plug-in 
cables for programming. 

A general purpose computer must be programmed for each particu- 
lar problem. This was done on the ENIAC by hand: by setting me- 
chanical switches of the program controls of each of the computing 
units used in the problem, interconnecting these program controls with 
cables, and setting the switches of the function tables. This program- 
ming procedure was long, laborious, and hard to check, and while it 
was being done the machine stood idle. After the ENIAC was com- 
pleted, von Neumann showed how to convert it into a centrally 
programmed computer in which all the programming could be done 



8 



THEORY OF SELF-REPRODUCING AUTOMATA 



by setting switches on the function tables. Each of the three function 
table units had a switch storage capacity of 104 entries, each entry 
consisting of 12 decimal digits and 2 sign digits. However, the pulses 
used to represent numbers were the same size and shape as the pulses 
used to stimulate program controls, so that the function table units 
could also be used to store program information. On von Neumann's 
scheme the outputs of the function tables were connected to the pro- 
gram controls of the other units through some special equipment and 
the master programmer, and the switches on the program controls 
of these units were set. All of this was done in such a way that it need 
not be changed from problem to problem. Programming was thus 
reduced to setting switches by hand on the function table units. 

In the meantime we were all concerned with the design of much 
more powerful computers. As mentioned earlier, the greatest weakness 
of the ENIAC was the smallness of its high-speed storage capacity, 
resulting from the technological fact that at the time the design of 
the ENIAC was fixed the vacuum tube was the only known reliable 
high-speed storage component. This limitation was overcome and the 
technology of computers changed abruptly when J. P. Eckert con- 
ceived of using an acoustic delay line as a high-speed storage device. 
Acoustic delay lines made of mercury had been used to delay pulses 
in war time radar equipment. Eckert's idea was to feed the output of 
a mercury delay line (through an amplifier and pulse reshaper) back 
into its input, thereby storing a large number of pulses in a circulat- 
ing memory. A circulating memory of, say, 1000 bits could be built 
with a mercury delay line and a few tubes, in contrast to the ENIAC 
where a double triode was required for each bit. 

In the ENIAC the few numbers being processed were stored in cir- 
cuits that could be changed both automatically and rapidly; all other 
numbers and the program information were stored in electromagnetic 
relays, switches, and cable interconnections. It now became possible to 
store all this information in mercury delay lines, where it would be 
quickly and automatically accessible. The ENIAC w r as a mixed syn- 
chronous, asynchronous machine. The use of pulses in the mercury 
delay lines made it natural to build a completely synchronous machine 
timed by a central source of pulses called the " clock.' ' Eckert and 
Mauchly designed circuits capable of operating at a pulse rate of 1 
megacycle, 10 times the basic pulse rate of the ENIAC, and gave con- 
siderable thought to the design of a mercury delay line machine. 
Goldstine brought von Neumann in as a consultant, and we all par- 
ticipated in discussions of the logical design of such a machine. It was 
decided to use the binary system. Since the delay lines operated 



editor's introduction 



g 



serially, the simplest way to process the bits was seriatim. All of this 
made it possible to build a machine much smaller than the ENIAC 
and yet much more powerful than the ENIAC. The proposed machine 
was to be called the EDVAC. It was estimated that it could be built 
with about 3000 vacuum tubes. 

Von Neumann then worked out in considerable detail the logical 
design of this computer. The result appeared in his First Draft of a 
Report on the EDVAC, 11 which was never published. Since this report 
contained the first logical design of an electronic computer in which 
the program could be stored and modified electronically, I will sum- 
marize its contents. Of particular interest to us here are the following 
features of his design: the separation of logical from circuit design, the 
comparison of the machine to the human nervous system, the general 
organization of the machine, and the treatment of programming and 
control. 

Von Neumann based his construction on idealized switch-delay ele- 
ments derived from the idealized neural elements of McCulloch and 
Pitts. 12 Each such element has one to three excitatory inputs, possibly 
one or two inhibitory inputs, a threshold number (1, 2, 3), and a unit 
delay. It emits a stimulus at time t + 1 if and only if two conditions / 
are satisfied at time t: (1) no inhibitory input is stimulated, (2) the 
number of excitatory inputs stimulated is at least as great as the 
threshold number. 13 

The use of idealized computing elements has two advantages. First, 
it enables the designer to separate the logical design from the circuit 
design of the computer. When designing the ENIAC, we developed 
logical design rules, but these were inextricably tied in with rules 
governing circuit design. With idealized computing elements one can 
distinguish the purely logical (memory and truth-functional) require- 
ments for a computer from the requirements imposed by the state of 
technology and ultimately by the physical limitations of the materials 
and components from which the computer is made. Logical design is 
the first step; circuit design follows. The elements for logical design 

11 The initials abbreviate "Electronic Discrete Variable Automatic Com- 
puter." The machine of this name actually constructed at the Moore School 
of Electrical Engineering was built after the people mentioned above were 
no longer connected with the Moore School. The logical design of the Cam- 
bridge University EDSAC was based on this report. Wilkes, "Progress in High- 
Speed Calculating Machine Design" and Automatic Digital Computers. 

12 "A Logical Calculus of the Ideas Immanent in Nervous Activity." 

13 The threshold elements of "Probabilistic Logics and the Synthesis of Re- 
liable Organisms from Unreliable Components," Collected Works 5.332, are 
similar, but differ with respect to the operation of inhibitory inputs. 



10 



THEORY OF SELF-REPRODUCING AUTOMATA 



should be chosen to correspond roughly with the resultant circuits; 
that is, the idealization should not be so extreme as to be unrealistic. 

Second, the use of idealized computing elements is a step in the 
direction of a theory of automata. Logical design in terms of these 
elements can be done with the rigor of mathematical logic, whereas 
engineering design is necessarily an art and a technique in part. More- 
over, this approach facilitates a comparison and contrast between 
different types of automata elements, in this case, between computer 
elements on the one hand and neurons on the other. Von Neumann 
made such comparisons in First Draft of a Report on the EDVAC, not- 
ing the differences as well as the similarities. Thus he observed that 
the circuits of the EDVAC were to be synchronous (timed by a pulse 
clock) while the nervous system is presumably asynchronous (timed 
autonomously by the successive reaction times of its own elements). 
He also noted the analogy between the associative, sensory, and motor 
neurons of the human nervous system on the one hand, and the 
central part of the computer, its input, and its output, respectively. 
This comparison of natural and artificial automata was to become a 
strong theme of his theory of automata. 

The organization of the EDVAC was to be radically different from 
that of the ENIAC. The ENIAC had a number of basic units, all 
capable of operating simultaneously, so that many streams of com- 
putation could proceed at the same time. In contrast, the proposed 
EDVAC had only one basic unit of each kind, and it never performed 
two arithmetical or logical operations simultaneously. These basic 
units were a high-speed memory M, a central arithmetic unit CA, an 
outside recording medium R, an input organ I, an output organ O, 
and a central control CC. 

The memory M was to be composed of possibly as many as 256 
delay lines each capable of storing 32 words of 32 bits each, together 
with the switching equipment for connecting a position of M to the 
rest of the machine. The memory was to store initial conditions and 
boundary conditions for partial differential equations, arbitrary 
numerical functions, partial results obtained during a computation, 
etc., as well as the program (sequence of orders) directing the com- 
putation. The outside recording medium R could be composed of 
punched cards, paper tape, magnetic wire or tape, or photographic 
film, or combinations thereof. It was to be used for input and output, 
as well as for auxiliary low-speed storage. The input organ I trans- 
ferred information from R to M; the output organ O transferred 
information from M to R. The notation of M was binary; that of R 
was decimal. 

The central arithmetic unit CA was to contain some auxiliary regis- 



editor's introduction 



1 1 



ters (one-word delay lines) for holding numbers. Under the direction 
of the central control CC it was to add, subtract, multiply, divide, 
compute square-roots, perform binary-decimal and decimal-binary 
conversions, transfer numbers among its registers and between its 
registers and M, and choose one of two numbers according to the sign 
of a third number. The last operation was to be used for transfer of 
control (jumping conditionally) from one order in the program to 
another. Numbers were processed in CA serially, the least significant 
bits being treated first, and only one operation was performed at a 
time. 

The first bit of each word was zero for a number, one for an order. 
Eight bits of an order were allotted to the specification of the operation 
to be performed and, if a reference to M was required, thirteen bits to 
an address. A typical sequence would go like this. Suppose an addition 
order with memory address x was located in position y of M, the 
addend in the next position y + 1, and the next order to be executed 
in the next position y + 2. The order at y would go into CC, the ad- 
dend at y + 1 into CA, and the augend would be found in CA; the 
sum would be placed in position x of M. The order at position y + 2 
would be executed next. 

Normally orders were taken from the delay lines in sequence, but 
one order with address z provided for CC to take its next order from 
memory position z. When a number was transferred from CA to 
address w of M, account was taken of the contents of w; if w contained 
an order (i.e., a word whose first bit was one), then the 13 most sig- 
nificant bits of the result in CA were substituted for the 13 address 
bits located in w. The addresses of orders could be modified auto- 
matically by the machine in this way. ThisNpro vision, together with 
the order for shift of control to an arbitrary memory position w and 
the power of CA to choose one of two numbers according to the sign 
of a third, made the machine a fully automatic stored program 
computer. 

At the same time that he worked out the logical design of the 
EDVAC von Neumann suggested the development of a high-speed 
memory incorporating the principle of the iconoscope. 14 Information 
is placed on the iconoscope by means of light and sensed by an electron 
beam. Von Neumann suggested that information could also be placed 
on the inside surface of such a tube by means of an electron beam. 
The net result would be storage in the form of electrostatic charges on 
a dielectric plate inside a cathode-ray tube. He predicted that such a 

14 First Draft of a Report on the EDVAC, Section 12.8. 



12 



THEORY OF SELF-REPRODUCING AUTOMATA 



memory would prove superior to the delay line memory. It soon be- 
came apparent that this was so, and von Neumann turned his atten- 
tion to an even more powerful computer based on such a memory. 

The new computer was to be much faster than any other machine 
under consideration, mainly for two reasons. First, in an electrostatic 
storage system each position is immediately accessible, whereas a bit 
or word stored in a delay line is not accessible until it travels to the end 
of the line. Second, it was decided to process all (40) bits of a word in 
parallel, thereby reducing the computation time. The logical design is 
given in Preliminary Discussion of the Logical Design of an Electronic 
Computing Instrument. 1 ^ The proposed computer was built at the 
Institute for Advanced Study by a number of engineers under the di- 
rection of Julian Bigelow, and was popularly known as the JONIAC. 16 
While the machine was still under construction, its logical and cir- 
cuit design was influential on many computers constructed in the 
United States, including computers at the University of Illinois, Los 
Alamos National Laboratory, Argonne National Laboratory, Oak 
Ridge National Laboratory, and the Rand Corporation, as well as 
some machines produced commercially. The JONIAC played an im- 
portant role in the development of the hydrogen bomb. 17 

Programming and Flow Diagrams. Von Neumann immediately 
recognized that these new computers could solve large problems so 
fast that new programming procedures would be needed to enable 
mathematicians and programmers to make full use of the powers of 
these machines. With the order code of the proposed Institute for Ad- 
vanced Study computer in mind he proceeded to develop new pro- 
gramming methods. The results were presented in the influential 
series of reports Planning and Coding of Problems for an Electronic 
Computing Instrument. 1 * 

One generally begins with a mathematical formulation of a problem 
and then decides what explicit computational methods he will em- 
ploy. These methods are almost always highly inductive, involving 
recursions within recursions many times over. What one has at this 

15 This was written in collaboration with H. H. Goldstine and the present 
writer. It was typical of von Neumann that he wanted the patentable material 
in this report to belong to the public domain, and at his suggestion we all signed 
a notarized statement to this effect. 

16 It is described by Estrin, ' 'The Electronic Computer at the Institute for 
Advanced Study." The original plan was to use the memory described by 
Rajchman in 'The Selectron — a Tube for Selective Electrostatic Storage,' ' but 
the actual memory consisted of cathode-ray tubes operated in the manner de- 
scribed by Williams in "A Cathode-Ray Digit Store." 

17 New York Times, Feb. 9, 1957, p. 19. 

18 Written in collaboration with H. H. Goldstine. 



editor's introduction 



13 



stage is a general description of the desired computation, expressed in 
the ordinary language and mathematical symbolism of the mathe- 
matician. The task is now to transform this description into a program 
expressed in machine language. This is not a simple, straightforward 
translation task, however, partly because of the generality of the 
description of the computation and partly because of the nature of 
recursive procedures. 

Recursive procedures, particularly when complicated, are better 
understood dynamically (in terms of their step by step effects) rather 
than statically (in terms of the static sequence of symbols defining 
them). The corresponding aspect of the machine language is that the 
effect of an order is dependent on the very computation which it itself 
is helping to direct: whether and how often an order is used and to what 
memory position it refers. All of these are a function of the whole 
program and the numbers being processed. Thus a program, though a 
static sequence of symbols, is usually best understood in terms of its 
dynamic effects, that is, its control of the actual sequential computa- 
tional process. 

To help bridge this gap between the mathematician's description of 
the desired computation in his own language and the corresponding 
program in the machine language, von Neumann invented the flow 
diagram. A flow diagram is a labeled graph composed of enclosures 
and points connected by lines. The enclosures are of various kinds: 
operation boxes (specifying non-recursive fragments of the computa- 
tion as symbolized in the box), alternative boxes (corresponding to 
conditional transfer of control orders and being labeled with the 
condition for transfer), substitution and assertion boxes (indicating 
the values of the indices of the recursions)^ storage boxes (giving the 
contents of crucial parts of the memory at certain stages of the com- 
putation), and labeled circles representing the beginning and terminus 
and interconnections. In executing the program corresponding to a 
given flow diagram, the computer in effect travels through the flow 
diagram, starting at the beginning circle, executing sequences of or- 
ders described in operation boxes, cycling back or branching off to a 
new part of the diagram according to the criteria stated in alternative 
boxes, leaving an exit circle in one part of the graph to enter an en- 
trance circle in another part of the graph, and finally stopping at the 
terminal circle. Direct lines are used to represent the direction of 
passage through the graph, converging lines meeting at points of the 
graph. An undirected line is used to connect a storage box to that 
point of the graph which corresponds to the stage of computation 
partly described by the contents of the storage box. 



11 



THEORY OF SELF-REPRODUCING AUTOMATA 



It is unnecessary for the programmer to prepare and code a com- 
plete flow diagram for a complicated problem. A problem of any con- 
siderable complexity is composed of many subproblems, and flow 
diagrams and subroutines can be prepared for these in advance. It 
was planned to code subroutines corresponding to a large number of 
basic algorithms employed in solving problems on a digital computer: 
decimal-to-binary and binary-to-decimal conversion, double precision 
arithmetics, various integration and interpolation methods, meshing 
and sorting algorithms, etc. These subroutines would be available in a 
library of tapes. To solve a particular problem, the programmer would 
merely write a "combining routine" which would direct the computer 
to take the proper subroutines from the tape and modify them ap- 
propriately to that particular problem. 

The use of combining routines and a library of subroutines was a 
first step in the direction of using a computer to help prepare programs 
for itself. Still, in this system, everything written by the programmer 
must be in the clumsy "machine language." A better procedure is to 
construct a "programmer's language" in which the programmer will 
write programs, and then to write a translation program in machine 
language which directs the machine to translate a program written in 
the programmer's language into a program stated in machine lan- 
guage. The programming language would be close to the natural and 
mathematical language normally used by mathematicians, scientists, 
and engineers, and hence would be easy for the programmer to use. 
This approach is currently being developed under the name of auto- 
matic programming. Von Neumann discussed it under the names 
"short code" (programmer's language) and "complete code" (machine 
language). 19 

Von Neumann recognized that the idea of automatic programming 
is a practical application of Turing's proof that there exists a universal 
computing machine. A Turing machine is a finite automaton with an 
indefinitely expandable tape. Any general purpose computer, together 
with an automatic factory which can augment its tape store without 
limit, is a Turing machine. Turing's universal computer U has this 
property: for any Turing machine M there is a finite program P such 
that machine U, operating under the direction of P, will compute the 
same results as M. That is, U with P simulates (imitates) M. 

Automatic programming also involves simulation. Let U c be a com- 
puter which operates with a machine language inconvenient for the 

19 The Computer and the Brain, pp. 70-73. 



editor's introduction 



15 



programmer to use. The programmer uses his more convenient pro- 
grammer's language. It is theoretically possible to build a machine 
which will understand the programmer's language directly; call this 
hypothetical computer M p . Let P t be the program (written in the 
language of machine U c ) which translates from the programmer's 
language to the machine language of U c . Then U c , operating under 
the direction of P t , will compute the same results as M p . That is, 
Uc with P t simulates M p , which is a special case of Turing's uni- 
versal U with P simulating M. 

Note that two languages are employed inside U c • a machine 
language which is used directly and a programmer's language which 
is used indirectly via the translation routine P t . Von Neumann re~ 
ferred to these as the "primary" and "secondary" language of the 
machine, respectively. The primary language is the language used for 
communication and control within the machine, while the secondary 
language is the language we humans use to communicate with the 
machine. Von Neumann suggested that by analogy there may be a 
primary and secondary language in the human nervous system, and 
that the primary language is very different from any language we 
know. 

Thus the nervous system appears to be using a radically different system 
of notation from the ones we are familiar with in ordinary arithmetics and 
mathematics. . . . 

. . . whatever language the central nervous system is using, it is characterized 
by less logical and arithmetical depth than what we are normally used to. 

Thus logics and mathematics in the central nervous system, when viewed as 
languages, must be structurally essentially different from those languages to 
which our common experience refers. ^ 

. . . when we talk mathematics, we may be discussing a secondary language, 
built on the primary language truly used by the central nervous system. 20 

He thought that the primary language of the nervous system was 
statistical in character. Hence his work on probabilistic logics was 
relevant to this language. See his discussion of probabilistic logics 
and reliability in the Third and Fourth Lectures of Part I below and 
in "Probabilistic Logics and the Synthesis of Reliable Organisms 
from Unreliable Components." 

Computer Circuits. From the beginning von Neumann had an in- 
terest in the circuits and components of electronic digital computers. 



The Computer and the Brain, pp. 79-82. 



10 



THEORY OF SELF-REPRODUCING AUTOMATA 



He analyzed the basic physical and chemical properties of matter 
with the purpose of developing improved computer components. 21 In 
his lectures on automata theory he compared natural and artificial 
components with respect to speed, size, reliability, and energy dis- 
sipation, and he computed the thermodynamic minimum of energy 
required for a binary decision. See the Fourth Lecture of Part I of the 
present volume. His search for physical phenomena and effects that 
could be used for computing led to his invention of a new component. 

This is a subharmonic generator which is driven by an excitation 
(power) source at frequency nf (n = 2, 3, 4, • • •) and which oscillates 
at the subharmonic frequency f. 22 The subharmonic generator cir- 
cuit incorporates an inductance and capacitance circuit tuned to the 
frequency /. Either the capacitance or inductance is non-linear, and 
its value varies periodically under the influence of the exciting signal 
(of frequency nf). The oscillation at frequency / can occur in any of n 
distinct phases. Each oscillation phase is highly stable when estab- 
lished, but, when the oscillation begins, the choice of phase can easily 
be controlled by a small input signal of frequency / and of the desired 
phase. Modulating (turning off and on) the exciting source (of fre- 
quency nf) with a square wave (clock signal) of much lower frequency 
produces alternate passive and active periods, and an input of fre- 
quency / can select one of the n phases of oscillation as the exciting 
signal appears. 

To transfer the phase state of one subharmonic generator (a trans- 
mitter) to another (a receiver), the transmitter and receiver are 
coupled through a transformer. The square-wave modulations into 
transmitter and receiver are of the same frequency but of different 
phase, so that the transmitter is still on while the receiver is just 

21 Most of his ideas in this area were only discussed with others and never 
published. A brief reference occurs in Preliminary Discussion of the Logical 
Design of an Electronic Computing Instrument, Collected Works 5.39. Booth, 
"The Future of Automatic Digital Computers," p. 341, mentions a supercon- 
ducting storage element discussed with von Neumann in 1947. Von Neumann 
also did some early work on the MASER. See Collected Works 5.420, Scientific 
American (February, 1963) p. 12, and Scientific American (April, 1963) pp. 
14-15. 

22 "Non-Linear Capacitance of Inductance Switching, Amplifying and 
Memory Devices." Von Neumann's ideas are also described by Wigington, "A 
New Concept in Computing." 

The parametron, invented independently by E. Goto, embodies essentially 
the same idea, but is far different in the suggested speed of implementation. See 
Goto, "The Parametron, a Digital Computing Element which Utilizes Para- 
metric Oscillation." The highest frequencies Goto reports are an exciting fre- 
quency (2/) of 6 X 10 6 cycles per second and a clock frequency of 10 5 cycles. 
According to Wigington, op. cit. y von Neumann estimated that an exciting fre- 
quency (20 of 5 X 10 10 and a clock rate of 10 9 were feasible. 



editor's introduction 



17 



coming on. As a result the receiver begins oscillating at frequency / in 
phase with the transmitter. The receiver can later transmit its state 
to another subharmonic generator, and so on down the line. One may 
use three clock signals, all of the same frequency but of three different 
phases, and, by exciting interconnected generators with the proper 
clock signals, transfer information around a system of generators. 
Each such generator then has an input and an output operating at 
frequency/, beside the exciting input of frequency nf; the phasing of 
the two different clock signals to two interconnected generators de- 
termines which generator is the receiver and which is the transmitter. 
The output signal (at/) has much more power than is required for the 
input signal (at /) to control the phase of the oscillation, and so the 
subharmonic generator is an amplifier at frequency /, the power for 
amplification coming from the exciting signal of frequency nf. 

Since the oscillation of the subharmonic generator is stable and con- 
tinues after the subharmonic input from another generator terminates, 
the device clearly has memory capacity. Switching can also be done 
with subharmonic generators, in the following way. Let n = 2; i.e., 
let there be two distinct phases of subharmonic oscillation at frequency 
/, so that the system is binary. Connect the outputs of three trans- 
mitting generators to the primary of a transmitter so that the voltages 
of these outputs add; connect a receiver generator to the secondary 
of this transformer. The voltage of the transformer secondary will then 
have the phase of the majority of the transmitting generators, so that 
the receiving generator will oscillate in this phase. This arrangement 
realizes a majority element, that is, a three-input switch with delay 
whose output state is "1" if and only if two or more inputs are in 
state "l". 23 A negation element may be realized by connecting the 
output of one generator to the input of another and reversing the 
direction of the transformer winding. The constants "0" and "1" are 
realized by sources of the two different phases of the signal of fre- 
quency /. The majority element, negation element, and the constant 
sources "0" and "1" are sufficient to do all computing, so that the 
central part of a computer can be completely constructed from sub- 
harmonic generators. 24 

Von Neumann's Theory of Automata 

Introduction. On reviewing the preceding sketch of von Neumann's 
research accomplishments, one is immediately struck by the tremen- 

23 "Probabilistic Logics and the Synthesis of Reliable Organisms from Un- 
reliable Components, " Collected Works 5.339. 

24 Many computers are so constructed. See Goto, op. ext. 



18 



THEORY OF SELF-REPRODUCING AUTOMATA 



dous combination of breadth and depth revealed in those accomplish- 
ments. Particularly notable is the extent to which von Neumann's 
achievements range from the purely theoretical to the extremely 
practical. It should be added in the latter connection that he was 
among the first to recognize and promote the tremendous potentiali- 
ties in computers for technological revolution and the prediction and 
control of man's environment, such as the weather. 

Von Neumann was able to make substantial contributions to so 
many different fields because he possessed a rare combination of differ- 
ent abilities along with wide interests. His quick understanding and 
powerful memory enabled him to absorb, organize, retain, and use 
large quantities of information. His wide interests led him to work in 
and keep contact with many areas. He was a virtuoso at solving diffi- 
cult problems of all kinds and at analyzing his way to the essence of 
any situation. 

This wide range of interests and abilities was one of von Neumann's 
great strengths as a mathematician and made him an applied mathe- 
matician par excellence. He was familiar with the actual problems of 
the natural and engineering sciences, on the one hand, and the ab- 
stract methods of pure mathematics on the other. He was rare among 
mathematicians in his ability to communicate with scientists and 
engineers. This combination of theory and practice was deliberately 
cultivated by von Neumann. He was a careful student of the history * 
and nature of scientific method and its relation to pure mathematics 25 
and believed that mathematics must get its inspiration from the 
empirical sciences. 

Given his background and type of mind, it was natural for von 
Neumann to begin to construct a general theory of computers. Being 
aware of the important similarities between computers and natural 
organisms, and of the heuristic advantages in comparing such different 
but related systems, he sought a theory that would cover them both. 
He called his proposed systematic theory the "theory of automata." 
This theory of automata was to be a coherent body of concepts and 
principles concerning the structure and organization of both natural 
and artificial systems, the role of language and information in such 
systems, and the programming and control of such systems. Von 
Neumann discussed the general nature of automata theory at several 
places in Part I and in Chapter 1 of Part II of the present volume. 

Von Neumann's early work on computer design and programming 

25 See Chapter 1 of Theory of Games and Economic Behavior; ' 'The Mathe- 
matician," Collected Works 1.1-9; and "Method in the Physical Sciences," 
Collected Works 6.491-498. 



editor's introduction 



I!) 



led him to recognize that mathematical logic would play a strong role 
in the new theory of automata. But for reasons to be mentioned later, 
he thought that mathematical logic in its present form, though useful 
in treating automata, is not adequate to serve as "the" logic of auto- 
mata. Instead, he believed that a new logic of automata will arise 
which will strongly resemble and interconnect with probability theory, 
thermodynamics, and information theory. It is obvious from all this 
that von Neumann's theory of automata will, in the beginning at 
least, be highly interdisciplinary. 

Unfortunately, because of his premature death, von Neumann was 
unable to put in final form any of the research he was doing in auto- 
mata theory. In his last work on this subject he said that "it would be 
very satisfactory if one could talk about a 'theory' of such automata. 
Regrettably, what at this moment exists . . . can as yet be described 
only as an imperfectly articulated and hardly formalized 'body of 
experience'." 26 Von Neumann's accomplishments in this area were 
nevertheless substantial. He outlined the general nature of automata 
theory: its structure, its materials, some of its problems, some of its 
applications, and the form of its mathematics. He began a compara- 
tive study of artificial and natural automata. Finally, he formulated 
and partially answered two basic questions of automata theory: How 
can reliable systems be constructed from unreliable components? 
What kind of logical organization is sufficient for an automaton to be 
able to reproduce itself? The first of these questions is discussed in his 
"Probabilistic Logics and the Synthesis of Reliable Organisms from 
Unreliable Components." The second question is discussed in the 
Fifth Lecture of Part I and in Part II of the present volume. 

I do not know how von Neumann was led to these two problems, 
but on the basis of his interests and what he has written it is plausible 
that they arose out of his actual work with computers in the following 
way. The new electronic computers were revolutionary because in con- 
trast to earlier computing systems (humans, mechanical and electro- 
mechanical machines, and combinations thereof) they could do large 
quantities of computation automatically and rapidly. The advances 
through the ENIAC, the proposed EDVAC, and the Institute for 
Advanced Study computer, were all big steps in the direction of more 
powerful computers. His interest in solving non-linear partial differ- 
ential equations in general, and in the equations for predicting the 
weather in particular, would naturally lead him to desire ever more 
powerful machines and to look for and try to remove the basic limita- 
tions blocking the construction of such machines. As a consultant for 

26 The Computer and the Brain, p. 2. 



20 



THEORY OF SELF-REPRODUCING AUTOMATA 



government and industry he was very influential in promoting the 
design and construction of larger computers. 

Von Neumann compared the best computers that could be built at 
the time with the most intelligent natural organisms and concluded 
that there were three fundamental factors limiting the engineer's abil- 
ity to build really powerful computers: the size of the available com- 
ponents, the reliability of these components, and a lack of a theory of 
the logical organization of complicated systems of computing ele- 
ments. Von Neumann's work on componentry was directed toward 
the first limitation, and his results on reliability and self -reproduction 
each contribute toward removing both the second and the third limita- 
tions. In his "Probabilistic Logics and the Synthesis of Reliable Or- 
ganisms from Unreliable Components, " he gave two methods of 
overcoming the unreliability of the components, not by making them 
more reliable, but by organizing them so that the reliability of the 
whole computer is greater than the reliability of its parts. He regarded 
his work on probabilistic logics as a step in the direction of the new 
logic of automata. His work on self -reproduction also belongs to the 
theory of complicated automata. He felt that there are qualitatively 
new principles involved in systems of great complexity and searched 
for these principles in the phenomenon of self -reproduction, which 
clearly depends on complexity. It is also to be expected that because 
of the close relation of self -reproduction to self -repair, results on self- 
reproduction would help solve the reliability problem. 

Thus von Neumann was especially interested in complex automata; 
he wanted a theory of the logical organization of complicated systems 
of computing elements. His questions about reliability and self-repro- 
duction are particularly germane to complex automata. 

Two further points are relevant. First, von Neumann believed that 
in starting a new science one should begin with problems that can be 
described clearly, even though they concern everyday phenomena and 
lead to well known results, for the rigorous theory developed to explain 
these phenomena can provide a base for further advances. 27 His prob- 
lems of reliability and self-reproduction are of this kind. Second, von 
Neumann believed that the lack of an adequate theory of complicated 
automata is an important practical barrier to building more powerful 
machines. He explicitly stated that until an adequate theory of auto- 
mata exists there is a limit in the complexity and capacity of the 
automata we can fabricate. 28 

27 Theory of Games and Economic Behavior, Sees. 1.3 and 1.4. 

28 ' 'The General and Logical Theory of Automata," Collected Works 5.302- 
306. 



editor's introduction 



21 



Natural and Artificial Automata. The scope of the theory of auto- 
mata and its interdisciplinary character are revealed by a considera- 
tion of the two main types of automata: the artificial and the natural. 
Analog and digital computers are the most important kinds of artificial 
automata, but other man-made systems for the communication and 
processing of information are also included, for example, telephone 
and radio systems. Natural automata include nervous systems, self- 
reproductive and self -repairing systems, and the evolutionary and 
adaptive aspects of organisms. 

Automata theory clearly overlaps communications and control 
engineering on the one hand, and biology on the other. In fact, arti- 
ficial and natural automata are so broadly defined that one can legiti- 
mately wonder what keeps automata theory from embracing both 
these subjects. Von Neumann never discussed this question, but there 
are limits to automata theory implicit in what he said. Automata 
theory differs from both subjects in the central role played by mathe- 
matical logic and digital computers. Though it has important engi- 
neering applications, it itself is a theoretical discipline rather than a 
practical one. Finally, automata theory differs from the biological 
sciences in its concentration on problems of organization, structure, 
language, information, and control. 

Automata theory seeks general principles of organization, structure, 
language, information, and control. Many of these principles are 
applicable to both natural and artificial systems, and so a comparative 
study of these two types of automata is a good starting point. Their 
similarities and differences should be described and explained. Mathe- 
matical principles applicable to both types of automata should be 
developed. Thus truth-functional logic and delay logic apply to both 
computer components and neurons, as does von Neumann's proba- 
bilistic logic. See the Second and Third Lectures of Part I of the 
present volume. Similarly, von Neumann's logical design of a self- 
reproducing cellular automaton provides a connecting link between 
natural organisms and digital computers. There is a striking analogy 
with the theory of games at this point. Economic systems are natural; 
games are artificial. The theory of games contains the mathematics 
common to both economic systems and games, 29 just as automata 
theory contains the mathematics common to both natural and artifi- 
cial automata. 

Von Neumann himself devoted considerable attention to the com- 
29 Theory of Games and Economic Behavior, Sees. 1.1.2 and 4.1.3. 



22 



THEORY OF SELF-REPRODUCING AUTOMATA 



parison of natural and artificial automata. 30 Scientific knowledge of 
the automata aspects of natural organisms has advanced very rapidly 
in recent years, and so there is now a much more detailed basis for the 
comparison than at the time von Neumann wrote, but his general ap- 
proach and conclusions are nevertheless of interest. We will outline 
his reflections under the following headings: (1) The analog-digital 
distinction, (2) the physical and biological materials used for com- 
ponents, (3) complexity, (4) logical organization, and (5) reliability. 

(1) Von Neumann discussed the analog-digital distinction at length 
and found it to be an illuminating guide in his examination of natural 
automata. See the First and Fourth Lectures of Part I. His most 
general conclusion was that natural organisms are mixed systems, in- L 
volving both analog and digital processes. There are many examples, 
of which two will suffice here. Truth -functional logic is applicable to 
neurons as a first approximation, but such neural phenomena as re- 
fraction and spatial summation are continuous rather than discrete. 
In complicated organisms digital operations often alternate with 
analog processes. For example, the genes are digital, while the enzymes 
they control function analogically. Influenced by his knowledge of 
natural automata von Neumann proposed a combined analog-digital 
computing scheme. 31 This is a good example of the effect of the study 
of natural systems on the design of artificial ones. 

(2) Von Neumann compared the components in existing natural 
and artificial automata with respect to size, speed, energy require- 
ments, and reliability, and he related these differences to such factors 
as the stability of materials and the organization of automata. Com- 
puter components are much larger and require greater energy than 
neurons, though this is compensated for in part by their much greater 
speed. These differences influence the organization of the system: 
natural automata are more parallel in operation, digital computers 
more serial. Part of the difference in size between a vacuum tube and a 
neuron can be accounted for in terms of the mechanical stability of 
the materials used. It is relatively easy to injure a vacuum tube and 
difficult to repair it. In contrast, the neuron membrane when injured 
is able to restore itself. Von Neumann calculated the thermodynamical 
minimum of energy that must be dissipated by a computing element 
and concluded that in theory computing elements could be of the 

30 Norbert Wiener also made valuable comparisons of natural and artificial 
systems in his Cybernetics, though in a somewhat different way. The two men 
were aware of each others work — see Cybernetics (particularly the "Intro- 
duction") and von Neumann's review of it. 

31 Sec. 12 of "Probabilistic Logics and the Synthesis of Reliable Organisms 
from Unreliable Components," Collected Works 5.372-377. 



editor's introduction 



23 



order of 10 10 times more efficient in the use of energy than neurons. 
See the Fourth Lecture of Part I. His comparison of natural and 
artificial components no doubt influenced his work on computer com- 
ponents. 

(3) Man is, inter alia, a natural automaton which is obviously very 
much more complex than any artificial automaton he has so far con- 
structed. Because of this complexity he understands the details of his 
own logical design much less than that of the largest computer he has 
built. Von Neumann thought that the chief problems of automata 
theory center around the concept of complexity. This very concept 
needs rigorous definition. Automata theory should relate the logical 
organization of complex automata to their behavior. A theory which 
did this would enable us to develop the logical design of artificial 
automata capable of carrying out some of the most difficult and ad- 
vanced functions performed by humans as well as many other complex 
functions that humans cannot perform, such as solving large systems 
of non-linear partial differential equations. The problem of reliability 
is especially crucial in complex systems. Von Neumann speculated 
that extremely complex systems involve new principles. He thought, 
for example, that below a certain level, complexity is degenerative, 
and self -reproduction is impossible. He suggested that, generally 
speaking, in the case of simple automata a symbolic description of the 
behavior of an automaton is simpler than the automaton itself, but 
that in the case of exceedingly complex automata the automaton is 
simpler than a symbolic description of its behavior. See the Second 
Lecture of Part I. 

(4) In discussing the relative speeds of natural and artificial com- 
ponents we noted that natural automata tend to be more parallel in 
operation and artificial automata tend to be more serial in operation. 
When planning an automaton or a computation, one can choose some- 
what the extent to which it is parallel or serial, but there are definite 
limits to this — e.g., in a serial computation a later operation may 
depend on an earlier one and hence cannot proceed simultaneously 
with it. Moreover, this choice affects other aspects of the machine, 
particularly the memory requirements, for a datum that is to be 
operated on later must be stored until it is needed. The memory of an 
artificial automaton is generally organized in a hierarchy, different 
levels of the hierarchy operating at different speeds. In a typical com- 
puter there are high-speed electronic registers, slower speed magnetic 
cores, and much slower magnetic tape units. In addition there is the 
wiring of the machine itself, which provides the unalterable organiza- 
tion of the system. Von Neumann discussed machine memory hier- 



24 



THEORY OP SELF-REPRODUCING AUTOMATA 



archies, and said that we should look for similar hierarchies in natural 
automata. Pulses circulating in neuron cycles, the change of neural 
thresholds with use, the organization of the nervous system, and the 
coding of the genes together constitute such a hierarchy. 

The organization of an automaton is to be distinguished from the 
organization of a particular computation in that automaton. When 
both are taken into account, the difference between natural and 
artificial automata with respect to serial vs. parallel operation seems 
to be accentuated. Von Neumann spoke in this connection of the 
"logical depth" of a computation. 32 A computation consists of a large 
number of basic logical steps (switching and delay), the result of each 
step depending on certain prior steps. We will call any sequence of 
steps, each of which depends critically on its predecessor, a "calcula- 
tion chain." The logical depth of a computation is the number of 
logical steps in its longest calculation chain. Because of their great 
speed, digital computers are used to perform computations of exceed- 
ingly great logical depth. For the final answer to be useful its error 
must be kept small, and this results in a very strong reliability require- 
ment on each logical step. This brings us to von Neumann's fifth and 
last main point of comparison between natural and artificial automata. 

(5) The first electronic digital computers had little equipment for 
the automatic detection of failure. They were designed and wired with 
extreme care and were constructed of components especially selected 
for great reliability. Programs were written with care and laboriously 
checked. Diagnostic programs were used to detect machine errors, and 
various procedures (e.g., differencing) were employed to check the 
computed results. Thus these machines were designed, built, and used 
in such a way that, hopefully, a single malfunction would be noted 
before a second occurred. The machine would then be stopped and the 
fault isolated by an analytic procedure. As von Neumann pointed out 
in the Fourth Lecture of Part I, this method of handling errors would 
obviously not be satisfactory for extremely complicated automata. 
The very design and construction of such large automata would 
result in many mistakes. Moreover, the large number of components 
would result in a very short mean free path between errors and make 
localization of failures too difficult. Natural automata are clearly 
superior to artificial ones in this regard, for they have strong powers of 
self -diagnosis and self -repair. For example, the human brain can suffer 
great damage from mechanical injury or disease and still continue to 

32 The Computer and the Brain, pp. 27, 79. He also spoke of the logical depth 
of a language. See ibid., pp. 81-82 and the discussion of the primary language 
of the nervous system at p. 15 above. 



editor's introduction 



25 



function remarkably well. Natural and artificial automata are thus 
organized in very different ways for protection against errors. Von 
Neumann's work on reliability serves to link these two types of autom- 
ata in this respect. 

Mathematics of Automata Theory. Von Neumann intended the 
theory of automata to be highly mathematical and logical. The study 
of actual automata, both natural and artificial, and of their operation 
and interaction, provides the empirical source of this formal compo- 
nent of automata theory. This is in keeping with von Neumann's 
belief that mathematics derives inspiration and ideas from empirical 
subject matter. 

The close connection between mathematical logic and automata 
was well known to von Neumann when he wrote on automata theory. 
Kurt Godel had reduced mathematical logic to computation theory 
by showing that the fundamental notions of logic (such as well-formed 
formula, axiom, rule of inference, proof) are essentially recursive 
(effective). 33 Recursive functions are those functions which can be 
computed on Turing machines, and so mathematical logic may be 
treated from the point of view of automata. 34 Conversely, mathe- 
matical logic may be applied to the analysis and synthesis of autom- 
ata. The logical organization of an automaton can be represented 
by a structure of idealized switch-delay elements and then translated 
into logical symbolism. See the Second Lecture of Part I. 

Because of the intimate connection between automata and logic, 
logic will be at the heart of the mathematics of automata theory. 
Indeed, von Neumann often spoke of a "logical theory of automata" 
rather than merely a "theory of automata." Nevertheless, he felt that 
the mathematics of automata theory would also have some formal 
characteristics very different from those of logic. Roughly speaking, 
mathematics can be divided into the discrete and the continuous. 
Logic is a branch of discrete mathematics and is highly combinatorial. 
Von Neumann thought that automata mathematics should be closer 
to the continuous and should draw heavily on analysis. He thought 
that the specific problems of automata theory require this, and he felt 
that there is a general advantage in an analytical as opposed to a 
combinatorial approach in mathematics. 

There is an important topic in the theory of automata that requires 

33 "Uber formal unentscheidbare Satze der Principia Mathematica und 
verwandter Systeme I." The notion of theoremhood is not in general recur- 
sive, but the theorems of a formal language are always recursively enumerable. 

34 Turing, "On Computable Numbers, with an Application to the Entschei- 
dungsproblem" and "Computability and X-Definability." 



26 



THEORY OF SELF-REPRODUCING AUTOMATA 



a more analytical treatment than is usual in logic. Automata theory 
must encompass the probability of failure of a component. Mathe- 
matical logic treats only the perfect or deterministic operation of 
idealized switch-delay elements; it provides no theoretical treatment 
of error. Hence in using mathematical logic for actual design one must 
supplement it by considerations that lie outside the subject itself. 
Von Neumann wanted a probabilistic logic which would handle com- 
ponent malfunction as an essential and integral part of automata 
operation. While probability theory is strongly combinatorial, it also 
makes important contacts with analysis. 

Including the probability of failure in the logic of automata forces 
one to consider the size of a computation. The usual approach in 
mathematical logic is to consider whether or not something can be 
accomplished by an automaton in a finite number of steps, regardless 
of how large the number is. But on any realistic assumption about 
component failure, the larger the calculation the more likely the 
machine will err during it, and the less Hkely the result will be correct. 
This concern for the size of a computation also arises from our practi- 
cal interests in automata. Computers are built in order to produce 
certain results in the available time. Since many of the functions we 
desire computers to perform are now performed by humans, it should 
be kept in mind in this connection that man is a finite automaton, not 
a Turing machine. Von Neumann did not suggest how to construct a 
theory of the sizes of computations. Presumably this theory would 
be based on a quantitative notion of " amount of computation' ' which 
would take into account both the length of a calculation (the "logical 
depth" of p. 24 above) and its width (the amount of parallelism in it). 

Thus a theory of the quantity of computation and the likelihood of 
its being wrong must include continuous as well as discrete mathe- 
matics. 

All of this will lead to theories which are much less rigidly of an all-or-none 
nature than past and present formal logic. They will be of a much less com- 
binatorial, and much more analytical, character. In fact, there are numerous 
indications to make us believe that this new system of formal logic will move 
closer to another discipline which has been little linked in the past with logic. 
This is thermodynamics, primarily in the form it was received from Boltz- 
mann, and is that part of theoretical physics which comes nearest in some of 
its aspects to manipulating and measuring information. Its techniques are 
indeed much more analytical than combinatorial, which again illustrates the 
point that I have been trying to make above. 85 

36 "The General and Logical Theory of Automata, " Collected Works 5.304. 
The next quotation is from the same article, 5.303. 



editor's introduction 



21 



Von Neumann also held that there is a methodological advantage 
in employing analysis in the mathematics of automata. 

Everybody who has worked in formal logic will confirm that it is one of the 
technically most refractory parts of mathematics. The reason for this is that 
it deals with rigid, all-or-none concepts, and has very little contact with the 
continuous concept of the real or of the complex number, that is, with mathe- 
matical analysis. Yet analysis is the technically most successful and best- 
elaborated part of mathematics. Thus formal logic is, by the nature of its 
approach, cut off from the best cultivated portions of mathematics, and 
forced onto the most difficult part of the mathematical terrain, into combina- 
torics. 

This comment is particularly significant since von Neumann made 
important contributions to discrete mathematics. In Theory of Games 
and Economic Behavior he stated that the mathematics to be developed 
for social theory should emphasize combinatorics and set theory 
rather than differential equations. 36 

In his own work in automata theory von Neumann moved from the 
discrete toward the continuous. His probabilistic logic is an example. 
After presenting this logic, he proposed a mixed analog-digital com- 
puting system closely related to it. 37 His first models of self -reproduc- 
tion were discrete, but he hoped later to develop a continuous model 
of self -reproduction. See Section 1.1.2.3 of Part II of the present 
volume. 

We noted before that von Neumann often referred to his theory of 
automata as a "logical theory of automata. " He also called it "theory 
of automata and information'' and sometimes just "theory of infor- 
mation," indicating the strong role that he expected information 
theory to play in the subject. He divided the theory of control and 
information into two parts: a strict part and a probabilistic part. The 
rigorous or strict part includes mathematical logic as extended to 
cover finite automata and Turing machines. The statistical or proba- 
bilistic part includes the work of Shannon on information theory 38 and 
von Neumann's probabilistic logic. Von Neumann regarded his 
probabilistic logic as an extension of rigorous logic. 

There is a close connection between information theory and thermo- 
dynamics, both subjects employing the concept of probability in very 
much the same way. See the Third Lecture of Part I, especially the 
quotation from von Neumann's review of Wiener's Cybernetics. 

36 Sec. 4.8.3. Cf. Sec. 1.2.5. 

37 Sec. 12 of "Probabilistic Logics and the Synthesis of Reliable Organisms 
from Unreliable Components," Collected Works 5.372-377. 

38 "A Mathematical Theory of Communication." 



28 



THEORY OF SELF-REPRODUCING AUTOMATA 



Von Neumann mentioned two further connections between ther- 
modynamics and automata theory. First, he found an analog of ther- 
modynamic degeneration in the theory of self -reproducing automata: 
below a certain minimum level, complexity and degree of organiza- 
tion are degenerative, but above that level they are not degenerative 
and may even increase. Second, he discussed the thermodynamic 
aspect of the concept of balance in computing machine design. The 
efficiency of a computer depends on the proper balance of its different 
parts with respect to speed and size. For example, in the memory 
hierarchy the different kinds of memory (e.g., transistor, core, tape) 
should be matched to one another in size and speed. A computer in 
which the arithmetic unit is too fast for the memory, or the memory is 
too small, is like a heat engine which is inefficient because large tem- 
perature differences exist between two parts of it. The efficiency of a 
computer must be defined relative to its environment (i.e., the prob- 
lems it is to solve), just as the efficiency of a heat engine depends on 
its environment. These problems of balance and matching are handled 
empirically by engineers. Von Neumann wanted a quantitative theory 
of balance akin to thermodynamics. 

To conclude, von Neumann thought that the mathematics of auto- 
mata theory should start with mathematical logic and move toward 
analysis, probability theory, and thermodynamics. When it is devel- 
oped, the theory of automata will enable us to understand automata 
of great complexity, in particular, the human nervous system. Mathe- 
matical reasoning is performed by the human nervous system, and the 
"primary" language in which mathematical reasoning takes place is 
analogous to the primary language of a computing machine (p. 
15 above). It is thus quite possible that automata theory will 
affect logic and our fundamental concepts of mathematics. 

I suspect that a deeper mathematical study of the nervous system . . . will 
affect our understanding of the aspects of mathematics itself that are in- 
volved. In fact, it may alter the way in which we look on mathematics and 
logics proper. 39 

Now logic lies at the foundation of mathematics; therefore, if von 
Neumann's suggestion is true, automata theory will move full circle: 
starting at the foundation of mathematics and ending there. 

Arthur W. Burks 

39 The Computer and the Brain, p. 2; cf. pp. 70-82. See also Ulam, "John 
von Neumann, 1903-1957," p. 12. 



part One 



Theory and Organization of 
Complicated Automata 



EDITORIAL NOTE 



[The editor's writing is in brackets. The reconstructed edition 
of von Neumann's work is not bracketed, but much of the unbrack- 
eted text is heavily edited. See the Preface.] 



First Lecture 



COMPUTING MACHINES IN GENERAL 



Conceptual and numerical methods in mathematics. The role of the latter 
in applied mathematics and in mathematical physics. Their role in pure 
mathematics. The situation in analysis. Numerical procedures as heuristic 
tools. 

Various forms of the numerical approach: Analog and digital. 

The analog procedure : The use of the physical experiment as a substitute 
for computing. Analog computing machines. 

The digital procedure: Manual computing. Simple machines. Fully auto- 
matic computing. 

The present status of computing machines. Present roles of analog and 
digital machines. Questions of speed, programming, and precision. 

The concept of an elementary operation in a computing machine. Its role 
in analog machines and in digital machines. Observations on analog com- 
ponentry. Observations on digital componentry. 

The relay organ. Main forms: The electro-mechanical relay. The vacuum 
tube. Other possible relay organs. 

Measurement of the length or complexity of a numerical calculation. 
Logical and arithmetical operations. Linear and non-linear arithmetical 
operations. The role of the number of multiplications. Stability of the statis- 
tical characteristics of various parts of mathematics. The special role of 
analysis. 

Various characteristic levels of length or complexity. Characteristic 
problem lengths for automatic digital machines. 
Precision requirements. 

Memory requirements: Measurement of memory capacity. The decisive 
characteristics of a memory: Access time and capacity. Reasons for a hier- 
archic organization of memory. Actual memory requirements of an automatic 
digital machine. 

Input-output: Main available media. 

The concept of balance: Speed balance of various components. Balance 
between memory capacity in various stages of the hierarchy and speeds. 
Balance between speed and precision. Balance between speed, memory 
capacity, and programming capacity. 

Thermodynamical aspects of the concept of balance. Thermodynamical 
aspects of the memory capacity. Need for a quantitative theory, contrasted 



31 



32 



THEORY OF SELF-REPRODUCING AUTOMATA 



with the present empirical procedures. Preliminary remarks on reliability and 
errors. 

Ladies and gentlemen, I wish to thank you for your very friendly 
welcome on my five occasions to talk to you, and I hope that I will be 
able to offer something for the variety of interests which are repre- 
sented here. I will talk about automata — the behavior of very com- 
plicated automata and the very specific difficulties caused by high 
complication. I shall discuss briefly the very plausible, very obvious 
analogies which come to mind between artificial automata and or- 
ganisms, which within a certain limit of their functioning are natural 
automata. We must consider the similarities, the dissimilarities, the 
extent to which the dissimilarities are due to our skill or clumsiness 
(the latter being the more normal phenomenon), and the extent to 
which these dissimilarities are really matters of principle. 

Today I will talk chiefly about artificial automata, and specifically 
about one variety of artificial automata, namely, computing machines. 
I will talk about their role in the near past and present and about what 
to expect from them in the future. 

I am talking about computing machines partly because my interests 
in the subject of automata are mathematical and, from the mathe- 
matical point of view, computing machines are the most interesting 
and most critical automata. But quite apart from this ex parte argu- 
ment from the mathematical side, there is the important question of 
automata of very, very high complexity. Of all automata of high com- 
plexity, computing machines are the ones which we have the best 
chance of understanding. In the case of computing machines the 
complications can be very high, and yet they pertain to an object 
which is primarily mathematical and which we understand better 
than we understand most natural objects. Therefore, by considering 
computing machines, we can discuss what we know and what we do 
not know, what is right and what is wrong, and what the limitations 
are, much more clearly than if we discussed other types of automata. 
You will see that our discussion of complex automata is very far from 
perfect and that one of our main conclusions is that we need very 
badly a theory which we do not at this moment possess. 

Let me first say something from the properly mathematical side, 
namely, the role which computing machinery has played or might play 
in mathematics and in adjacent subjects. Speaking of numerical com- 
puting in general, it is not necessary to discuss what role it can play 
in many applications of mathematical methods. It's perfectly clear 
that numerical computing plays a large role in engineering. If more 



COMPUTING MACHINES 



33 



computing and faster computing could be done, one would have even 
more uses for computing in engineering. 

Let me come now to the less obvious things. In physics, particularly 
in theoretical physics, it is clear that mathematical methods play a 
great role and that to a large extent they are of the same variety as 
pure mathematics, that is, they are abstract and analytical. However, 
effective computing plays a role in physics which is larger than the 
role one would expect it to have in mathematics proper. For instance, 
there are large areas of modern quantum theory in which effective 
iterative computing could play a large role. A considerable segment of 
chemistry could be moved from the laboratory field into the purely 
theoretical and mathematical field if one could integrate the applicable 
equations of quantum theory. Quantum mechanics and chemistry 
offer a continuous spectrum of problems of increasing difficulty and 
increasing complexity, treating, for example, atoms with increasing 
numbers of electrons and molecules with increasing numbers of 
valence electrons. Almost any improvement in our standards of com- 
puting would open important new areas of application and would 
make new areas of chemistry accessible to strictly theoretical methods. 

However, I will not go into great detail on this subject either but 
would like to give you a brief indication of what role this kind of com- 
puting might play in mathematics proper, that is, in pure mathe- 
matics. In pure mathematics the really powerful methods are only 
effective when one already has some intuitive connection with the 
subject, when one already has, before a proof has been carried out, 
some intuitive insight, some expectation which, in a majority of cases, 
proves to be right. In this case one is already ahead of the game and 
suspects the direction in which the result lies. A very great difficulty 
in any new kind of mathematics is that there is a vicious circle: you 
are at a terrible disadvantage in applying the proper pure mathe- 
matical methods unless you already have a reasonably intuitive 
heuristic relation to the subject and unless you have had some sub- 
stantive mathematical successes in it already. In the early stages of 
any discipline this is an enormous difficulty; progress has an auto- 
catalytic feature. This difficulty may be overcome by some excep- 
tionally lucky or exceptionally ingenious performance, but there are 
several outstanding instances where this has failed to happen for two, 
three, or four generations. 

One of these areas which has been conspicuous for some time is the 
area of non-linear problems. The great successes of the nineteenth 
century, as well as of modern analysis, were in linear problems. We 
have much less experience with non-linear problems, and we can say 



34 



THEORY OF SELF-REPRODUCING AUTOMATA 



practically nothing about the majority of non-linear partial differ- 
ential equations. We have never been successful with these at all, and 
therefore we have absolutely no idea what the difficulties are. 

In those few domains where some progress had been made it was 
usually for different reasons, for instance, because some very usual 
physical phenomenon was tied up with the mathematical problems 
and therefore one had a non-mathematical, physical approach. In 
these domains scientists discovered the most surprising types of sin- 
gularities, which have absolutely no analogs in the linear domain we 
know so well, that is, absolutely no analogs in those parts of mathe- 
matical analysis like complex number theory, and so on. These experi- 
ences make a fairly convincing case that completely new methods will 
be needed for non-linear problems. The classical example for this is a 
non-linear partial differential equation for compressible, non-viscous 
flow, which led to the discovery of the phenomenon of shocks. In a 
problem in which it seemed that only continuous solutions should 
exist, discontinuous solutions suddenly play an enormous role, and 
without proper regard for these one cannot prove the uniqueness or 
the existence of solutions. Furthermore, these irregular solutions be- 
have in a very peculiar manner and violate a number of the regulari- 
ties which we had reason to believe, from other forms of analysis, were 
well established. 

Another good example is the phenomenon of turbulence in the 
viscous case, where one suddenly discovers that the really important 
solutions to a problem which has very high symmetry do not possess 
that symmetry. From a heuristic point of view, the important thing 
is not to find the simplest solution of the problem, but rather to ana- 
lyze statistically certain large families of solutions which have nothing 
in common with each other except certain statistical traits. These 
prevalent statistical traits are the real roots of the problem and cause 
very peculiar singularities in many individual solutions. In all these 
cases there is reason to believe that we will have great difficulty in 
making analytical progress. The problem of turbulence has been 
around for 60 years, and analytical progress in solving it has been very 
small. 1 

Almost all of the correct mathematical surmises in this area have 
come in a very hybrid manner from experimentation. If one could 
calculate solutions in certain critical situations like those we have 
mentioned, one would probably get much better heuristic ideas. I 

1 [ See further von Neumann's Collected Works 5.2-5 and Birkhoff's Hydro- 
dynamics.] 



COMPUTING MACHINES 



35 



will try to give some indications of this later, but I wanted to point 
out that there are large areas in pure mathematics where we are 
blocked by a peculiar inter-relation of rigor and intuitive insight, each 
of which is needed for the other, and where the unmathematical 
process of experimentation with physical problems has produced 
almost the only progress which has been made. Computing, which is 
not too mathematical either in the traditional sense but is still closer 
to the central area of mathematics than this sort of experimentation 
is, might be a more flexible and more adequate tool in these areas than 
experimentation. 

Let me come to the subject proper and first say a few things about 
the general traits of computing processes and computing machines. 
As you probably know, the main types of computing machines exist- 
ing or being discussed or planned at this moment fall into two large 
classes: super-analog devices and digital devices. Let me first de- 
scribe the analog devices or the wider class, inasmuch as a proper 
definition is usually given for the digital class, and analogs are essen- 
tially everything else. 

Roughly speaking, an analog calculation is one in which you look 
at some physical process which happens to have the same mathemati- 
cal equations as the process you're interested in, and you investigate 
this physical process physically. You do not take the physical process 
which you are interested in, because that is your whole reason to cal- 
culate. You always look for something which is like it but not exactly 
the same thing. 

The smallest modification you may make is to use a different scale, 
which is possible in certain problems. A slightly larger modification 
is to use a different scale and also change certain things which are not 
exactly scales. For instance, when you try an aerodynamical experi- 
ment in a wind tunnel you scale it, but you scale not only the linear 
dimensions but also the velocity of sound. The only way to scale the 
velocity of sound is to go to a lower temperature, and there you really 
need insight. You must know that the phenomenon you're concerned 
with does not depend on temperature. You then discover that it is 
easier to try it at a lower temperature and with much smaller dimen- 
sions than to carry out the actual process you are interested in. In this 
way a wind tunnel for aerodynamical experimentation is in a sense an 
analog computing device. This is not a completely fair comparison 
because a wind tunnel does a good deal beside computing, but still in 
a large range of application (which is certainly not much less than 50 
per cent) it is just an analog computing device. 

You come very quickly then to cases in which you will not do 



36 



THEORY OF SELF-REPRODUCING AUTOMATA 



exactly this because it is not possible or not convenient to find a physi- 
cal process which has exactly the same equations as the problem you 
are interested in. But you may still find, for example, three different 
processes which have the same equations as three parts of the problem 
and which can be aligned in such a manner that if you perform one 
after the other you get the complete answer. From this there is a con- 
tinuous transition to situations where you actually break up the 
problem mathematically into the elementary operations of arithmetic: 
multiplication, addition, subtraction, and division. 

[ Von Neumann next discussed physical analog processes for adding, 
multiplying, subtracting, and dividing. He covered both electrical and 
mechanical analog processes, and the way numbers are represented in 
each. He said, "What passes in any axiomatic treatment of mathe- 
matics for the elementary operations of arithmetic, the four species 
of arithmetical addition, subtraction, etc., need not be the elementary 
operations of a computing machine, specifically of an analog com- 
puting machine." He explained how a differential analyzer multiplies 
two constants by integrating and subtracting. See The Computer and 
the Brain 8-5, Collected Works 5.293.] 

[ Von Neumann then took up digital machines. He remarked that 
in the last 10 years purely digital devices had become relatively much 
more important than analog devices. He discussed the components of 
digital machines (toothed wheels, electromechanical relays, vacuum 
tubes, and nerve cells), the speeds of these components (including both 
response time and recovery time), and the need for power amplifica- 
tion in these components. He stressed the role of the basic logical 
operations (such as sensing a coincidence) in control mechanisms, 
including "the most elaborate control mechanism known, namely, the 
human nervous system." See The Computer and the Brain 7-10, 30, 
39-47. He next turned to the problem of measuring the complexity 
of automata.] 

It is not completely obvious how to measure the complexity of an 
automaton. For computing machines, probably the reasonable way is 
to count how many vacuum tubes are involved. This is somewhat 
ambiguous, because certain current types of vacuum tubes are in 
reality two vacuum tubes inside one envelope, in which case one is 
never quite sure which one of the two he is talking about. Another 
reason is that a great deal enters into computing machine circuitry 
aside from vacuum tubes: electrical equipment like resistors, capaci- 
tances, and possibly inductances. Nevertheless, the ratio of these to 
the vacuum tubes is tolerably constant, and therefore the number of 
tubes is probably a reasonable measure of complexity. 



COMPUTING MACHINES 



37 



The largest calculating machine ever used to date contains twenty 
thousand vacuum tubes. 2 Now the design of this machine is very 
different from what any vacuum tube machine of the future is likely 
to be like, and so this machine is not quite typical. The computing 
machines which most people are thinking about as the computing 
machines of the immediate future are to be smaller than this, probably 
having 2 to 5 thousand tubes. So, roughly speaking, the order of 
magnitude of the complexity of these machines is 10 thousand. 

To give you a comparison with natural organisms, the number of 
nerve cells in a natural organism can be very different from this. The 
number of nerve cells in the human central nervous system has been 
estimated to be 10 billion. This number is so large that of course one 
has absolutely no experience with such orders of magnitude. It's 
terribly difficult to form any reasonable guess as to whether things 
which are as complex as the behavior of a human being can or cannot 
be administered by 10 billion switching organs. No one knows exactly 
what a human being is doing, and nobody has seen a switching organ 
of 10 billion units; therefore one would be comparing two unknown 
objects. 

Let me say a few things which relate more specifically to computing 
machines. If you can repeat an elementary act like switching with a 
vacuum tube 1 million times per second, that does not mean of course 
that you will perform anything that is mathematically relevant 1 
million times per second. In estimating how fast a computing machine 
can operate, there are all kinds of standards. There's a reasonable 
agreement that one ought to count the number of multiplications 
performed in a second. By multiplications I mean the multiplication 
of two full sized numbers with the precision with which the machine 
is running. There is good reason to believe that the precision with 
which these things ought to run is of the order of 10, 12, or 14 decimal 
digits. A machine of reasonable design in which the elements have a 
speed of about 1 million per second will probably multiply somewhere 
in the neighborhood of 1 millisecond. 

No matter how you organize a computing machine, you simply 
cannot count on using it to get 100 per cent efficiency. By that I mean 
that it's impossible, with our present information on the subject, to 
organize the machine in such a manner that a multiplier which can 
multiply in one thousandth of a second will really be fed the necessary 

2 [ This is the ENIAC, which is described in Burks, "Electronic Computing 
Circuits of the ENIAC" and "Super Electronic Computing Machine," Gold- 
stine and Goldstine, "The Electronic Numerical Integrator and Computer 
(ENIAC)," and Brainerd and Sharpless, "The ENIAC."] 



38 



THEORY OF SELF-KEPRODUCI N( J AUTOMATA 



data for multiplication. There is a good deal else to be done, namely, 
making up your mind what numbers you want, getting the numbers, 
disposing of the result, deciding whether you should do the same thing 
once again or whether you should do something else, and so on. This 
is remarkably similar to paper pushing and adding on paper directly, 
except that what you're pushing is not on paper. 

From a logical point of view the efficiency is probably of the order 
of 1 in 10 or a little better. By that I mean that in any reasonable, 
logical description of what you are doing, in a code which corresponds 
to prevalent procedures in formal logics, somewhere between a fifth 
and a tenth of the orders will be orders to multiply. Since multiplica- 
tion is somewhat slower than the other operations, in what most 
people think is a well integrated, well balanced machine, you will 
probably spend something like one quarter to one half of the time 
multiplying. So, if you have a multiplier which can effect a multiplica- 
tion in 1 millisecond, you are doing fairly well to get 500 multiplica- 
tions per second. 

In human computing aided by a desk machine the same number will 
be perhaps, 2 multiplications per minute. So the discrepancy, the 
acceleration factor could probably be pushed to 100 thousand or 
something like that. But to get out of this range we'll probably have 
to depart from present techniques quite radically. 

From the mathematical point of view the question arises whether 
anything could be done with this speed if one had it. I would like to 
point out very emphatically that there are very good reasons for 
asking for anything the traffic will bear, for this speed, 10 times more, 
a hundred times, a thousand times, or a million times. Problems there 
are good reasons to solve would justify a great deal more speed than 
anyone can think of at this moment. [Von Neumann gave as examples 
quantum mechanical calculations on atomic and molecular wave 
functions (where the combinatorial difficulties go up very fast as 
the number of electrons goes up) , and the problem of turbulence.] 

Although it doesn't belong strictly to the subject, let me point out 
that we will probably not want to produce vast amounts of numerical 
material with computing machines, for example, enormous tables of 
functions. The reason for using a fast computing machine is not that 
you want to produce a lot of information. After all, the mere fact that 
you want some information means that you somehow imagine that 
you can absorb it, and, therefore, wherever there may be bottlenecks 
in the automatic arrangement which produces and processes this 
information, there is a worse bottleneck at the human intellect into 
which the information ultimately seeps. 



COMPUTING MACHINES 



39 



The really difficult problems are of such a nature that the number of 
data which enter is quite small. All you may want to know is a few 
numbers, which give a rough curve, or one number. All you may want 
in fact is a "yes" or a "no," the answer as to whether something is or 
is not stable, or whether turbulence has or has not set in. The point is 
that you may not be able to get from an input of, say, 80 numbers to 
an output of 20 numbers without having, in the process, produced a 
few billion numbers in which nobody is interested. But the process is 
such that the volume of numerical material handled first expands and 
then contracts again, and, while it starts on a low level, say with 100 
numbers, and ends on a low level, say with 10 numbers, its maximum 
in between is large, say a few thousand, and the number of successive 
generations is large, so that you have handled 10 billion numbers 
before you are through. These figures are quite realistic; it would be 
easy to find problems which have about this numerical makeup. 

You may have noticed that I have already introduced one distinc- 
tion, namely, the total numerical material produced in a process. The 
other thing which matters is how much you need simultaneously. 
This is probably the most vexing problem in modern computing 
machine technology. It's also quite a problem from the point of view 
of the human organism, namely, the problem of memory. You see, 
all these automata really consist of two important parts: the general 
switching part (an active part which affects the logical operations 
the automaton is supposed to perform), and the memory (which 
stores information, chiefly intermediate results which are needed for 
a while and are then discarded and replaced by others) . 

In computing machines, the methods to do the active part, the 
arithmetical and control circuits, have been well known for years. 
The memory questions were much more critical and much more open 
throughout the last decade, and are even more critical and more open 
now. In the human organism, we know that the switching part is 
composed of nerve cells, and we know a certain amount about their 
functioning. As to the memory organs, we haven't the faintest idea 
where or what they are. We know that the memory requirements of 
the human organism are large, but on the basis of any experience 
that one has with the subject, it's not likely that the memory sits in 
the nervous system, and it's really very obscure what sort of thing it 
is. 3 Hence in both the computer and the human nervous system, the 
dynamic part (the switching part) of the automaton is simpler than 
the memory. 

3 [ This point is discussed further in The Computer and the Brain 63-69.] 



40 



THEORY OF SELF-REPRODUCING AUTOMATA 



[ Von Neumann next discussed how to measure memory capacity. 
He suggested using the logarithm (to the base two) of the configura- 
tion number (i.e., the number of alternatives). See Collected Works 
5.341-342. He then estimated the memory capacity of an ordinary 
printed page to be about 20 thousand units, and remarked that this 
is about the memory capacity of the digital computers under con- 
sideration at that time.] 

This shows where the real price of speed lies. A large modern com- 
puting machine is a very expensive object, an object which it takes 
a long time to build and which is a very tricky thing after you've got 
it. Yet it is supposed to get along on a memory which is equivalent to 
a printed page! When such a machine is properly run it will, in half 
an hour, do the work which a computing group of 20 people would do 
in about 2 or 3 years. Yet it's supposed to get along on a memory of 
one printed page. Imagine that you take 20 people, lock them up in 
a room for 3 years, provide them with 20 desk multipliers, and in- 
stitute this rule: during the entire proceedings all of them together 
may never have more than one page written full. They can erase any 
amount, put it back again, but they are entitled at any given time 
only to one page. It's clear where the bottleneck of this process lies. 
The planning may be difficult, input and output may be cumbersome, 
and so on, but the main trouble is that it has a phenomenally low 
memory for the computing to be done. The whole technique of com- 
puting will be completely distorted by this modus operandi. 

This is an extremely abnormal economy. By going to high speed,'- 
you cut yourself off from the efficient methods of storing information 
and push yourself into an inefficient one. A thousand-number com- 
puter memory is a very large object, an object which it took years 
to develop ; all existing types are very much in an experimental stage 
at this moment, and none of them are small or cheap. Yet they are 
the equivalent of one printed page. The reason why one is forced to 
use these memories is this. [Each multiplication requires certain num- 
bers from the memory and the product often goes to the memory. 
There are other arithmetic operations, and these require access to 
the memory. The orders to control these arithmetic operations come 
from the memory.] One probably needs anywhere between five and 
eight accesses to the memory for each multiplication. Thus it is un- 
reasonable to get a millisecond multiplier unless you have a memory 
to which you can get access in something of the order of a tenth of 
a millisecond. Now to get access to a printed book takes seconds, to 
get access to anything punched or written on paper takes a fraction 
of a second. Since one needs an access time of something like one 



COMPUTING MACHINES 



II 



ten-thousandth of a second, one is forced out of these efficient tech- 
niques of storing information, into a highly inefficient and expensive 
technique. 

In comparing artificial with natural automata there is one very 
important thing we do not know: whether nature has ever been sub- 
ject to this handicap, or whether natural organisms involve some 
much better memory device. That the secondary memory devices 
which humans have developed, namely, libraries, etc., are so vastly 
more efficient than this, is some reason to suspect that natural mecha- 
nisms for memory may also be quite as clumsy as the high speed 
memories with which we think we have to operate. But we know 
practically nothing about this. 

Let me, before I close today, mention one more thing. In any fast 
machine the memory you need is characterized by two data: the 
capacity and the access time. [Von Neumann said that at that mo- 
ment there was no technique for building a memory with both an 
adequate capacity and a sufficiently good access time. What is done 
is to construct a hierarchy of memories. The first memory has the 
required speed and is as large as you can make it, but it is not large 
enough. The second memory is much larger, but slower. Numbers 
are transferred from the second memory to the first memory when 
needed. There may be a third memory which is larger but slower, and 
so on. An electrostatic memory tube, a magnetic tape, and a card 
file would constitute such a hierarchy of memories. See The Computer 
and the Brain 33-37.] 



Second Lecture 



RIGOROUS THEORIES OF CONTROL 
AND INFORMATION 



Theory of information: The strict part. The concept of information. The 
corresponding mathematical-logical concept of sets and partitions. 

Close connection with formal logics. Alternative approach by way of 
model-automata. Common traits in these two approaches: All-or-none char- 
acter. The work which connects these two approaches. 

Methods of describing automata : Syntheses from components or treatment 
as a whole. 

Synthetic approach: Nature of the element-organs. Their similarity to 
neurons. The program of McCulloch and Pitts: Formal neural networks. Their 
main result. 

Treatment as a whole: Turing's theory of automata. The relationship of 
an automaton and of the mathematical problems that can be solved with its 
help. The concept of a universal automaton. Turing's main result. 

Limitations of the McCulloch-Pitts and Turing automata. Input and out- 
put organs. Generalizations of these. Interpretation as sensor and motor 
organs. 

[ Von Neumann said that there are two parts to information theory: 
the rigorous and the probabilistic. The probabilistic part is probably 
more important for modern computing machinery, but the rigorous 
part is a necessary preliminary to it. The rigorous part of information 
theory is just a different way of dealing with formal logics.] 

[ He then explained some of the basic ideas of formal logics. He dis- 
cussed briefly truth-functional connectives such as "and," "not," 
"if . . . then," and "not both," and their interdefinability. He ex- 
plained the idea of a variable and the quantifiers "all" and "some." 
He concluded: "If you have this machinery you can express anything 
that is dealt with in mathematics; or that is dealt with in any subject, 
for that matter, as long as it's dealt with sharply."] 

I am not going to go into this subject, because in order to make a 
theory of information, another machinery which is quite closely re- 
lated to this but looks somewhat different, is more cogent. This is con- 



42 



RIGOROUS THEORIES OF CONTROL AND INFORMATION 43 

nected with the work of McCulloch and Pitts, 1 on the one hand, and 
the logician Turing on the other. 2 Both endeavors in the subject 
replace formal logics as indicated here and as classically pursued, by 
the discussion of certain fictitious mechanisms or axiomatic paper 
automata, which are merely outlined, but which nobody is particularly 
concerned to build. Both of them show that their fictitious mecha- 
nisms are exactly co-extensional with formal logics; in other words, 
that what their automata can do can be described in logical terms 
and, conversely, that anything which can be described rigorously in 
logical terms can also be done by automata. [Von Neumann was as- 
suming that a finite McCulloch-Pitts neuron net is supplied with an 
infinite blank tape. The result to which he referred is the equivalence 
of Turing computability, X-definability, and general recursiveness. 
See Turing's "Computability and X-Definability."] 

I'm going to describe both the work of McCulloch and Pitts and 
the work of Turing, because they reflect two very important ways to 
get at the subject: the synthetic way, and the integral way. McCulloch 
and Pitts described structures which are built up from very simple 
elements, so that all you have to define axiomatically are the ele- 
ments, and then their combination can be extremely complex. Turing 
started by axiomatically describing what the whole automaton is 
supposed to be, without telling what its elements are, just by describ- 
ing how it's supposed to function. 

The work of McCulloch and Pitts was definitely meant as a simple 
mathematical, logical model to be used in discussions of the human 
nervous system. That it wound up with something which is actually 
an equivalent of formal logics is very remarkable and was part of the 
point McCulloch and Pitts wanted to drive home, but only part of 
that point. Their model also has a meaning which concerns me at this 
moment a little less, but about which I will tell, without immediately 
stating where it ties in to formal logics. They wanted to discuss 
neurons. They took the position that they did not want to get tied up 
in the physiological and chemical complexities of what a neuron really 
is. They used what is known in mathematics as the axiomatic method, 
stating a few simple postulates and not being concerned with how 
nature manages to achieve such a gadget. 

1 [ McCulloch and Pitts, "A Logical Calculus of the Ideas Immanent in 
Nervous Activity. " See also Sees. 1-7 of von Neumann, "Probabilistic Logics 
and the Synthesis of Reliable Organisms from Unreliable Components, " Burks 
and Wright, "Theory of Logical Nets," and Kleene, "Representation of Events 
in Nerve Nets and Finite Automata."] 

2 [ Turing, "On Computable Numbers, with an Application to the Entschei- 
dungsproblem."] 



it 



THEORY OF SELF-REPRODUCING AUTOMATA 



They went one step further. This has been emphasized very strongly 
by those who criticize their work, although it seems to me that the 
extent to which they went further can be justified. They said that they 
did not want to axiomatize the neuron as it actually exists, but they 
wanted to axio matize an i de alized neuron, w hictiJs-jjiu ch simple r 
than the_rSxl,_one. They believed that the extremely amputated, 
simplified, idealized object which they axiomatized possessed the 
essential traits bf the neuron, and that all else are incidental complica- 
tions, which in a first analysis are better forgotten. Now, I am quite 
sure that it will be a long t ime before this point is generally agreed to 
by everybody, if ever; namely, whether or not what one overlooks 
in this simplification had really better be forgotten or not. But it's 
certainly true that one gets a quick understanding of a part of the 
subject by making this idealization. 

The definition of what we call a neuron is this. One should perhaps 
call it a formal neuron, because it certainly is not the real thing, though 
it has a number of the essential traits of the real thing. A neuron will 
be symbolically designated by a circle, which symbolizes the body of 
the neuron, and a line branching out from the circle, which symbolizes 
the axon of the neuron. An arrow is used to indicate that the axon 
of one neuron is incident on the body of another. A neuron has two 
states: it's excited or not. As to what excitation is, one need not tell. 
Its main characteristic is its operational characteristic and that has a 
3ertain circularity about it: its main trait is that it can excite other 
neurons. Somewhere at the end of an involved network of neurons 
the excited neuron excites something which is not a neuron. For 
instance, it excites a muscle, which then produces physical motion; or 
it excites a gland which can produce a secretion, in which case you get 
a chemical change. So, the ultimate output of the excited state really 
produces phenomena which fall outside our present treatment. These 
phenomena will, for the sake of the present discussion, be entirely 
disregarded. 

[Von Neumann stated the axioms governing the interaction of 
neurons. Following McCulloch and Pitts he assumed a uniform delay 
and for the time being disregarded "the important phenomenon of 
fatigue, the fact that after a neuron has been excited it is not usable 
for a while. " Fatigue plays an important role in the functioning of an 
organism (see p. 48 below), but in spite of fatigue one can get con- 
tinuous action by using a chain of neurons, each feeding its successor. 
Von Neumann defined the threshold of a neuron and introduced 
inhibitory synapses, symbolized by a circle (instead of an arrowhead).] 

[Von Neumann next presented what he called "the important 



RIGOROUS THEORIES OF CONTROL AND INFORMATION 45 

result of McCulloch and Pitts." Imagine a black box with a number 
of inputs and a single output. Select two times, ti and t 2 . Specify 
which patterns of inputs from time t\ to time U are to produce an 
output and which are not.] No matter how you formulate your condi- 
tions, you can always put a neural network in the box which will 
realize these conditions, which means that the generality of neural 
systems is exactly the same as the generality of logics. The fact that 
something has been done with the system means not more and not 
less than you know what you are talking about, that you can state it 
in a finite number of words unambiguously and rigorously. I will not 
attempt to give the proof, which like all proofs in formal logics is not 
quite easy to render. [We will sketch the proof very briefly. It follows 
from the construction that von Neumann referred to that every switch- 
ing function (truth function, Boolean function) can be realized by a 
neural network with some fixed delay. A cyclic neural memory of 
arbitrary finite capacity can be attached to this switching net. When 
this composite network is augmented by an infinite tape, the result is 
a Turing machine. Moreover, corresponding to each Turing machine 
M , there is a network of this kind which computes the same number 
as M.] 

[ Von Neumann showed how to construct a few sample networks. 
The first is a network in which a dominates 6, b dominates c, but c 
dominates a, shown in Figure 1. Each neuron is excited (fires) if it is 
stimulated on its excitatory input (the input with an arrow) but is 
not stimulated on its inhibitory input (the input with a small circle) . 
Hence if a and b are both stimulated, output a will be active but not 
/5; if b and c are both stimulated, output 0 will be active but not 7; 
while if a and c are both stimulated, output 7 will be active but not a. 
Von Neumann used this network to illustrate a certain point. People 
have made statements about the non-quantitative character of 
human behavior, which statements seem to imply that in any quanti- 
tative mechanism, if a is stronger than b and b is stronger than c, 
then a is stronger than c. But in the above neural network a is stronger 
than b and b is stronger than c, while c is stronger than a.] 

[ Von Neumann then synthesized a number of other networks : 
simple memories, counters, and an elementary learning circuit. These 
are approximately the circuits of Collected Works 5.342-345. The 
learning circuit has two inputs, a and 6. It counts the number of times 
a stimulus of a is followed by a stimulus of b. When this number 
reaches 256, the circuit emits a pulse whenever b is stimulated, inde- 
pendently of whether a is stimulated or not.] You see that you can 
produce circuits which look complicated, but which are actually quite 



4G THEORY OF SELF-REPRODUCING AUTOMATA 

simple from the point of view of how they are synthesized and which 
have about the same complexity that they should have, namely, the 
complexity that grammar has. It is no more difficult to make this 
drawing up than to make up a sentence which describes what you 
want, and the essence of the result of McCulloch and Pitts is that 
there really isn't much difference between the two things. The rigorous 
verbal description is co-extensive with the description in terms of 
relay organs. 

May I point out what follows from this from a philosophical point 
of view, and what does not follow. It certainly follows that anything 
that you can describe in words can also be done with the neuron 
method. And it follows that the nerves need not be supernaturally 
clever or complicated. In fact, they needn't be quite as clever and 
complicated as they are in reality, because an object which is a con- 
siderably amputated and emasculated neuron, which has many fewer 
attributes and responds in a much more schematic manner than a 
neuron, already can do everything you can think up. 

What is not demonstrated by the McCulloch and Pitts result is 
equally important. It does not prove that any circuit you are designing 
in this manner really occurs in nature. It does not follow that the other 
functions of the nerve cell which have been dropped from this descrip- 
tion are not essential. It does not follow that there is not a considerable 
problem left just in saying what you think is to be described. Let me 
try to put this in another way. If you consider certain activities of the 
human nervous system, you find that some of them are such that all 
parts of them can be described, but one is flabbergasted by the totality 
of what has to be described. 

Suppose you want to describe the fact that when you look at a 
triangle you realize that it's a triangle, and you realize this whether 
it's small or large. It's relatively simple to describe geometrically 
what is meant: a triangle is a group of three lines arranged in a certain 
manner. Well, that's fine, except that you also recognize as a triangle 
something whose sides are curved, and a situation where only the 
vertices are indicated, and something where the interior is shaded and 
the exterior is not. You can recognize as a triangle many different 
things, all of which have some indication of a triangle in them, but 
the more details you try to put in a description of it the longer the 
description becomes. 

In addition, the ability to recognize triangles is just an infinitesimal 
fraction of the analogies you can visually recognize in geometry, 
which in turn is an infinitesimal fraction of all the visual analogies you 
can recognize, each of which you can still describe. But with respect 



RIGOROUS THEORIES OF CONTROL AND INFORMATION 47 

to the whole visual machinery of interpreting a picture, of putting 
something into a picture, we get into domains which you certainly 
cannot describe in those terms. Everybody will put an interpretation 
into a Rorschach test, but what interpretation he puts into it is a 
function of his whole personality and his whole previous history, and 
this is supposed to be a very good method of making inferences as to 
what kind of a person he is. 

In fine, now, all of this may seem a little arbitrary and accidental, 
but the basic fact involved is this, that our brains are exceedingly 
complicated. About one fifth of the brain is a visual brain, which, as 
far as we know, does nothing except make decisions about visual 
analogies. So, using the figures we have, which are not very good, but 
which are probably all right for an orientation, we conclude that ap- 
parently a network of about 2 billion relays does nothing but deter- 
mine how to organize a visual picture. It is absolutely not clear a 
priori that there is any simpler description of what constitutes a 
visual analogy than a description of the visual brain. 

Normally, a literary description of what an automaton is supposed 
to do is simpler than the complete diagram of the automaton. It is 
not true a priori that this will always be so. There is a good deal in 
formal logics to indicate that the description of the functions of an 
automaton is simpler than the automaton itself, as long as the auto- 
maton is not very complicated, but that when you get to high compli- 
cations, the actual object is simpler than the literary description. 

I am twisting a logical theorem a little, but it's a perfectly good 
logical theorem. It's a theorem of Godel that the next logical step, the 
description of an object, is one class type higher than the object and 
is therefore asymptotically [?] infinitely longer to describe. I say that 
it's absolutely necessary; it's just a matter of complication when you 
get to this point. I think that there is a good deal of reason to suspect 
that this is so with things which have this disagreeably vague and 
fluid impression (like "What is a visual analogy?"), where one feels 
that one will never get to the end of the description. They may easily 
be in this condition already, where doing a thing is quicker than 
describing it, where the circuit is more quickly enumerated than a 
total description of all its functions in all conceivable conditions. 

The insight that a formal neuron network can do anything which 
you can describe in words is a very important insight and simplifies 
matters enormously at low complication levels. It is by no means 
certain that it is a simplification on high complication levels. It is 
perfectly possible that on high complication levels the value of the 
theorem is in the reverse direction, that it simplifies matters because 



48 



THEORY OF SELF-REPRODUCING AUTOMATA 



it guarantees the reverse, namely, that you can express logics in 
terms of these efforts and the converse may not be true. [Von Neu- 
mann returned to this point on p. 51, after his discussion of Turing 
machines.] 

[ Von Neumann next discussed two cases in which circuits of ideal- 
ized neurons do not seem to provide an explanation of how the nervous 
system actually performs a given function. The first case concerns the 
transmission by a nerve of a continuous number which represents some 
quantity such as blood pressure. The nerve does this by emitting 
pulses at a frequency which is a monotone function of the blood pres- 
sure. This behavior is explained in terms of neural fatigue: after a 
neuron responds, it is unable to respond for a certain period, called 
the refractory period, and the stronger the next stimulus the sooner 
it responds. He then raised the question "Why has the digital notation 
never been used in nature, as far as we know, and why has this pulse 
notation been used instead ?" and said that this was the kind of ques- 
tion he was interested in. He suggested an answer: that the frequency 
modulation scheme is more reliable than the digital scheme. See 
Section 1.1.2.3 below, The Computer and the Brain 77-79, and Collected 
Works 5.306-308 and 5.375-376.] 

[ The second case in which circuits of idealized neurons do not seem 
to provide an explanation of how the nervous system actually per- 
forms a given function concerns memory. Von Neumann had earlier 
synthesized memory circuits from idealized neurons and he remarked 
that such memory circuits could be arbitrarily large. But he thought 
it probable that this is not the major mechanism used for memory in 
the nervous system.] This is not the way to make a memory for the 
simple reason that to use a switching organ like a neuron, or six to a 
dozen switching organs, as you actually would have to use because of 
fatigue, in order to do as small a thing as remember one binary digit, 
is a terrible waste, because a switching organ can do vastly more than 
store. In computing machines the classical example of a machine in 
which the switching organs were used to remember numbers is the 
ENIAC, an enormous gadget which has about 20 thousand vacuum 
tubes in it. The ENIAC is about five times larger than later machines 
which will presumably be far more efficient; it is an excellent machine 
in many ways, but it has one phenomenal shortcoming, namely, a 
very small memory. It has only a memory of 20 decimal numbers at 
points where it matters; in spite of this it is enormous. The reason is 
that vacuum tubes, in other words, switching organs, are used for 
that memory. All improvements on this machine postulate that some 
other components than standard vacuum tubes will be used for 
memory. 



RIGOROUS THEORIES OF CONTROL AND INFORMATION 



The memory requirements of the human nervous system are prob- 
ably very large. Estimates have been made and they are of the order 
of 10 16 binary digits. I will not attempt to justify this estimate; a great 
deal can be said for any count. I think there is a good deal of reason to 
believe that 10 10 switching organs, which is about what we have, is 
probably not the right order of magnitude for storing the kind of 
memory that we use, and that it's probably best to admit that we 
simply do not know where the memory is. One can make all kinds of 
statements about it. One can surmise thai the memory consists in a 
change of the synapses of the nerve cells, which is not decided by 
design. I don't know whether there is any good evidence for this, but 
I rather think there is not. You may suspect that the nerve cells 
contain a lot else than the switching trait, and that the memory sits 
there. It may be so, but I think that we simply know nothing. It may 
well be that the memory organs are of a completely different nature 
than the neurons. 

The main difficulty with the memory organ is that it appears to be 
nowhere in particular. It is never very simple to locate anything in 
the brain, because the brain has an enormous ability to re-organize. 
Even when you have localized a function in a particular part of it, if 
you remove that part, you may discover that the brain has reorganized 
itself, reassigned its responsibilities, and the function is again being 
performed. The flexibility of the brain is very great, and this makes 
localization difficult. I suspect that the memory function is less 
localized than anything else. [Cf. The Computer and the Brain 63-68.] 

I wanted to mention these two things [fatigue and memory] as very 
obvious lacunae in the McCulloch and Pitts approach to the nervous 
system. I want to talk next about the approach of Turing. In the 
McCulloch and Pitts theory the conclusion was that actual automata, 
properly described and axiomatized, are equivalent to formal logics. 
In Turing's theory the conclusion is the reverse. Turing was interested 
in formal logics, not in automata. He was concerned to prove certain 
theorems about an important problem of formal logics, the so-called 
Entscheidungsproblem, th e, problem of decision . The problem is to 
determine, for a class of logical expressions or propositions, whether 
there is a mechanical method for deciding whether an expression of 
this class is true or false. Turing's discussion of automata was really 
a formal, logical trick to deal with this problem in a somewhat more 
transparent and more consistent way than it had been dealt with 
before. 

[ Von Neumann then outlined Turing's definition of an automaton. 
Whereas McCulloch and Pitts started with components or elements, 
Turing started with states. At any time the automaton is in one of a 



50 



THEORY OF SELF-REPRODUCING AUTOMATA 



finite number of states. "The outside world" is a tape. The automaton 
senses one square of the tape, and it can change the contents of the 
square and move the tape one square to the left or right. A dictionary 
specifies, for each state and each tape symbol, what the next state will 
be and what will be done to the tape. The tape has a distinguished 
square. A finite program may be placed on the tape initially. The 
binary number computed by the automaton is recorded in alternate 
squares, starting with the distinguished square.] 

[ Von Neumann next described Turing's result concerning universal 
automata. There is a universal automaton A with the following prop- 
erties: For each automaton A there is a sequence of instructions I A 
such that for any sequence of instructions /, A supplied with both 
instructions I A and / computes the same number as is computed by A 
supplied with instructions /.] A is able to imitate any automaton, 
even a much more complicated one. Thus a lesser degree of complexity 
in an automaton can be compensated for by an appropriate increase ot 
complexity of the instructions. The importance of Turing's research 
is just this: that if you construct an automaton right, then any addi- 
tional requirements about the automaton can be handled by suffi- 
ciently elaborate instructions. This is only true if A is sufficiently 
complicated, if it has reached a certain minimum level of complexity. 
In other words, a simpler thing will never perform certain operations, 
no matter what instructions you give it; but there is a very definite 
finite point where an automaton of this complexity can, when given 
suitable instructions, do anything that can be done by automata at 
all. 

[Von Neumann then explained how the universal automaton A 
simulates an arbitrary automaton A. The instructions I A contain a 
representation of the automaton A in the form of a dictionary, which 
tells, for each state of A and each tape symbol, the next state of A 
and what is to be done to the tape. The universal automaton A has 
the power to read any such dictionary and act on it. A writes on its 
tape, in sequence, the successive states of A and what is produced on 
the tape of A.] I will not go further in giving the details of this. I 
have gone into it to the point to which I did in order to point out that 
here, for the first time, one deals with something which has the at- 
tribute of universality, which has the ability to do anything that any- 
body can do. You also see that there is no vicious circle in it, because 
of the manner in which the extra complexity is brought in (by giving 
more elaborate instructions). You also see that the operation which 
ultimately leads to universality is connected with a rigorous theory of 



RIGOROUS THEORIES OF CONTROL ANJ) INFORMATION 51 

how one describes objects and a rigorous routine of how to look up 
statements in a dictionary and obey them. 

The formal logical investigations of Turing went a good deal further 
than this. Turing proved that there is something for which you cannot 
construct an automaton; namely, you cannot construct an automaton 
which can predict in how many steps another automaton which can 
solve a certain problem will actually solve it. So, you can construct an 
automaton which can do anything any automaton can do, but you 
cannot construct an automaton which will predict the behavior of any 
arbitrary automaton. In other words, you can build an organ which 
can do anything that can be done, but you cannot build an organ 
which tells you whether it can be done. 

This is connected with the structure of formal logics and is specifi- 
cally connected with a feature which I will not attempt to discuss, 
but which I would like to mention in the proper jargon for those of 
you who are familiar with modern formal logics. It is connected with 
the theory of types and with the results of Godel. The feature is just 
this, that you can perform within the logical type that's involved 
everything that's feasible, but the question of whether something is 
feasible in a type belongs to a higher logical type. It's connected with 
the remark I made earlier (pp. 47-48) : that it is characteristic of objects 
of low complexity that it is easier to talk about the object than produce 
it and easier to predict its properties than to build it. But in the com- 
plicated parts of formal logic it is always one order of magnitude 
harder to tell what an object can do than to produce the object. The 
domain of the validity of the question is of a higher type than the 
question itself. 

[ This is the end of von Neumann's Second Lecture. I will add a 
commentary on his last two paragraphs, beginning with some general 
remarks on Turing machines. 

A Turing machine is basically a finite automaton with an indefi- 
nitely extendible tape. But there are many different ways of using a 
Turing machine. Let the squares of the tape be numbered 0, 1, 2, 
3, • • • , with even numbered squares reserved for working space and 
odd numbered squares reserved for the program or problem statement 
(if there is one) and for the answer. Let the answer symbols be zero 
and one, in addition to the blank; these could, of course, be coded 
sequences of two basic symbols, a blank and a mark. Assume finally 
that the machine writes the answer digits (zero and one) in successive 
answer squares. 

A "concrete Turing machine" is a Turing machine which has a 



UNIVERSITY OP 

ItUNOiS LIBRARY 



52 



THEORY OF SELF-REPRODUCING AUTOMATA 



finite "program" or problem statement on its tape initially. An 
"abstract Turing machine' ' is the class of all concrete Turing machines 
which contain a given finite automaton. We can think of an abstract 
Turing machine as a finite automaton with an indefinitely extendible 
blank tape on which any program or problem may be placed initially. 

Concrete Turing machines may be divided into two classes: the 
circular and the circle-free. A "circular" machine prints a finite se- 
quence of binary digits and "halts." A "circle-free" machine continues 
to print binary digits in alternate squares forever; we will speak of it 
as computing an infinite sequence. 

Von Neumann discussed above a universal Turing machine which 
consists of a finite automaton A and an indefinitely extendible tape. 
A universal Turing machine is an abstract Turing machine which 
computes every sequence computed by Turing machines. More 
precisely, for each concrete Turing machine with finite automaton A 
and program /, there is a program I A such that machine A with pro- 
grams I A and / computes the sequence computed by machine A with 
program /. A universal Turing machine can be characterized in 
another way. Let r be the class of finite and infinite sequences com- 
puted by concrete Turing machines. Then, every sequence of T is 
computed by the abstract Turing machine A + I A + /, where I A and 
/ vary over all programs. Since the concatenation of two programs is 
a program, every sequence of r is computed by the abstract Turing 
machine A + /, where / varies over all programs. 

A "decision machine" for a given class of questions is an abstract 
Turing machine which, when given a question of that class, prints 
a one if the answer to the question is "yes" and a zero if the answer 
to the question is "no." 

The "halting problem" is the problem of deciding whether an 
arbitrary concrete Turing machine is circular (will halt sometime) or 
circle-free. Turing showed that the halting problem is undecidable, 
that is, that there is no decision machine for halting. 3 The proof is 
given below in Sec. 1.6.3.2 of Part II. Turing proved as a corollary to 
this that there is no decision machine for deciding whether an arbi- 
trary concrete Turing machine will ever print a given symbol (for 
example, a zero). Since both halting and printing a given symbol are 
aspects of the behavior of a Turing machine, it follows from Turing's 
results that automata behavior is not completely predictable by 
automata. As von Neumann put it above, "you cannot construct an 

3 [ Turing, "On Computable Numbers, with an Application to the Entschei- 
dungsproblem," Sec. 8.] 



RIGOROUS THEORIES OF CONTROL AND INFORMATION 53 

automaton which will predict the behavior of any arbitrary autom- 
aton." 

Concrete Turing machines can be enumerated and thereby placed 
in one to one correspondence with the non-negative integers. Consider 
all these machines and let the variable "t" range over the integers 
representing them. We define the number theoretic function n(t) as 
the number of steps which machine t takes to print its first zero. If 
machine t never prints a zero, then n(t) is defined to be zero. 

Note that a sequence of n ones followed by a zero can be inter- 
preted as the integer n. This leads to the question: Is there an abstract 
Turing machine which can compute n(t) for any t? It follows immedi- 
ately from Turing's corollary that there is not, for if we could com- 
pute n(t) we could decide whether or not machine t ever prints a zero. 
I think that this is what von Neumann had in mind when he said 
"Turing proved that you cannot construct an automaton which can 
predict in how many steps another automaton which can solve a 
certain problem will actually solve it." 

In the last paragraph of his Second Lecture von Neumann referred 
to a theorem of Godel "that you can perform within the logical type 
that's involved everything that's feasible, but the question of whether 
something is feasible in a type belongs to a higher logical type." 
Since I knew of no such theorem by Godel I found this reference 
puzzling, as well as the earlier reference to Godel (p. 47) and a related 
reference in von Neumann's Hixon Symposium paper, "The General 
and Logical Theory of Automata" (Collected Works 5.310-311). I 
wrote Professor Kurt Godel to see whether he could throw any light 
on it. His answer gives, I think, the most plausible explanation of the 
reference, and so I include the relevant parts of our correspondence, 
with minor editing. 

I wrote to Professor Godel as follows: "I am engaged in editing two 
of John von Neumann's uncompleted manuscripts on the theory of 
automata. In one of these, a series of lectures he delivered at the 
University of Illinois in 1949, he makes a reference to your work which 
I have been unable to figure out. Since there is the possibility he may 
have discussed this point with you, I am taking the liberty of writing 
you about it. 

"The story begins with Johnny's Hixon Symposium talk at Pasa- 
dena in 1948. He discusses there the problem of giving a rigorous 
description of a visual analogy. In recognizing visual patterns, the 
human eye and nervous system function as a finite automaton with 
a certain behavior. Von Neumann seems to suggest that possibly the 
simplest way to describe the behavior of this finite automaton is to 



54 



THEORY OF SELF-REPRODUCING AUTOMATA 



describe the structure of the automaton itself. This is certainly plausi- 
ble. But he then expresses the point in a way I do not understand: 
'It is not at all certain that in this domain a real object might not 
constitute the simplest description of itself. That is, any attempt to 
describe it by the usual literary or formal-logical method may lead to 
something less manageable and more involved. In fact, some results in 
modern logic would tend to indicate that phenomena like this have to 
be expected when we come to really complicated entities/ The under- 
lined passage seems to refer to your work. I enclose a copy of the full 
context. 

"In his Illinois lectures, given in 1949, Johnny seems to be making 
the same point, namely, that the simplest way to describe accurately 
what constitutes a visual analogy is to specify the connections of the 
visual part of the brain. He then proceeds to say that there is a good 
deal in formal logic which indicates that when an automaton is not 
very complicated the description of the functions of that automaton 
is simpler than a description of the automaton itself but that the 
situation is reversed with respect to complicated automata. His 
reference to you then appears explicitly. He says, T am a little twisting 
a logical theorem, but it's a perfectly good logical theorem. It's a 
theorem of Godel that the next logical step, the description of an 
object, is one class type higher than the object and is therefore asymp- 
totically [?] infinitely longer to describe.' 

"He returns to this point later after discussing Turing machines 
and mentioning Turing's result about the undecidability of the halting 
problem. He then says that all of this is connected with the theory of 
types and with your results. The recording transcript is mangled at 
this point and I will reconstruct it as best I can. Tt is connected with 
the theory of types and with the results of Godel. The feature is just 
this, that you can perform within the logical type that's involved 
everything that's feasible but the question of whether something is 
feasible in a type belongs to a higher logical type. It's connected with 
the remark I made earlier: that it is characteristic of objects of low 
complexity that it is easier to talk about the object than produce it 
and easier to predict its properties than to build it. But in the com- 
plicated parts of formal logic it is always one order of magnitude 
harder to tell what an object can do than to produce the object. The 
domain of the validity of the question is of a higher type than the 
question itself.' I enclose copies of the relevant pages of the Illinois 
lectures. 

"It is easy to regard the description of an object as of one type level 
higher than the object itself, but beyond this I do not see what von 



RIGOROUS THEORIES OF CONTROL AND INFORMATION 55 

Neumann has in mind. Two possibilities occured to me but both give 
a result opposite to that which Johnny needs. One may regard a Godel 
number as a description of a formula. However, in some cases at least, 
the Godel number of a formula may be described in fewer symbols 
than the formula, else the self-referring undecidable formula could 
not exist. 4 The other possibility concerns a theorem in your 1936 
paper, "tlber die Lange der Beweise. ,, Given a system S and a larger 
system Si . The theorem says that for every recursive function F, 
there exists a sentence which is provable in both systems and such 
that the shortest proofs in these two systems satisfy the inequality 
that the Godel number of the proof in the smaller system is larger than 
the recursive function F applied to the Godel number of the proof in 
the larger system. This fits everything that von Neumann says except 
that the result seems to go in the opposite direction: namely, the 
higher the type the shorter the proof. 

"I would appreciate very much any light that you could throw on 
these puzzling passages of von Neumann." 

Professor Godel replied as follows. "I have some conjecture as to 
what von Neumann may have had in mind in the passages you quote, 
but since I never discussed these matters with him it is only a guess. 

"I think the theorem of mine which von Neumann refers to is not 
that on the existence of undecidable propositions or that on the lengths 
of proofs but rather the fact that a complete epistemological descrip- 
tion of a language A cannot be given in the same language A , because 
the concept of truth of sentences of A cannot be defined in A . It is this 
theorem which is the true reason for the existence of undecidable prop- 
ositions in the formal systems containing arithmetic. I did not, how- 
ever, formulate it explicitly in my paper of 1931 but only in my Prince- 
ton lectures of 1934. 5 The same theorem was proved by Tarski in his 
paper on the concept of truth published in 1933 in Act. Soc. Sci. Lit. 
Vars., translated on pp. 152-278 of Logic, Semantics, and Metamaihe- 
matics* 

"Now this theorem certainly shows that the description of what a 
mechanism is doing in certain cases is more involved than the descrip- 

4 [ See Godel's "Uber formal unentscheidbare Satze der Principia Mathema- 
tica und verwandter Systeme I." The undecidable formula has the Godel 
number n and says 'The formula whose Godel number is n is not a theorem.' ' 
Thus, via Godel's coding, the undecidable formula refers to itself. It is un- 
decidable in the sense that neither it nor its negation is a theorem of the system 
Godel is studying.] 

5 [ Godel, "On Undecidable Propositions of Formal Mathematical Sys- 
tems. "] 

6 [ The exact reference to Tarski 's paper was added later.] 



56 



THEORY OF SELF-REPRODUCING AUTOMATA 



tion of the mechanism, in the sense that it requires new and more 
abstract primitive terms, namely higher types. However, this implies 
nothing as to the number of symbols necessary, where the relationship 
may very well be in the opposite direction, as you rightly remark. 

"However, what von Neumann perhaps had in mind appears more 
clearly from the universal Turing machine. There it might be said 
that the complete description of its behavior is infinite because, in 
view of the non-existence of a decision procedure predicting its be- 
havior, the complete description could be given only by an enumera- 
tion of all instances. Of course this presupposes that only decidable 
descriptions are considered to be complete descriptions, but this is in 
line with the finitistic way of thinking. The universal Turing machine, 
where the ratio of the two complexities is infinity, might then be con- 
sidered to be a limiting case of other finite mechanisms. This immedi- 
ately leads to von Neumann's conjecture. ,, ] 



Third Lecture 



STATISTICAL THEORIES OF 
INFORMATION 



Theory of information: Probabilistic part. Relationship of strict and of 
probabilistic logics. Keynes' interpretation of probability theory. Exemplifi- 
cation of the relationship of logics to strict classical mechanics on the one 
hand, and to statistical mechanics on the other. Corresponding situation in 
quantum mechanics. 

The mathematical aspects of the transition from strict to probabilistic 
logics. Analysis and combinatorics. 

The thermodynamical aspect: Information and entropy. 

The theory of Szilard. 

The theory of Shannon. 

Additional remarks on the thermodynamical nature of the internal balance 
of a computing machine. 

I conclude my remarks about strict and rigorous questions of infor- 
mation at this point and pass on to statistical considerations involving 
information. That this is the important thing in dealing with automata 
and their functions is fairly evident, for two reasons at least. The first 
of these reasons may seem somewhat extraneous and accidental, al- 
though I think it is not, but the second reason is certainly not. 

The first reason is that in no practical way can we imagine an autom- 
aton which is really reliable. If you axiomatize an automaton by 
telling exactly what it will do in every completely denned situation 
you are missing an important part of the problem. The axiomatization 
of automata for the completely denned situation is a very nice exercise 
for one who faces the problem for the first time, but everybody who 
has had experience with it knows that it's only a very preliminary 
stage of the problem. 

The second reason for the importance of statistical considerations 
in the theory of automata is this. If you look at automata which have 
been built by men or which exist in nature you will very frequently 
notice that their structure is controlled only partly by rigorous re- 
quirements and is controlled to a much larger extent by the manner 



57 



58 



THEORY OF SELF-REPRODUCING AUTOMATA 



in which they might fail and by the (more or less effective) precau- 
tionary measures which have been taken against their failure. And to 
say that they are precautions against failure is to overstate the case, 
to use an optimistic terminology which is completely alien to the sub- 
ject. Rather than precautions against failure, they are arrangements 
by which it is attempted to achieve a state where at least a majority 
of all failures will not be lethal. There can be no question of eliminat- 
ing failures or of completely paralyzing the effects of failures. All we 
can try to do is to arrange an automaton so that in the vast majority 
of failures it can continue to operate. These arrangements give pallia- 
tives of failures, not cures. Most of the arrangements of artificial and 
natural automata and the principles involved therein are of this sort. 

To permit failure as an independent logical entity means that one 
does not state the axioms in a rigorous manner. The axioms are not 
of the form: if A and B happen, C will follow. The axioms are always of 
this variety: if A and B happen, C will follow with a certain specified 
probability, D will follow with another specified probability, and so 
on. In other words, in every situation several alternatives are per- 
mitted with various probabilities. Mathematically it is simplest to- 
say that anything can follow upon anything in accordance with a 
probability matrix. You may put your question in this manner: If A 
and B have happened, what is the probability that C will follow? This 
probability pattern gives you a probabilistic system of logics. Both 
artificial and natural automata should be discussed in this system as 
soon as there is any degree of involvement. 1 I will come later to the 
question as to why it is just complexity which pushes one into this 
kind of axiomatization instead of a strict one. 2 

Now this inclines one to view probability as a branch of logics, or 
rather, to view logics affected with probability as an extension of 
ordinary rigorous logics. The view that probability is an extension of 
logics is not trivial, is not generally accepted, and is not the major 
interpretation of probability. It is, however, the classical interpreta- 
tion. The competing interpretation is the frequency interpretation, 
the attitude that logic is completely rigorous, and with respect to 
phenomena about which you are not completely informed, you can 
only make statements of frequencies. 

This distinction was, I think, quite clear to Laplace, who pointed 

1 [ See von Neumann's "Probabilistic Logics and the Synthesis of Reliable 
Organs from Unreliable Components" for a detailed treatment of automata 
from this point of view.] 

2 [ For a given probability of malfunction of a component, the more complex 
the automaton the more likely it is that a lethal failure will occur.] 



STATISTICAL THEORIES OF INFORMATION 



59 



out that there are two possible attitudes toward probability: the fre- 
quency and the logical. 3 In more recent times the distinction was em- 
phasized strongly and made the basis of a system by the economist 
Keynes, who wrote his thesis on probability. 4 He analyzed this ques- 
tion in considerable detail and showed that, aside from the more con- 
ventional frequency viewpoint about probability, the logical one also 
exists. But he made no attempt to separate strict logics and probabil- 
ity and simply said that, if you view a sequence of events A and B, 
they have the quantitative characteristic, "the probability with which 
B follows A" The only tie to strict logics is that when the probability 
is one you have an implication, and when the probability is zero you 
have an exclusion, and when the probability is close to one or close to 
zero you can still make those inferences in a less rigorous domain. 

There are undeniable weaknesses of the logical position. In some 
ways of looking at probability it is opportune not to identify zero 
probability with absurdity. Also, it is not quite clear in what sense a 
low probability means that one might expect that the thing will not 
happen. However, Keynes produced a self -consistent axiomatic sys- 
tem. There's a great deal in other modern theories, for instance, in 
quantum mechanics, which inclines one very strongly to take this 
philosophical position, although the last word about this subject has 
certainly not been said and is not going to be said for a long time. 
Anyway, one is also tempted in the case of quantum mechanics to 
modify one's outlook on logics and to view probability as intrinsically 
tied to logics. 5 

[ Von Neumann discussed next two theories of probability and in- 
formation "which are quite relevant in this context although they are 
not conceived from the strictly logical point of view." The first is the 
theory of entropy and information in thermodynamics; the second 
is Shannon's information theory. 

In connection with entropy and information von Neumann referred 
to Boltzmann, Hartley, and Szilard. He explained at length the para- 

3 [ A Philosophical Essay on Probabilities.] 

4 [ A Treatise on Probability.] 

5 [ In his "Quantum Logics (Strict-and-Probability-Logics)", von Neumann 
concluded: ^Probability logics cannot be reduced to strict logics, but constitute an 
essentially wider system than the latter, and statements of the form P(a, b) = <j> 
(0 < <f> < 1) are perfectly new and sui generis aspects of physical reality. 

"So probability logics appear as an essential extension of strict logics. This 
view, the so-called 'logical theory of probability' is the foundation of J. M. 
Keynes's work on this subject." 

Compare von Neumann and Birkhoff, "The Logic of Quantum Mechanics," 
and von Neumann and Morgenstern, Theory of Games and Economic Behavior, 
Sec. 3.3.3.] 



60 



THEORY OF SELF-REPRODUCING AUTOMATA 



dox of Maxwell's demon and how Szilard resolved it by working out 
the relation of entropy to information. 6 Von Neumann said that 
Shannon's theory is a quantitative theory of measuring the capacity 
of a communications channel. He explained and illustrated the con- 
cept of redundancy. He pointed out that redundancy makes it possible 
to correct errors, for example, to read proof. Redundancy "is the only 
thing which makes it possible to write a text which is longer than, say, 
ten pages. In other words, a language which has maximum compres- 
sion would actually be completely unsuited to conveying information 
beyond a certain degree of complexity, because you could never find 
out whether a text is right or wrong. And this is a question of principle. 
It follows, therefore, that the complexity of the medium in which you 
work has something to do with redundancy." 

In his review of Wiener's Cybernetics von Neumann made an ex- 
tended statement about entropy and information which it is appro- 
priate to quote here. "Entropy for the physicist is a concept belonging 
to the discipline of thermodynamics where the transformations among 
the various forms of energy are studied. It is well known that the 
total energy of a complete, closed system is always conserved : energy 
is neither created nor lost but only transformed. This constitutes the 
first fundamental theorem of thermodynamics, or the energy theorem. 
There is, however, in addition, the second fundamental theorem of 
thermodynamics, or entropy theorem, which states that a hierarchy 
exists among the forms of energy: mechanical (kinetic or potential) 
energy, constituting the highest form, thermal energies constituting 
under it a decreasing hierarchical sequence in the order of decreasing 
temperature, and all other forms of energy permitting a complete 
classification relative to the gradations of this schema. It states, fur- 
thermore, that energy is always degraded, that is, that it always moves 
spontaneously from a higher form to a lower one, or if the opposite 
should happen in a part of the system, a compensatory degradation 
will have to take place in some other part. The bookkeeping that is 
required to account for this continuing overall degradation is effected 
by a certain well defined physical quantity, the entropy, which meas- 
ures the hierarchic position held or the degree of degradation suffered 
by any form of energy. 

"The thermodynamical methods of measuring entropy were known 
in the mid-nineteenth century. Already in the early work on statistical 
physics (L. Boltzmann, 1896) it was observed that entropy was closely 

6 [ There is a good exposition of the work of Szilard, as well as that of Shan- 
non and Hamming, in Brillouin's Science and Information Theory.] 



STATISTICAL THEORIES OF INFORMATION 



61 



connected with information: Boltzmann found entropy to be propor- 
tional to the logarithm of the number of alternatives which are possi- 
ble for a physical system after all the information that one possesses 
about that system macroscopically (that is, on the directly, humanly 
observable scale) has been recorded. 7 In other words, it is proportional 
to the logarithm of the amount of missing information. This concept 
was elaborated further by various authors for various applications: 
H. Nyquist and R. V. L. Hartley, for transmission of information in 
the technical communication media (Bell System Technical Journal, 
Vol. 3, 1924, and Vol. 7, 1928) ; L. Szilard, for information in physics 
in general (Zschr. f. Phys., Vol. 53, 1929) ; and the reviewer, for quan- 
tum mechanics and elementary particle physics (Mathematical 
Foundations of Quantum Mechanics, Berlin, 1932, Chapter V). 

"The technically well-equipped reader is advised to consult at this 
point some additional literature, primarily L. Szilard's work, referred 
to above, which also contains a particularly instructive analysis of 
the famous thermodynamical paradox of "Maxwell's demon, " and 
C. E. Shannon's very important and interesting recent work on the 
"Theory of Information, " "Artificial Languages," "Codes," etc. 
(Bell System Technical Journal, Vol. 27, 1948). There is reason to 
believe that the general degeneration laws, which hold when entropy 
is used as a measure of the hierarchic position of energy, have valid 
analogs when entropy is used as a measure of information. On this 
basis one may suspect the existence of connections between thermody- 
namics and new extensions of logics." 

In the Illinois lectures von Neumann next discussed Hamming's 
work on error-detecting and error-correcting codes. He then showed 
how the digital system with a base (binary, decimal, etc.) is an applica- 
tion of information theory. "Digitalization is just a very clever trick 
to produce extreme precision out of poor precision. By writing down 
30 binary digits with 30 instruments, each of which is only good 
enough that you can distinguish two states of it (with intrinsic errors 
maybe on the 10 per cent level), you can represent a number to ap- 
proximately one part in a billion. The main virtue of the digital sys- 
tem is that we know no other trick which can achieve this. From the 
information point of view it is clear that this can be done, because 
the entropy in 30 binary instruments is 30 units, and something which 

7 [ Vorlesungen iiber Gastheorie, Vol. I, Sec. 6. Boltzmann's result appeared 
originally in 1877 in "Uber die Beziehung zwischen dem zweiten Hauptsatze 
der mechanischen Warmetheorie und der Wahrscheinlichkeitsrechnung respek- 
tive den Satzen iiber das Warrnleichgewicht," Wissenschaftliche Abhandlungen , 
Vol. II, pp. 164-223.] 



G2 



THEORY OF SELF-REPRODUCING AUTOMATA 



is known to one part in a billion has an entropy of the logarithm of a 
billion (to the base two), or about 30 units." 

He then pointed out that while organisms use mixed analog-pulse 
systems for transmitting information, they never (to the best of our 
knowledge) use a coded digital system with a base. Rather, when 
"the nervous system transmits a number, it transmits it by what is 
essentially a frequency modulation trick and not as a coded digital 
aggregate. " He suggested that the reason for this is that the frequency 
modulation method is more reliable than the digital system.] 

I have been trying to justify the suspicion that a theory of informa- 
tion is needed and that very little of what is needed exists yet. Such 
small traces of it which do exist, and such information as one has 
about adjacent fields indicate that, if found, it is likely to be similar 
to two of our existing theories: formal logics and thermodynamics. 
It is not surprising that this new theory of information should be like 
formal logics, but it is surprising that it is likely to have a lot in com- 
mon with thermodynamics. 

Though this new theory of information will be similar to formal 
logics in many respects, it will probably be closer to ordinary mathe- 
matics than formal logics is. The reason for this is that present day 
formal logics has a very un-analytical, un-mathematical charac- 
teristic: it deals with absolutely all-or-none processes, where every- 
thing that either does or does not happen is finitely feasible or not 
finitely feasible. These all-or-none processes are only weakly connected 
to analysis, which is the best developed and best known part of mathe- 
matics, while they are closely eonnected to combinatorics, that part 
of mathematics of which we know the least. There is reason to believe 
that the kind of formal logical machinery we will have to use here 
will be closer to ordinary mathematics than present day logics is. 
Specifically, it will be closer to analysis, because all axioms are likely 
to be of a probabilistic and not of a rigorous character. Such a phe- 
nomenon has taken place in the foundations of quantum mechanics. 

Thermodynamical concepts will probably enter into this new theory 
of information. There are strong indications that information is similar 
to entropy and that degenerative processes of entropy are paralleled 
by degenerative processes in the processing of information. It is likely 
that you cannot define the function of an automaton, or its efficiency, 
without characterizing the milieu in which it works by means of sta- 
tistical traits like the ones used to characterize a milieu in thermo- 
dynamics. The statistical variables of the automaton's milieu will, 
of course, be somewhat more involved than the standard thermody- 
namical variable of temperature, but they will probably be similar 
in character. 



STATISTICAL THEORIES OP INFORMATION 



63 



Also, it is quite clear from the practice of building computing ma- 
chines that the decisive properties of computing machines involve 
balance: balances between the speeds of various parts, balances be- 
tween the speed of one part and the sizes of other parts, even balances 
between the speed ratio of two parts and the sizes of other parts. I 
mentioned this in the case of the hierarchic structure of memory 
[p. 41]. All of these requirements look like the balance requirements 
one makes in thermodynamics for the sake of efficiency. An automaton 
in which one part is too fast for another part, or where the memory is 
too small, or where the speed ratio of two memory stages is too large 
for the size of one, looks very much like a heat engine which doesn't 
run properly because excessively high temperature differences exist 
between its parts. I will not go into the details of this, but I would 
like to emphasize that this thermodynamical link is probably quite 
a close one. 



Fourth Lecture 



THE ROLE OF HIGH AND OF 
EXTREMELY HIGH 
COMPLICATION 



Comparisons between computing machines and the nervous systems. 
Estimates of size for computing machines, present and near future. 

Estimates for size for the human central nervous system. Excursus about 
the "mixed" character of living organisms. Analog and digital elements. 
Observations about the "mixed" character of all componentry, artificial as 
well as natural. Interpretation of the position to be taken with respect to 
these. 

Evaluation of the discrepancy in size between artificial and natural auto- 
mata. Interpretation of this discrepancy in terms of physical factors. Nature 
of the materials used. 

The probability of the presence of other intellectual factors. The role of 
complication and the theoretical penetration that it requires. 

Questions of reliability and errors reconsidered. Probability of individual 
errors and length of procedure. Typical lengths of procedure for computing 
machines and for living organisms — that is, for artificial and for natural 
automata. Upper limits on acceptable probability of error in individual 
operations. Compensation by checking and self -correcting features. 

Differences of principle in the way in which errors are dealt with in artificial 
and in natural automata. The "single error" principle in artificial automata. 
Crudeness of our approach in this case, due to the lack of adequate theory. 
More sophisticated treatment of this problem in natural automata : The role 
of the autonomy of parts. Connections between this autonomy and evolution. 

After the broad general discussions of the last two lectures I would 
like to return to the subject of the specific automata which we know. 
I would like to compare artificial automata, specifically computing 
machines, with natural automata, particularly the human nervous 
system. In order to do this, I must say a few things in both cases 
about components and I must make certain comparisons of sizes. 

As I mentioned before, in estimating the size of the human nervous 
system one is limited to a figure which is not very well established, 
but which is probably right in its order of magnitude. This is the 



64 



ROLE OF HIGH COMPLICATION 65 

statement that there are 10 10 neurons in the human brain. The 
number of nerves present elsewhere in the human organism is proba- 
bly much smaller than this. Also, a large number of these other 
nerves originate in the brain anyway. The largest aggregation of 
nerves of the periphery is on the retina, and the optic nerve going 
from the retina to the brain is part of the brain. 

Compared to this, the number of vacuum tubes involved in the 
computing machines we know of is very small, a million times smaller. 
The largest existing computing machine, the ENIAC, has 2 X 10 4 
vacuum tubes. Another large computing machine, the SSEC, which 
belongs to the IBM Company, contains a mixture of vacuum tubes 
and relays, about 10 thousand of each. The fastest computing ma- 
chines now under construction are designed to have several thousand 
vacuum tubes, perhaps 3 thousand. The reason for this difference 
in size between the ENIAC and the fast machines now under con- 
struction is a difference in the treatment of memory, which I will 
discuss later. 

So the human nervous system is roughly a million times more - 
complicated than these large computing machines. The increase in 
complexity from these computing machines to the central nervous 
system is more than the increase in complexity from a single vacuum 
tube to these computing machines. Even measuring complexity on a 
logarithmic scale, which is highly generous, we have not yet come 
half the way. I think that in any sensible definition of complexity, 
it would be much less than half way. 

There is, however, a factor in favor of these machines: they're 
faster than the human brain. The time in which a human nerve can 
respond is about \ millisecond. However, that time is not a fair 
measure of the speed of the neuron, because what matters is not the 
time in which the neuron responds, but the time in which it recovers, 
the time from one response to the next potential response. That 
time is, at best, 5 milliseconds. In the case of a vacuum tube it's 
difficult to estimate the speed, but present designs call for repetition 
rates which are not much in excess of a million per second. 

Thus the nervous system has a million times as many components 
as these machines have, but each component of the machine is about 
5 thousand times faster than a neuron. Counting what can be done, 
hour by hour, the nervous system outperforms the machine by a 
factor of roughly 200. This estimate, however, favors the automaton, 
because an n-fold increase in size brings much more than an n-fold 
increase in what can be done. What can be done is a matter of the 
interrelationships between the components, and the number of 



66 



THEORY OF SELF-REPRODUCING AUTOMATA 



interrelationships increases with the square of the number of com- 
ponents. And apart from this, what can be done depends on certain 
minima. Below a certain minimum level of complexity you cannot 
do a certain thing, but above this minimum level of complexity you 
can do it. 

[Von Neumann next compared the human nervous system and 
computers with respect to volume. The decisive factor is the space in 
which the control and amplifying functions are performed. In the case 
of the vacuum tube this is essentially the space between the cathode 
and the control grid, which is of the order of magnitude of a millime- 
ter. In the case of the nerve cell it is the thickness of the nerve mem- 
brane, which is of the order of 1 micron. The ratio in size is about 1000 
to 1, and this is also the ratio in voltage, so that the intensity of the 
field which is used for control and amplification is about the same in 
the vacuum tube and the nerve cell. This means that differences in 
total energy dissipation are mainly due to differences in size. "A dis- 
crepancy of 10 3 in linear size means a discrepancy of 10 9 in volume, 
and probably a not very different discrepancy in energy." See also 
Collected Works 5.299-302 and The Computer and the Brain 44-52. 

He then calculated the energy which is dissipated "per elementary 
act of information, that is, per elementary decision of a two-way 
alternative and per elementary transmittal of 1 unit of information." 
He did this for three cases: the thermodynamical minimum, the 
vacuum tube, and the neuron. 

In the third lecture he said that thermodynamical information is 
measured by the logarithm, to the base two, of the number of alterna- 
tives involved. The thermodynamical information in the case of two 
alternatives is thus one, "except that this is not the unit in which you 
measure energy. Entropy is energy only if you specify the tempera- 
ture. So, running at low temperature you can say what energy should 
be dissipated." He then computed the thermodynamical minimum 
of energy per elementary act of information from the formula kT 
\og e N ergs, where k is Boltzmann's constant (1.4 X 10~ 16 ergs per 
degree), T is the temperature in absolute units, and N is the number 
of alternatives. For a binary act N = 2, and taking the temperature 
to be about 300 degrees absolute, he obtained 3 X 10~~ 14 ergs for the 
thermodynamical minimum. 

Von Neumann then estimated that the brain dissipates 25 watts, 
has 10 10 neurons, and that on the average a neuron is activated about 
10 times per second. Hence the energy dissipation per binary act in a 
nerve cell is roughly 3 X 10~ 3 ergs. He estimated that a vacuum tube 
dissipates 6 watts, is activated about 100,000 times per second, and 
thus dissipates 6 X 10 2 ergs per binary act.] 



ROLE OF HIGH COMPLICATION 



67 



So our present machinery is about 200 thousand times less efficient 
than the nervous system is. Computing machines will be improved in 
the next few years, perhaps by replacing vacuum tubes with amplify- 
ing crystals, but even then they will be of the order of 10 thousand 
times less efficient than the nervous system. The remarkable thing, 
however, is the enormous gap between the thermodynamical mini- 
mum (3 X 10~ 14 ergs) and the energy dissipation per binary act in the 
neuron (3 X 10~ 3 ergs). The factor here is 10 11 . This shows that the 
thermodynamical analysis is missing a large part of the story. Meas- 
ured on a logarithmic scale, the gap between our instrumentation, 
which is obviously amateurish, and the procedures of nature, which 
show a professional touch, is about half the gap between the best 
devices we know about and the thermodynamical minimum. What 
this gap is due to I don't know. I suspect that it's due to something 
like a desire for reliability of operation. 

Thus, for an elementary act of information, nature does not use ,„ 
what, from the point of view of physics, is an elementary system with 
two stable states, such as a hydrogen atom. All the switching organs 
used are much larger. If nature really operated with these elementary 
systems, switching organs would have dimensions of the order of a 
few angstroms, while the smallest switching organs we know have di- 
mensions of the order of thousands or tens of thousands of angstroms. 
There is obviously something which forces one to use organs several 
orders of magnitude larger than is required by the strict thermo- 
dynamical argument. Thus, though the observation that information 
is entropy tells an important part of the story, it by no means tells the 
whole story. There is a factor of 10 11 still to be accounted for. 

[Von Neumann then discussed memory components. Vacuum 
tubes, which are switching organs, may be used for memory. But 
since the standard circuit for storing a binary digit has two tubes, and 
additional tubes are needed for transmitting the information in and 
out, it is not feasible to build a large memory out of vacuum tubes. 
"The actual devices which are used are of such a nature that the 
store is effected, not in a macroscopic object like a vacuum tube, but 
in something which is microscopic and has only a virtual existence.' ' 
Von Neumann describes two devices of this sort: acoustic delay line 
storage and cathode ray tube storage. 

An acoustic delay line is a tube which is filled with a medium such 
as mercury and which has a piezo-electric crystal at each end. When 
the transmitting crystal is stimulated electrically, it produces an 
acoustic wave that travels through the mercury and causes the re- 
ceiving crystal to produce an electrical signal. This signal is amplified, 
reshaped, and retimed and sent to the transmitting crystal again. 



68 



THEORY OF SELF-REPRODUCING AUTOMATA 



This acoustic-electrical cycle can be repeated indefinitely, thereby 
providing storage. A binary digit is represented by the presence or 
absence of a pulse at a given position at a given time, and since the 
pulses circulate around the system, the digit is not stored in any fixed 
position. "The thing which remembers is nowhere in particular." 

Information may be stored in a cathode ray tube in the form of 
electric charges on the inside surface of the tube. A binary digit is 
represented by the charge stored in a small area. These charges are 
deposited and sensed by means of the electron beam of the cathode 
ray tube. Since the area associated with a given binary digit must be 
recharged frequently, and since this area may be moved by changing 
the position of the electron beam, this memory is also virtual. "The 
site of the memory is really nowhere organically, and the mode of 
control produces the memory organ in a virtual sense, because no 
permanent physical changes ever occur."] 

There's therefore no reason to believe that the memory of the 
central nervous system is in the switching organs (the neurons). The 
size of the human memory must be very great, much greater than 
10 10 binary units. If you count the impressions which a human gets in 
his life or other things which appear to be critical, you obtain numbers 
like 10 15 . One cannot place much faith in these estimates, but I think 
it likely that the memory capacity of the human nervous system is 
greater than 10 10 . I don't know how legitimate it is to transfer our 
experience with computing machines to natural systems, but if our 
experience is worth anything it is highly unlikely that the natural 
memory should be in switching organs or should consist of anything 
as unsophisticated and crude as the modification of a switching organ. 
It has been suggested that memory consists in a change of threshold 
at a synapse. I don't know if this is true, but the memory of comput- 
ing machines does not consist of bending a grid. A comparison be- 
tween artificial automata and the central nervous system makes it 
probable that the memory of the latter is more sophisticated and more 
virtual than this. Therefore, I think that all guesses about what the 
memory of the human organism is, and where it sits, are premature. 

Another thing of which I would like to talk is this. I have been 
talking as if a nerve cell were really a pure switching organ. It has 
been pointed out by many experts in neurology and adjacent fields 
that the nerve cell is not a pure switching organ but a very delicate 
continuous organ. In the lingo of computing machinery one would 
say it is an analog device that can do vastly more than transmit or 
not transmit a pulse. There is a possible answer to this, namely, that 
vacuum tubes, electromechanical relays, etc. are not switching devices 



ROLE OF HIGH COMPLICATION 



69 



either, since they have continuous properties. They are all charac- 
terized by this, however, that there is at least one way to run them 
where they have essentially an all-or-none response. What matters is 
how the component runs when the organism is functioning normally. 
Now nerve cells do not usually run as all-or-none organs. For in- 
stance, the method of translating a stimulus intensity into a fre- 
quency of response depends on fatigue and the time of recovery, which 
is a continuous or analog response. However, it is quite clear that 
the all-or-none character of a neuron is a very important part of the 
story. 

The human organism is not a digital organ either, though one part 
of it, the nervous system, is essentially digital. Almost all the nervous 
stimuli end in organs which are not digital, such as a contracting 
muscle or an organ which causes secretions to produce a chemical. 
To control the production of a chemical and rely on the diffusion rate 
of a chemical is to employ a much more sophisticated analog pro- 
6edure than we ever use in analog computing machines. The most 
important loops in the human system are of this nature. A system of 
nervous stimuli goes through a complicated network of nerves and 
then controls the operation of what is essentially a chemical factory. 
The chemicals are distributed by a very complicated hydrodynamical 
system, which is completely analog. These chemicals produce nerv- 
ous stimuli which travel in a digital manner through the nervous 
system. There are loops where this change from digital into analog 
occurs several times. So the human organism is essentially a mixed 
system. But this does not decrease the necessity for understanding 
the digital part of it. 

Computing machines aren't purely digital either. The way we run 
them now, their inputs and outputs are digital. But it's quite clear 
that we need certain non-digital inputs and outputs. It's frequently 
desirable to display the result, not in digits, but, say, as a curve on 
an oscilloscope screen. This is an analog output. Moreover, I think 
that the important applications of these devices will come when you 
can use them to control complicated machinery, for example, the 
flight of a missile or of a plane. In this case the inputs will come from 
an analog source and the outputs will control an analog process. 
This whole trans-continuous alternation between digital and analog 
mechanisms is probably characteristic of every field. 

The digital aspect of automata should be emphasized at the present 
time, for we now have some logical tools to deal with digital mech- 
anisms, and our understanding of digital mechanisms is behind our 
understanding of analog mechanisms. Also, it appears that digital 



70 



THEORY OF SELF-KEPKODUCING AUTOMATA 



mechanisms are necessary for coin plicated functions. Pure analog 
mechanisms are usually not suited for very complicated situations. 
The only way to handle a complicated situation with analog mech- 
anisms is to break it up into parts and deal with the parts separately 
and alternately, and this is a digital trick. 

Let me now come to the following question. Our artificial automata 
are much smaller than natural automata in what they do and in the 
number of components they have, and they're phenomenally more 
expensive in terms of space and energy. Why is this so? It's mani- 
festly hopeless to produce a true answer at the present time: We can 
hardly explain why two objects are different if we understand one a 
little and the other not at all. However, there are some obvious dis- 
crepancies in the tools with which we operate, which make it clear 
that we would have difficulty in going much further with these tools. 

The materials which we are using are by their very nature not well 
suited for the small dimensions nature uses. Our combinations of 
metals, insulators, and vacuums are much more unstable than the 
materials used by nature; that they have higher tensile strengths is 
completely incidental. If a membrane is damaged it will reconstruct 
itself, but if a vacuum tube develops a short between its grid and 
cathode it will not reconstruct itself. Thus the natural materials 
have some sort of mechanical stability and are well balanced with 
respect to mechanical properties, electrical properties, and reliability 
requirements. Our artificial systems are patchworks in which we 
achieve desirable electrical traits at the price of mechanically unsound 
things. We use techniques which are excellent for fitting metal to 
metal but are not very good for fitting metal to vacuum. To obtain 
millimeter spacings in an inaccessible vacuum space is a great me- 
chanical achievement, and we will not be able to decrease the size by 
large factors here. And so the differences in size between artificial 
and natural automata are probably connected essentially with quite 
radical differences in materials. 

[ Von Neumann proceeded to discuss what he thought was a deeper 
cause of the discrepancy in size between natural and artificial auto- 
mata. This is that many of the components of the natural system \/ 
serve to make the system reliable. As he noted in the third lecture, 
actual computing elements function correctly with a certain proba- 
bility only, not with certainty. In small systems the probability 
that the whole system will behave incorrectly is relatively small and 
may often be neglected, but this is not the case with large systems. 
Thus error considerations become more important as the system be- 
comes more complex. 

Von Neumann made some very rough calculations to justify this 



ROLE OF HIGH COMPLICATION 



71 



conclusion. Assuming that the system is designed in such a way that 
the failure of a single element would result in failure of the whole 
system, he calculated the error probability required for a given mean 
free path between system errors. For the human nervous system he 
used the following figures: 10 10 neurons; each neuron activated 10 
times per second on the average; a mean free path between fatal 
errors of 60 years (the average life span). Since 60 years is about 
2 X 10 9 seconds, the product of these numbers is 2 X 10 20 . Hence 
an error probability of 0.5 X 10~ 20 for each activation of an element 
is required under these assumptions. For a digital computer he used 
the figures: 5 X 10 3 vacuum tubes, 10 5 activations per tube per sec- 
ond, and a desired mean free path between system errors of 7 hours 
(about 2 X 10 4 seconds). An error probability of 10~ 13 per tube ac- 
tivation is required for this degree of reliability. Compare the calcu- 
lations at Collected Works 5.366-367. 

He pointed out that vacuum tubes, and artificial components 
generally, do not have an error probability as low as 10~ 13 , and that 
neurons probably do not either. We try to design computing ma- 
chines so that they will stop when they make an error and the opera- 
tor can then locate it and correct it. For example, a computer may 
perform a certain operation twice, compare the results, and stop if 
the results differ.] 

It's very likely that on the basis of the philosophy that every error 
has to be caught, explained, and corrected, a system of the complexity 
of the living organism would not run for a millisecond. Such a system 
is so well integrated that it can operate across errors. An error in it 
does not in general indicate a degenerative tendency. The system is 
sufficiently flexible and well organized that as soon as an error shows 
up in any part of it, the system automatically senses whether this 
error matters or not. If it doesn't matter, the system continues to 
operate without paying any attention to it. If the error seems to the 
system to be important, the system blocks that region out, by-passes 
it, and proceeds along other channels. The system then analyzes the 
region separately at leisure and corrects what goes on there, and if 
correction is impossible the system just blocks the region off and by- 
passes it forever. The duration of operability of the automaton is 
determined by the time it takes until so many incurable errors have 
occurred, so many alterations and permanent by-passes have been 
made, that finally the operability is really impaired. This is a com- 
pletely different philosophy from the philosophy which proclaims 
that the end of the world is at hand as soon as the first error has 
occurred. 

To apply the philosophy underlying natural automata to artificial 



THEORY OF SELF-REPRODUCING AUTOMATA 



automata we must understand complicated mechanisms better than 
we do, we must have more elaborate statistics about what goes wrong, 
and we must have much more perfect statistical information about 
the milieu in which a mechanism lives than we now have, ^m^iiaiigm^ 
■ ^akm can not be sep arated fro m the milieu to which it respon ds. 
By that 1 mean that it's meaningless to say that an automaton is 
good or bad, fast or slow, reliable or unreliable, without telling in 
what milieu it operates. The characteristics of a human for survival 
are well defined on the surface of the earth in its present state, though 
for most types of humans you must actually specialize the situation 
a little further than this. But it is meaningless to argue how the hu- 
man would survive on the bottom of the ocean or in a temperature of 
1000 degrees centigrade. Similarly, in discussing a computing machine 
it is meaningless to ask how fast or how slow it is, unless you specify 
what type of problems will be given to it. 

It makes an enormous difference whether a computing machine 
is designed, say, for more or less typical problems of mathematical 
analysis, or for number theory, or combinatorics, or for translating 
a text. We have an approximate idea of how to design a machine to 
handle the typical general problems of mathematical analysis. I 
doubt that we will produce a machine which is very good for number 
theory except on the basis of our present knowledge of the statistical 
properties of number theory. I think we have very little idea as to 
how to design good machines for combinatorics and translation. 

What matters is that the statistical properties of problems of 
mathematical analysis are reasonably well known, and as far as we 
know, reasonably homogeneous. Consider some problems in mathe- 
matical analysis which look fairly different from each other and which 
by mathematical standards are very different: finding the roots of an 
equation of the tenth order, inverting a matrix of the twentieth order, 
solving a proper value problem, solving an integral equation, or 
solving an integral differential equation. These problems are sur- 
prisingly homogeneous with respect to the statistical properties which 
matter for a computing machine: the fraction of multiplications to 
other operations, the number of memory references per multiplication, 
and the optimal hierarchic structure of the memory with respect to 
access time. There's vastly less homogeneity in number theory. There 
are viewpoints under which number theory is homogeneous, but we 
don't know them. 

So, it is true for all these automata that you can only assign them 
a value in combination with the milieu which they have to face. 
Natural automata are much better suited to their milieu than any 



ROLE OF HIGH COMPLICATION 



73 



artifacts we know. It is therefore quite possible that we are not too 
far from the limits of complication which can be achieved in artificial 
automata without really fundamental insights into a theory of in- 
formation, although one should be very careful with such statements 
because they can sound awfully ridiculous 5 years later. 

[ Von Neumann then explained why computing machines are de- 
signed to stop when a single error occurs. The fault must be located 
and corrected by the engineer, and it is very difficult for him to lo- 
calize a fault if there are several of them. If there is only one fault 
he can often divide the machine into two parts and determine which 
part made the error. This process can be repeated until he isolates 
the fault. This general method becomes much more complicated if 
there are two or three faults, and breaks down when there are many 
faults.] 

The fact that natural organisms have such a radically different 
attitude about errors and behave so differently when an error occurs 
is probably connected with some other traits of natural organisms, 
which are entirely absent from our automata. The ability of a natural 
organism to survive in spite of a high incidence of error (which our 
artificial automata are incapable of) probably requires a very high 
flexibility and ability of the automaton to watch itself and reorganize 
itself. And this probably requires a very considerable autonomy of 
parts. There is a high autonomy of parts in the human nervous sys- 
tem. This autonomy of parts of a system has an effect which is ob- 
servable in the human nervous system but not in artificial automata. 
When parts are autonomous and able to reorganize themselves, when 
there are several organs each capable of taking control in an emer- 
gency, an antagonistic relation can develop between the parts so that 
they are no longer friendly and cooperative. It is quite likely that 
all these phenomena are connected. 



Fifth Lecture 



RE - EVALUATION OF THE PROBLEMS 
OF COMPLICATED AUTOMATA- 
PROBLEMS OF HIERARCHY 
AND EVOLUTION 



Analysis of componentry and analysis of integration. Although these parts 
have to appear together in a complete theory, the present state of our informa- 
tion does not justify this yet. 

The first problem: Reasons for not going into it in detail here. Questions 
of principle regarding the nature of relay organs. 

The second problem: Coincides with a theory of information and of auto- 
mata. Reconsideration of the broader program regarding a theoretical discus- 
sion of automata as indicated at the end of the second lecture. 

Synthesis of automata. Automata which can effect such syntheses. 

The intuitive concept of "complication." Surmise of its degenerative 
character: In connection with descriptions of processes by automata and in 
connection with syntheses of automata by automata. 

Qualifications and difficulties regarding this concept of degeneracy. 

Rigorous discussion: Automata and their "elementary" parts. Definition 
and listing of elementary parts. Synthesis of automata by automata. The 
problem of self -reproduction. 

Main types of constructive automata which are relevant in this connection : 
The concept of a general instruction. The general constructive automaton 
which can follow an instruction. The general copying automaton. The self- 
reproducing combination. 

Self-reproduction combined with synthesis of other automata: The en- 
zymatic function. Comparison with the known major traits of genetic and 
mutation mechanisms. 

The questions on which IVe talked so far all bear on automata 
whose operations are not directed at themselves, so that they produce 
results which are of a completely different character than themselves. 
This is obvious in each of the three cases I have referred to. 

It is evident in the case of a Turing automaton, which is a box 
with a finite number of states. Its outputs are modifications of another 
entity, which, for the sake of convenience, I call a punched tape. 



74 



COMPLICATED AUTOMATA — HIERARCHY AND EVOLUTION 75 

This tape is not itself an object which has states between which it 
can move of its own accord. Furthermore, it is not finite, but is as- 
sumed to be infinite in both directions. Thus this tape is qualitatively 
completely different from the automaton which does the punching, 
and so the automaton is working into a qualitatively different me- 
dium. 

This is equally true for the automata discussed by McCulloch and 
Pitts, which are made of units, called neurons, that produce pulses. 
The inputs and outputs of these automata are not the neurons but 
the pulses. It is true that these pulses may go to peripheral organs, 
thereby producing entirely different reactions. But even there one 
primarily thinks, say, of feeding the pulses into motor or secretory 
organs, so it is still true that the inputs and outputs are completely 
different from the automaton itself. 

finally, it is entirely true for computing machines, which can be 
thought of as machines which are fed, and emit, some medium like 
punched tape. Of course, I do not consider it essentially different 
whether the medium is a punched card, a magnetic wire, a magnetized 
metal tape with many channels on it, or a piece of film with points 
photographed on it. In all these cases the medium which is fed to 
the automaton and which is produced by the automaton is completely 
different from the automaton. In fact, the automaton doesn't produce 
any medium at all; it merely modifies a medium which is completely 
different from it. One can also imagine a computing machine with 
an output of pulses which are fed to control completely different 
entities. But again, the automaton is completely different from the 
electrical pulses it emits. So there's this qualitative difference. 

A complete discussion of automata can be obtained only by taking 
a broader view of these things and considering automata which can 
have outputs something like themselves. Now, one has to be careful 
what one means by this. There is no question of producing matter 
out of nothing. Rather, one imagines automata which can modify 
objects similar to themselves, or effect syntheses by picking up parts 
and putting them together, or take synthesized entities apart. In 
order to discuss these things, one has to imagine a formal set-up like 
this. Draw up a list of unambiguously defined elementary parts. 
Imagine that there is a practically unlimited supply of these parts 
floating around in a large container. One can then imagine an autom- 
aton functioning in the following manner: It also is floating around 
in this medium ; its essential activity is to pick up parts and put them 
together, or, if aggregates of parts are found, to take them apart. 

This is an axiomatically shortened and simplified description of 



76 



THEORY OF SELF-REPRODUCING AUTOMATA 



what an organism does. It's true that this view has certain limitations, 
but they are not fundamentally different from the inherent limitations 
of the axiomatic method. Any result one might reach in this manner 
will depend quite essentially on how one has chosen to define the 
elementary parts. It is a commonplace of all axiomatic methods that 
it is very difficult to give rigorous rules as to how one should choose 
the elementary parts, so that whether the choice of the elements was 
reasonable is a matter of common sense judgment. There is no rigorous 
description of what choice is reasonable and what choice is not. 

First of all, one may define parts in such numbers, and each of 
them so large and involved, that one has defined the whole problem 
away. If you chose to define as elementary objects things which are 
analogous to whole living organisms, then you obviously have killed 
the problem, because you would have to attribute to these parts 
just those functions of the living organism which you would like to 
describe or to understand. So, by choosing the parts too large, by 
attributing too many and too complex functions to them, you lose 
the problem at the moment of defining it. 

One also loses the problem by defining the parts too small, for 
instance, by insisting that nothing larger than a single molecule, 
single atom, or single elementary particle will rate as a part. In this 
case one would probably get completely bogged down in questions 
which, while very important and interesting, are entirely anterior 
to our problem. We are interested here in organizational questions 
about complicated organisms, and not in questions about the struc- 
ture of matter or the quantum mechanical background of valency 
chemistry. So, it is clear that one has to use some common sense 
criteria about choosing the parts neither too large nor too small. 

Even if one chooses the parts in the right order of magnitude, there 
are many ways of choosing them, none of which is intrinsically much 
better than any other. There is in formal logics a very similar diffi- 
culty, that the whole system requires an agreement on axioms, and 
that there are no rigorous rules on how axioms should be chosen, 
just the common sense rules that one would like to get the system 
one is interested in and would not like to state in his axioms either 
things which are really terminal theorems of his theory or things 
which belong to vastly anterior fields. For example, in axiomatizing 
geometry one should assume theorems from set theory, because one 
is not interested in how to get from sets to numbers, or from numbers 
to geometry. Again, one does not choose the more sophisticated 
theorems of analytic number theory as axioms of geometry, because 
one wants to cut in at an earlier point. 



COMPLICATED AUTOMATA — HIERARCHY AND EVOLUTION 77 

Even if the axioms are chosen within the common sense area, it is 
usually very difficult to achieve an agreement between two people 
who have done this independently. For instance, in the literature of 
formal logics there are about as many notations as there are authors, 
and anybody who has used a notation for a few weeks feels that it's 
more or less superior to any other. So, while the choice of notations, 
of the elements, is enormously important and absolutely basic for 
an application of the axiomatic method, this choice is neither rigor- 
ously justifiable nor humanly unambiguously justifiable. All one can 
do is to try to submit a system which will stand up under common 
sense criteria. I will give an indication of how one system can be 
constructed, but I want to emphasize very strongly how relatively 
I state this system. 

I will introduce as elementary units neurons, a "muscle," entities 
which make and cut fixed contacts, and entities which supply energy, 
all defined with about that degree of superficiality with which the 
formal theory of McCulloch and Pitts describes an actual neuron. 
If you describe muscles, connective tissues, "disconnecting tissues," 
and means of providing metabolic energy, all with this degree of 
schematization, you wind up with a system of elements with which 
you can work in a reasonably uncomplicated manner. You probably 
wind up with something like 10 or 12 or 15 elementary parts. 

By axiomatizing automata in this manner, one has thrown half 
of the problem out the window, and it may be the more important 
half. One has resigned oneself not to explain how these parts are 
made up of real things, specifically, how these parts are made up of 
actual elementary particles, or even of higher chemical molecules. 
One does not ask the most intriguing, exciting, and important ques- 
tion of why the molecules or aggregates which in nature really occur 
in these parts are the sort of things they are, why they are essentially 
very large molecules in some cases but large aggregations in other 
cases, why they always lie in a range beginning at a few microns and 
ending at a few decimeters. This is a very peculiar range for an ele- 
mentary object, since it is, even on a linear scale, at least five powers 
of ten away from the sizes of really elementary entities. 

These things will not be explained; we will simply assume that 
elementary parts with certain properties exist. The question that 
one can then hope to answer, or at least investigate, is: What prin- 
ciples are involved in organizing these elementary parts into func- 
tioning organisms, what are the traits of such organisms, and what 
are the essential quantitative characteristics of such organisms? I 
will discuss the matter entirely from this limited point of view. 



78 THEORY OF SELF-REPRODUCING AUTOMATA 

[At this point von Neumann made the remarks on information, 
logic, thermodynamics, and balance which now appear at the end of 
the Third Lecture. They are placed there because that is where von 
Neumann's detailed outline located them. Those remarks are relevant 
to the present discussion because the concept of complication which 
von Neumann introduced next belongs to information theory.] 

There is a concept which will be quite useful here, of which we have 
a certain intuitive idea, but which is vague, unscientific, and imper- 
fect. This concept clearly belongs to the subject of information, and 
quasi-thermodynamical considerations are relevant to it. I know no 
adequate name for it, but it is best described by calling it "complica- 
tion." It is effectivity in complication, or the potentiality to do things. 
I am not thinking about how involved the object is, but how involved 
its purposive operations are. In this sense, an object is of the highest 
degree of complexity if it can do very difficult and involved things. 

I mention this because when you consider automata whose normal 
function is to synthesize other automata from elementary parts 
(living organisms and such familiar artificial automata as machine 
tools), you find the following remarkable thing. There are two states 
of mind, in each of which one can put himself in a minute, and in 
each of which we feel that a certain statement is obvious. But each 
of these two statements is the opposite or negation of the other! 

Anybody who looks at living organisms knows perfectly well that 
they can produce other organisms like themselves. This is their nor- 
mal function, they wouldn't exist if they didn't do this, and it's 
plausible that this is the reason why they abound in the world. In 
other words, living organisms are very complicated aggregations of 
elementary parts, and by any reasonable theory of probability or 
thermodynamics highly improbable. That they should occur in the 
world at all is a miracle of the first magnitude; the only thing which 
removes, or mitigates, this miracle is that they reproduce themselves. 
Therefore, if by any peculiar accident there should ever be one of 
them, from there on the rules of probability do not apply, and there 
will be many of them, at least if the milieu is reasonable. But a reason- 
able milieu is already a thermodynamically much less improbable 
thing. So, the operations of probability somehow leave a loophole at 
this point, and it is by the process of self -reproduction that they are 
pierced. 

Furthermore, it's equally evident that what goes on is actually one 
degree better than self -reproduction, for organisms appear to have 
gotten more elaborate in the course of time. Today's organisms are 
phylogenetically descended from others which were vastly simpler 



COMPLICATED AUTOMATA — HIERARCHY AND EVOLUTION 79 

than they are, so much simpler, in fact, that it's inconceivable how 
any kind of description of the later, complex organism could have 
existed in the earlier one. It's not easy to imagine in what sense a 
gene, which is probably a low order affair, can contain a description 
of the human being which will come from it. But in this case you can 
say that since the gene has its effect only within another human or- 
ganism, it probably need not contain a complete description of what 
is to happen, but only a few cues for a few alternatives. However, 
this is not so in phylogenetic evolution. That starts from simple 
entities, surrounded by an unliving amorphous milieu, and produces 
something more complicated. Evidently, these organisms have the 
ability to produce something more complicated than themselves. 

The other line of argument, which leads to the opposite conclusion, 
arises from looking at artificial automata. Everyone knows that a 
machine tool is more complicated than the elements which can be 
made with it, and that, generally speaking, an automaton A, which 
can make an automaton B, must contain a complete description of 
B and also rules on how to behave while effecting the synthesis. So, 
one gets a very strong impression that complication, or productive 
potentiality in an organization, is degenerative, that an organization 
which synthesizes something is necessarily more complicated, of a 
higher order, than the organization it synthesizes. This conclusion, 
arrived at by considering artificial automata, is clearly opposite to 
our early conclusion, arrived at by considering living organisms. 

I think that some relatively simple combinatorial discussions of 
artificial automata can contribute to mitigating this dilemma. Ap- 
pealing to the organic, living world does not help us greatly, because 
we do not understand enough about how natural organisms function. 
We will stick to automata which we know completely because we 
made them, either actual artificial automata or paper automata 
described completely by some finite set of logical axioms. It is possible 
in this domain to describe automata which can reproduce themselves. 
So at least one can show that on the site where one would expect 
complication to be degenerative it is not necessarily degenerative at 
all, and, in fact, the production of a more complicated object from a 
less complicated object is possible. 

The conclusion one should draw from this is that complication is 
degenerative below a certain minimum level. This conclusion is quite 
in harmony with other results in formal logics, to which I have re- 
ferred a few times earlier during these lectures. 1 We do not now know 

1 [ See the end of the Second Lecture.] 



80 



THEORY OF SELF-REPRODUCING AUTOMATA 



what complication is, or how to measure it, but I think that' some- 
thing like this conclusion is true even if one measures complication 
by the crudest possible standard, the number of elementary parts. 
There is a minimum number of parts below which complication is 
degenerative, in the sense that if one automaton makes another the 
second is less complex than the first, but above which it is possible 
for an automaton to construct other automata of equal or higher 
complexity. Where this number lies depends upon how you define 
the parts. I think that with reasonable definitions of parts, like those 
I will partially indicate later, which give one or two dozen parts with 
simple properties, this minimum number is large, in the millions. I 
don't have a good estimate of it, although I think that one will be 
produced before terribly long, but to do so will be laborious. 

There is thus this completely decisive property of complexity, 
that there exists a critical size below which the process of synthesis 
is degenerative, but above which the phenomenon of synthesis, if 
properly arranged, can become explosive, in other words, where 
syntheses of automata can proceed in such a manner that each autom- 
aton will produce other automata which are more complex and of 
higher potentialities than itself. 

Now, none of this can get out of the realm of vague statement 
until one has defined the concept of complication correctly. And one 
cannot define the concept of complication correctly until one has 
seen in greater detail some critical examples, that is, some of the 
constructs which exhibit the critical and paradoxical properties of 
complication. There is nothing new about this. It was exactly the 
same with conservation and non-conservation properties in physics, 
with the concepts of energy and entropy, and with other critical 
concepts. The simplest mechanical and thermodynamic systems had 
to be discussed for a long time before the correct concepts of energy 
and entropy could be abstracted from them. 

[ Von Neumann only briefly described the kinds of elements or 
parts he planned to use. There are neurons like those of McCulloch 
and Pitts. There are elements "that have absolutely no function 
except that they are rigid and produce a geometrical tie between 
their ends. ,, Another kind of element is called a "motor organ' ' and 
a "muscle-like affair"; it contracts to zero length when stimulated. 
There is an organ which, when pulsed, "can either make or break a 
connection." He said that less than a dozen kinds of elements are 
needed. An automaton composed of these parts can catch other parts 
which accidentally come in contact with it; "it is possible to invent 
a system by which it can sense" what part it has caught. 



COMPLICATED AUTOMATA HIERARCHY AND EVOLUTION 81 

In June of 1948 von Neumann gave three lectures on automata 
at the Institute for Advanced Study to a small group of friends. He 
probably did this in preparation for the Hixon Symposium which 
took place in September of that year. 2 These lectures contained the 
most detailed description of the parts of his self -reproducing autom- 
aton that I know of. For this reason, I have attempted to recon- 
struct, from the notes and memories of the audience, what he said 
about these parts and how they would function. 

Von Neumann described eight kinds of parts. All seem to have 
been symbolized with straight lines; inputs and outputs were in- 
dicated at the ends and/or the middle. The temporal reference frame 
was discrete, each element taking a unit of time to respond. It is not 
clear whether he intended this list to be complete; I suspect that he 
had not yet made up his mind on this point. 

Four of the parts perform logical and information processing opera- 
tions. A stimulus organ receives and transmits stimuli; it receives 
them disjunctively, that is, it realizes the truth-function u p or q." 
A coincidence organ realizes the truth-function u p and q." An in- 
hibitory organ realizes the truth-function u p and not-g." A stimuli 
producer serves as a source of stimuli. 

The fifth part is a rigid member, from which a rigid frame for an 
automaton can be constructed. A rigid member does not carry any 
stimuli; that is, it is an insulated girder. A rigid member may be 
connected to other rigid members as well as to parts which are not 
rigid members. These connections are made by a fusing organ which, 
when stimulated, welds or solders two parts together. Presumably 
the fusing organ is used in the following way. Suppose point a of one 
girder is to be joined to point b of another girder. The active or out- 
put end of the fusing organ is placed in contact with points a and b. 
A stimulus into the input end of the fusing organ at time t causes 
points a and b to be welded together at time t + 1. The fusing organ 
can be withdrawn later. Connections may be broken by a cutting 
organ which, when stimulated, unsolders a connection. 

The eighth part is a muscle, used to produce motion. A muscle is 
normally rigid. It may be connected to other parts. If stimulated at 
time t it will contract to length zero by time t + 1, keeping all its 
connections. It will remain contracted as long as it is stimulated. 
Presumably muscles can be used to move parts and make connections 
in the following way. Suppose that muscle 1 lies between point a of 

2 [ "The General and Logical Theory of Automata.' ' Collected Works 5.288- 
328. It will be recalled that the Illinois lectures were delivered in December of 
1949.] 



82 



THEORY OF SELF-REPRODUCING AUTOMATA 



one girder and point 6 of another girder, and muscle 2 lies between 
point a and the active end c of a fusing organ. When both muscles 
are stimulated, they will contract, thereby bringing points a, 6, and 
c together. When the fusing organ is stimulated, it will weld points 
a and b together. Finally, when the stimuli to the muscles are stopped, 
the muscles will return to their original length, at least one end of 
muscle 1 separating from the point ab. Von Neumann does not seem 
to have discussed the question of how the connections between mus- 
cles and other parts are made and broken. 

Von Neumann conceived of an automaton constructing other 
automata in the following manner. The constructing automaton 
floats on a surface, surrounded by an unlimited supply of parts. The 
constructing automaton contains in its memory a description of the 
automaton to be constructed. Operating under the direction of this 
description, it picks up the parts it needs and assembles them into 
the desired automaton. To do this, it must contain a device which 
catches and identifies the parts that come in contact with it. The 
June, 1948 lectures contain only a few remarks on how this device 
might operate. Two stimulus units protrude from the constructing 
automaton. When a part touches them tests can be made to see what 
kind of part it is. For example, a stimulus organ will transmit a sig- 
nal; a girder will not. A muscle might be identified by determining 
that it contracts when stimulated. 

Von Neumann intended to disregard the fuel and energy problem 
in his first design attempt. He planned to consider it later, perhaps 
by introducing a battery as an additional elementary part. Except 
for this addition, von Neumann's early model of self -reproduction 
deals with the geometrical-kinematic problems of movement, contact, 
positioning, fusing, and cutting, and ignores the truly mechanical 
and chemical questions of force and energy. Hence I call it his kine- 
matic model of self -reproduction. This early model is to be contrasted 
with his later cellular model of self -reproduction, which is presented 
in Part II of the present work. 

In his June, 1948 lectures von Neumann raised the question of 
whether kinematic self -reproduction requires three dimensions. He 
suspected that either three dimensions or a Riemann surface (multi- 
ply-connected plane) would be needed. We will see in Part II that 
only two dimensions are required for self -reproduction in von Neu- 
mann's cellular model. This is a strong indication that two dimensions 
are sufficient for kinematic self -reproduction. 

We return now to the Illinois lectures. Von Neumann discussed 
the general design of a self -reproducing automaton. He said that it 



COMPLICATED AUTOMATA — HIERARCHY AND EVOLUTION 83 

is in principle possible to set up a machine shop which can make a 
copy of any machine, given enough time and raw materials. This shop 
would contain a machine tool B with the following powers. Given a 
pattern or object X, it would search over X and list its parts and 
their connections, thereby obtaining a description of X. Using this 
description, the tool B would then make a copy of X, "This is quite 
close to self-reproduction, because you can furnish B with itself."] 

But it is easier, and for the ultimate purpose just as effective, not 
to construct an automaton which can copy any pattern or specimen 
given to it, but to construct an automaton which can produce an 
object starting from a logical description. In any conceivable method 
ever invented by man, an automaton which produces an object by 
copying a pattern will go first from the pattern to a description and 
then from the description to the object. It first abstracts what the 
thing is like, and then carries it out. It's therefore simpler not to 
extract from a real object its definition, but to start from the defini- 
tion. 

To proceed in this manner one must have axiomatic descriptions 
of automata. You see, I'm coming quite close to Turing's trick with 
universal automata, which also started with a general formal descrip- 
tion of automata. If you take those dozen elements I referred to in a 
rather vague and general way and give exact descriptions of them 
(which could be done on two printed pages or less), you will have a 
formal language for describing automata unambiguously. Now any 
notation can be expressed as a binary notation, which can be recorded 
on a punched tape with a single channel. Hence any automaton 
description could be punched on a piece of tape. At first, it is better 
not to use a description of the pieces and how they fit together, but 
rather a description of the consecutive steps to be used in building 
the automaton. 

[ Von Neumann then showed how to construct a binary tape out of 
rigid elements. See Figure 2. A binary character is represented at 
each intersection of the basic chain; "one" is represented by an at- 
tached rigid element, "zero" by the absence of a side element. Writing 
and erasing are accomplished by adding and removing side elements.] 

I have simplified unnecessarily, just because of a purely mathe- 
matical habit of trying to do things with a minimum of notation. 
Since I'm using a binary notation, all I'm attaching here is no side 
chain, or a one-step side chain. Existing languages and practical 
notations use more symbols than the binary system. There is no 
difficulty in using more symbols here; you simply attach more com- 
plex side chains. In fact, the very linearity of our logical notation is 



84 



THEORY OF SELF-REPRODUCING AUTOMATA 



completely unnecessary here. You could use more complicated looped 
chains, which would be perfectly good carriers for a code, but it would 
not be a linear code. There is reason to suspect that our predilection 
for linear codes, which have a simple, almost temporal sequence, is 
chiefly a literary habit, corresponding to our not particularly high 
level of combinatorial cleverness, and that a very efficient language 
would probably depart from linearity. 3 

There is no great difficulty in giving a complete axiomatic account 
of how to describe any conceivable automaton in a binary code. Any 
such description can then be represented by a chain of rigid elements 
like that of Figure 2. Given any automaton X, let 0(X) designate the 
chain which represents X. Once you have done this, you can design a 
universal machine tool A which, when furnished with such a chain 
0(X), will take it and gradually consume it, at the same time building 
up the automaton X from the parts floating around freely in the sur- 
rounding milieu. All this design is laborious, but it is not difficult in 
principle, for it's a succession of steps in formal logics. It is not quali- 
tatively different from the type of argumentation with which Turing 
constructed his universal automaton. 

Another thing which one needs is this. I stated earlier that it might 
be quite complicated to construct a machine which will copy an 
automaton that is given it, and that it is preferable to proceed, not 
from original to copy, but from verbal description to copy. I would 
like to make one exception; I would like to be able to copy linear 
chains of rigid elements. Now this is very easy. For the real reason it 
is harder to copy an existing automaton than its description is that 
the existing automaton does not conform with our habit of linearity, 
its parts being connected with each other in all possible directions, 
and it's quite difficult just to check off the pieces that have already 
been described. 4 But it's not difficult to copy a linear chain of rigid 
elements. So I will assume that there exists an automaton B which 
has this property: If you provide B with a description of anything, 
it consumes it and produces two copies of this description. 

Please consider that after I have described these two elementary 
steps, one may still hold the illusion that I have not broken the prin- 
ciple of the degeneracy of complication. It is still not true that, start- 
ing from something, I have made something more subtle and more 

3 [ The programming language of flow diagrams, invented by von Neumann, 
is a possible example. See p. 13 of the Introduction to the present volume.] 

4 [ Compare Sec. 1.6.3 of Part II, written about 3 years later. Here von 
Neumann gives a more fundamental reason for having the constructing auto- 
maton work from a description of an automaton rather than from the automa- 
ton itself.] 



COMPLICATED AUTOMATA — HIERARCHY AND EVOLUTION 85 

involved. The general constructive automaton A produces only X 
when a complete description of X is furnished it, and on any reason 
able view of what constitutes complexity, this description of X is as 
complex as X itself. The general copying automaton B produces two 
copies of <j>(X), but the juxtaposition of two copies of the same thing 
is in no sense of higher order than the thing itself. Furthermore, the 
extra unit B is required for this copying. 

Now we can do the following thing. We can add a certain amount 
of control equipment C to the automaton A + B. The automaton C 
dominates both A and 5, actuating them alternately according to 
the following pattern. The control C will first cause B to make two 
copies of <t>(X). The control C will next cause A to construct X at the 
price of destroying one copy of </>(X). Finally, the control C will tie 
X and the remaining copy of <f>{X) together and cut them loose from 
the complex (A + B + C). At the end the entity X + 4>(X) has been 
produced. 

Now choose the aggregate (A + B + C) for X. The automaton 
(A + B + C) + 4>(A + B + C) will produce (A + B + C) + 
<t>(A + B + C). Hence auto-reproduction has taken place. 

[ The details are as follows. We are given the universal constructor 
(A + B + C), to which is attached a description of itself, <f>(A + B + 
C). Thus the process of self -reproduction starts with (A + B + C) + 
<t>{A + B + C). Control C directs B to copy the description twice; 
the result is (A + B + C) + <t>(A + B + C) + <t>(A + B + C). 
Then C directs A to produce the automaton A + B + C from one 
copy of the description; the result is (A + B + C) + (A + B + C) + 
<j){A + B + C). Finally, C ties the new automaton and its description 
together and cuts them loose. The final result consists of the two 
automata (A + B + C) and (A + B + C) + <j>(A + B + C). If 
B were to copy the description thrice, the process would start with 
one copy of (A + B + C) + <f>(A + B + C) and terminate with 
two copies of this automaton. In this way, the universal constructor 
reproduces itself.] 

This is not a vicious circle. It is quite true that I argued with a 
variable X first, describing what C is supposed to do, and then put 
something which involved C for X. But I defined A and B exactly, 
before I ever mentioned this particular X, and I defined C in terms 
which apply to any X. Therefore, in defining A, B, and (7, I did not 
make use of what X is to be, and I am entitled later on to use an X 
which refers explicitly to A, B, and C. The process is not circular. 

The general constructive automaton A has a certain creative ability, 
the ability to go from a description of an object to the object. Like- 



80 



THEORY OF SELF-REPRODUCING AUTOMATA 



wise, the general copying automaton B has the creative ability to go 
from an object to two copies of it. Neither of these automata, however, 
is self -reproductive. Moreover, the control automaton C is far from 
having any kind of creative or reproductive ability. All it can do is to 
stimulate two other organs so that they act in certain ways, tie cer- 
tain things together, and cut these things loose from the original 
system. Yet the combination of the three automata A, B, and C is 
auto-reproductive. Thus you may break a self -reproductive system 
into parts whose functioning is necessary for the whole system to be 
self -reproductive, but which are not themselves self -reproductive. 

You can do one more thing. Let XbeA + B + C + D, where D 
is any automaton. Then (A + B + C) + <j>(A + B + C + D) pro- 
duces {A + B + C + D) + <t>(A + B + C + D). In other words, 
our constructing automaton is now of such a nature that in its normal 
operation it produces another object D as well as making a copy of 
itself. This is the normal function of an auto-reproductive organism: 
it creates byproducts in addition to reproducing itself. 

The system (A + B + C + D) can undergo processes similar to 
the process of mutation. One of the difficulties in defining what one 
means by self -reproduction is that certain organizations, such as 
growing crystals, are self -reproductive by any naive definition of 
self-reproduction, yet nobody is willing to award them the distinc- 
tion of being self -reproductive. A way around this difficulty is to 
say that self -reproduction includes the ability to undergo inheritable 
mutations as well as the ability to make another organism like the 
original. 

Consider the situation with respect to the automaton (A + B + 
C + D) + <t>(A + B + C + D). By a mutation I will simply mean a 
random change of one element anywhere. If an element is changed 
at random in one of the automata A, B, or C, the system will usually 
not completely reproduce itself. For example, if an element is changed 
in C, C may fail to stimulate A and B at the proper time, or it may 
fail to make the connections and disconnections which are required. 
Such a mutation is lethal. 

If there is a change in the description <j>(A + B + C + D), the 
system will produce, not itself, but a modification of itself. Whether 
the next generation can produce anything or not depends on where 
the change is. If the change is in A, B, or (7, the next generation will 
be sterile. If the change occurs in Z), the system with the mutation 
is exactly like the original system, except that D has been replaced 
by D'. This system can reproduce itself, but its by-product will be 



COMPLICATED AUTOMATA — HIERARCHY AND EVOLUTION 87 

D' rather than D. This is the normal pattern of an inheritable muta- 
tion. 

So, while this system is exceedingly primitive, it has the trait of an 
inheritable mutation, even to the point that a mutation made at 
random is most probably lethal, but may be non-lethal and inherita- 
ble. 



PART 



Two 



The Theory of Automata: 
Construction, Reproduction, 
Homogeneity 



EDITORIAL NOTE 



[The editor's insertions, commentaries, explanations, sum- 
maries, and Chapter 5 are in brackets. The figures are at the end of 
the volume. 

The reader who wishes a general view of the contents of this 
part should examine Sections 1.1.2.3, 1.3.3.5, 2.8.2, 2.8.3, 4.1.1, 4.3.1, 
and 5.3.] 



Chapter 1 



GENERAL CONSIDERATIONS 



1.1 Introduction 

1.1.1 J The theory of automata. The formalistic study of automata 
is a subject lying in the intermediate area between logics, communica- 
tion theory, and physiology. It implies abstractions that make it an 
imperfect entity when viewed exclusively from the point of view of 
any one of the three above disciplines — the imperfection being prob- 
ably worst in the last mentioned instance. Nevertheless an assimila- 
tion of certain viewpoints from each one of these three disciplines 
seems to be necessary for a proper approach to that theory. Hence 
it will have to be viewed synoptically, from the combined point of 
view of all three, and will probably, in the end, be best regarded as a 
separate discipline in its own right. 1 

1.1.1.2 The constructive method and its limitations. The present 
paper deals with a particular and limited phase of the formalistic 
theory of automata. The decisive limitation is that we will establish 
certain existence theorems, without, however, being able to prove 
that the constructions on which they are based are in any sense op- 
tima^ or that the postulates that they use are in any sense minimal. 
These questions of optimality and minimality could presumably be 
treated only if the methods for the formation of invariant quantitative 
concepts, and for their measurement, their evaluation, and the like, 
had been much further evolved in this subject of automata, control, 
and organization, than they are at present. We believe that such a 
development is possible and to be expected, and that it will to an 
important extent follow the patterns and the concept formations of 
thermodynamics. 2 The methods that will be used in this paper con- 

1 [Von Neumann here referred to Wiener. See Wiener's Cybernetics and von 
Neumann's review of it.] 

2 Von Neumann, "The General and Logical Theory of Automata" and 
' 'Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable 
Components." [See also the Third and Fourth Lectures of Part I of the present 
volume.] 



91 



92 



THEORY OF SELF-REPRODUCING AUTOMATA 



tribute, however, only very partially to the effort that is needed in 
that direction, and, at any rate, we will limit ourselves at this occasion 
to the establishing of certain existences (by suitable, ad hoc construc- 
tions) in the sense outlined above. 

1.1.2.1 The main questions: (A)-(E). Within the above limitations, 
however, we will deal with problems that are rather central — at least 
for the initial phases of the subject. We will investigate automata 
under two important, and connected, aspects: those of logics and of 
construction. We can organize our considerations under the headings 
of five main questions : 

(A) Logical universality. When is a class of automata logically 
universal, i.e., able to perform all those logical operations that are 
at all performable with finite (but arbitrarily extensive) means? 
Also, with what additional — variable, but in the essential respects 
standard — attachments is a single automaton logically universal? 

(B) Construe tibility. Can an automaton be constructed, i.e., as- 
sembled and built from appropriately defined "raw materials/' by 
another automaton? Or, starting from the other end and extending 
the question, what class of automata can be constructed by one, 
suitably given, automaton? The variable, but essentially standard, 
attachments to the latter, in the sense of the second question of (A), 
may here be permitted. 

(C) Construction-universality. Making the second question of (B) 
more specific, can any one, suitably given, automaton be construction- 
universal, i.e., be able to construct in the sense of question (B) (with 
suitable, but essentially standard, attachments) every other auto- 
maton? 

(D) Self -reproduction. Narrowing question (C), can any auto- 
maton construct other automata that are exactly like it? Can it be 
made, in addition, to perform further tasks, e.g., also construct cer- 
tain other, prescribed automata? 

(E) Evolution. Combining questions (C) and (D), can the con- 
struction of automata by automata progress from simpler types to 
increasingly complicated types? Also, assuming some suitable defini- 
tion of "efficiency," can this evolution go from less efficient to more 
efficient automata? 

1.1.2.2 The nature of the answers to be obtained. The answer to 
question (A) is known. 3 We will establish affirmative answers to 

3 Turing, "On Computable Numbers, with an Application to the Entschei- 
dungsproblem." [See the discussion of Turing machines and universal Turing 
machines at p. 49 ff . above. The indefinitely extendible tape of a Turing ma- 
chine is the ' 'variable, but essentially standard, attachment" von Neumann 
referred to in questions (A) and (B) above.] 



GENERAL CONSIDERATIONS 



93 



questions (B)-(D) as well. 4 An important limitation to the relevance 
of a similar answer to question (E) lies in the need for a more unam- 
biguous formulation of the question, particularly of the meaning of 
"efficiency." In addition, we will be able to treat questions (A)-(E) 
in this sense with much more rigid determinations as to what consti- 
tutes an automaton, namely with the imposition of what is best de- 
scribed as a crystalline regularity. In fact, this further result would 
seem to be at least as essential and instructive as the ability to answer 
questions (A)-(D) {and, to some extent question (E); cf. above} 
affirmatively. 

In the balance of Chapter 1 we carry out a heuristic and preliminary 
discussion of questions (A)-(E). In Chapter 2 we develop a specific 
model, within the terms of which we can and will deal in full detail 
and rigorously with questions (A)-(D). Chapter 3 contains the analy- 
sis of another more natural, but technically more refractory, model. 
Chapter 4 is devoted to further heuristic considerations, which are 
more conveniently made after (and to some extent presuppose) the 
detailed constructions of Chapters 2 and 3. 

[ 1.1.2.3 Von Neumann's models of self -reproduction. The preceding 
paragraph gives the plan von Neumann had in mind when he wrote 
the present chapter. Unfortunately, von Neumann was only able to 
carry out his intention through part of the planned Chapter 2. To 
understand the plan, and the several references he makes to it in the 
balance of the present chapter, one must know something about the 
various models of self -reproduction he considered. We will describe 
these models briefly in the present subsection. Of necessity, much of 
what we say here is based on personal communications from people 
with whom von Neumann discussed his models of self-reproduction. 

Altogether, von Neumann considered five models of self-reproduc- 
tion. We will call these the kinematic model, the cellular model, the 
excitation-threshold-fatigue model, the continuous model, and the 
probabilistic model. 

The kinematic model deals with the geometric -kinematic problems 
of movement, contact, positioning, fusing, and cutting, but ignores 
problems of force and energy. The primitive elements of the kinematic 
model are of the following kinds : logical (switch) and memory (delay) 
elements, which store and process information; girders, which provide 
structural rigidity; sensing elements, which sense objects in the en- 
vironment; kinematic (muscle-like) elements, which move objects 
around; and joining (welding) and cutting elements, which connect 

4 Von Neumann, "The General and Logical Theory of Automata," Collected 
Works 5.315-318. 



94 



THEORY OF SELF-REPRODUCING AUTOMATA 



and disconnect elements. The kinematic model of self -reproduction 
is described in the Fifth Lecture of Part I of the present work. As 
indicated there, von Neumann was thinking about it at least by 1948. 

Von Neumann's second model of self -reproduction is his cellular 
model. It was stimulated by S. M. Ulam, who suggested during a dis- 
cussion of the kinematic model that a cellular framework would be 
more amenable to logical and mathematical treatment than the frame- 
work of the kinematic model. 5 In the cellular model, self -reproduction 
takes place in an indefinitely large space which is divided into cells, 
each cell containing the same finite automaton. Von Neumann spoke 
of this space as a "crystalline regularity, " a "crystalline medium/' a 
"granular structure, " and as a "cellular structure. " 6 We will use the 
term cellular structure. 

There are many possible forms of cellular structure which may be 
used for self-reproduction. Von Neumann chose, for detailed develop- 
ment, an infinite array of square cells. Each cell contains the same 
29 -state finite automaton. Each cell communicates directly with its 
four contiguous neighbors with a delay of at least. 1 unit of time. Von 
Neumann developed this model in a manuscript entitled "Theory of 
Automata: Construction, Reproduction, Homogeneity/ ' which con- 
stitutes the present Part II of the present volume. In a letter to me, 
Mrs. Klara von Neumann said of her husband's manuscript: "I am 
quite positive that it was started by him in late September 1952 and 
that he continued working on it until sometime in mid late 1953." 
As far as I can tell, von Neumann did little or nothing with the manu- 
script after 1953. 

The manuscript as left by von Neumann had two completed chap- 
ters and a long but incomplete third chapter. Chapter 1 of the manu- 
script is the present chapter. Chapter 2 of the manuscript states the 
transition rule governing the 29-state cellular system; this is Chapter 
2 below. The incomplete Chapter 3 of the manuscript carries out the 
fundamental steps in the design of a cellular self -reproducing auto- 
maton; it appears below as Chapters 3 and 4. Von Neumann never 
completed the design of his cellular self -reproducing automaton; I 
indicate how to do this in Chapter 5 below. 

Von Neumann's cellular model of self -reproduction should be com- 
pared with some work of Ulam on cellular automata. In his A Collec- 

6 [See footnote 17 of Sec. 1.3.1.2 below. In his "Random Processes and Trans- 
formations," presented in 1950, Ulam described a cellular framework briefly 
and stated that it had been considered by von Neumann and him.] 

6 [Moore, "Machine Models of Self -Reproduction," suggested the name 
"tessellation model."] 



GENERAL CONSIDERATIONS 



95 



tion of Mathematical Problems, Ulam formulated a matrix problem 
arising out of the cellular model. In his "On Some Mathematical 
Problems Connected with Patterns of Growth of Figures" and "Elec- 
tronic Computers and Scientific Research, " Ulam studied the growth 
of figures in cellular automata with simple transition rules. He also \_y 
studied the evolution of successive generations of individuals with 
simple properties, each generation producing its successor in accord- 
ance with a simple, but non-linear recursive transformation. 

Under a covering letter dated October 28, 1952, von Neumann sent 
a copy of the present Chapter 1 to H. H. Goldstine. This letter elabo- 
rates the plan of Section 1.1.2.2 above. 

This is the introduction — or " Chapter 1" — that I promised you. It is 
tentative and incomplete in the following respects particularly: 

(1) It is mainly an introduction for "Chapter 2" which will deal with a 
model where every cell has about 30 states. It refers only very incompletely 
to "Chapter 3" in which a model with excitation-threshold-fatigue mecha- 
nisms alone will be discussed, and to "Chapter 4" where I hope to say some- 
thing about a "continuous" rather than "crystalline" model. There, as far 
as I can now see, a system of non-linear partial differential equations, essen- 
tially of the diffusion type, will be used. 

(2) The write-up is still in a quite "unliterary" form, i.e., there are no 
footnotes (only their places are indicated), references, explanations of the 
motivation, origin of the ideas, etc. 

It is clear that when von Neumann wrote the present Chapter 1 he 
had the following plan in mind. Chapter 2 was to contain a complete 
development of the cellular model of self -reproduction. Chapter 3 
was to treat an excitation-threshold-fatigue model of self-reproduc- 
tion. Finally, Chapter 4 was to discuss a continuous model of self- 
reproduction. Von Neumann finished the essential steps in the design 
of the cellular model and then stopped. Unfortunately, he never found 
time to finish the cellular model or write about the other two models. 

Von Neumann delivered the Vanuxem Lectures at Princeton 
University on March 2 through 5, 1953. There were four lectures, 
entitled "Machines and Organisms. " The fourth was devoted to self- 
reproduction; the kinematic model, the cellular model, the excitation- 
threshold-fatigue model, and the continuous model were all men- 
tioned. Since he had already agreed to give the manuscript "Theory 
of Automata: Construction, Reproduction, Homogeneity" to the 
University of Illinois Press, von Neumann did not himself want to 
write up these lectures separately. Instead, it was arranged that John 
Kemeny should write an article based on these lectures and the first 
two chapters of the manuscript. This was published in 1955 under 
the title "Man Viewed as a Machine." Much of the material of the 



90 



THEORY OF SELF-REPRODUCING AUTOMATA 



first three Vanuxem Lectures appeared later in The Computer and 
the Brain. 

The excitation-threshold-fatigue model? of self -reproduction was to 
be based on the cellular model. Each cell of the infinite structure of the 
cellular model contains a 29-state automaton. Von Neumann's idea 
was to construct this 29-state automaton out of a neuron-like element 
which had a fatigue mechanism as well as a threshold. Since fatigue 
plays an important role in the operation of neurons, an excitation- \/ 
threshold-fatigue model would be closer to actual systems than the 
cellular model. Von Neumann never discussed how an idealized neuron 
with fatigue would work, but we can design one by combining what 
he said about idealized neurons without fatigue with his account of 
the absolute and relative refractory periods of an actual neuron (cf. 
pp. 44-48 above and Collected Works 5.375-376). 
a„ An idealized excitation-threshold-fatigue neuron has a designated 
threshold and a designated refractory period. The refractory period is 
divided into two parts, an absolute refractory period and a relative 
refractory period. If a neuron is not fatigued, it becomes excited when- 
ever the number of active inputs equals or exceeds its threshold. When 
the neuron becomes excited two things happen: it emits an output 
signal after a specified delay, and the refractory period begins. The 
neuron cannot be excited at all during the absolute refractory period; 
it can be excited during the relative refractory period, but only if the 
number of active inputs equals or exceeds a threshold which is higher 
than the normal threshold. 

When an excitation-threshold-fatigue neuron becomes excited, it 
must remember this fact for the length of the refractory period and 
use this information to prevent input stimuli from having their normal 
effect on itself. Hence this kind of neuron combines switching, delay 
of output, and an internal memory with feedback to control the effect 
of incoming signals. Such a device is, in fact, a small finite automaton, 
that is, a device with inputs and outputs and a finite number of inter- 
nal states. In the fourth Vanuxem Lecture von Neumann suggested 
that a neuron with threshold 2 and fatigue period 6 might supply most 
of the states of the 29-state finite automaton needed in each cell of 
his cellular framework. 

7 [It should be noted that the phenomenon that von Neumann calls fatigue 
is more often called refractoriness. In this more common usage, fatigue is a 
phenomenon involving many refractory periods. The absolute refractory period 
of a neuron determines a maximum rate at which it can be fired. Repeated firing 
of a neuron at, or close to, this rate produces an increase in threshold, making it 
more difficult to fire the neuron. This increase in threshold is the phenomenon 
commonly called "fatigue."] 



GENERAL CONSIDERATIONS 



97 



The fourth model of self -reproduction which von Neumann con- 
sidered was a continuous model. He planned to base this on a system 
of non-linear partial differential equations of the type which govern 
diffusion processes in a fluid. Von Neumann had worked on non- 
linear partial differential equations, and wanted to use automata 
heuristically to solve theoretical problems about such equations (cf. 
pp. 33-35 above). In the case of the continuous model of self -reproduc- 
tion, he planned to proceed in the reverse direction, using non-linear 
partial differential equations to solve a problem of automata theory: 
the logical and mathematical nature of the process of self-reproduc- 
tion. This was part of von Neumann's general plan to employ the 
techniques and results of that branch of mathematics known as 

y analysis to solve problems in automata theory (cf. pp. 25-28 above). 
The physics, chemistry, biology, and logic of a self -reproducing 
system are very complex, involving a large number of factors; for 
example, mass, entropy, kinetic energy, reaction rates, concentration 
of enzymes and hormones, transport processes, coding, and control. 
All the essential properties of the self -reproducing system must be 
represented in the equations by functions or dependent variables. 
Von Neumann recognized that a system of simultaneous non-linear 
partial differential equations adequate to account for self -reproduction 
would be much more complex than the systems usually studied. 

Von Neumann had been trained as a chemical engineer and was 
therefore familiar with complex chemical reactions. He had also 
applied mathematics to complex physical systems of various kinds. 
He probably thought of the differential equations of self -reproduction 
in connection with his proposed excitation-threshold-fatigue model 
of self -reproduction. Assume that the cellular model is reduced to the 
excitation-threshold-fatigue model. The task then becomes that of 
formulating the differential equations governing the excitation, 
threshold, and fatigue properties of a neuron. The following processes 
are involved in neural activity. 8 The neuron is stimulated by inputs 1 
from other neurons. When the aggregate of these inputs reaches the 
threshold of the neuron, it excites the neuron by triggering a flow of 
sodium ions from the outside to the inside of the cell body. The flow 
or diffusion of ions causes the cell body to become depolarized. This 
diffusion and depolarization is then transmitted down the axon and 
constitutes the firing of the neuron. The firing is followed by a diffu- 

y sion of potassium ions from the inside of the neuron to the outside, 

8 [For a complete description see Eccles, The Neurophysiological Basis of 
Mind.] 



98 



THEORY OF SELF-REPRODUCING AUTOMATA 



which repolarizes the neuron. The chemical balance of sodium and 
potassium is restored still later. 

It is clear from the preceding description of the excitation, thresh- 
old, and fatigue processes of the neuron that chemical diffusion plays 
a fundamental role in these processes. This explains why von Neu- 
mann chose partial differential equations of the diffusion type for his 
continuous model of self -reproduction. The reason for von Neumann's 
choice of non-linear, rather than linear, differential equations is also 
k clear. The kinematic, cellular, and excitation-threshold-fatigue models 
all show that switching operations (e.g., threshold, negation) as well 
as control loops involving branching, feedback, and delay, are essen- 
tial to the logical, informational, and organizational aspects of self- 
reproduction. To model these discrete phenomena in a continuous 
system it is necessary to use non-linear partial differential equations. 

The preceding plan for constructing the continuous model starts 
with a discrete system and proceeds to a continuous system. The 
cellular model of self -reproduction is developed first, it is then reduced 
to the excitation-threshold-fatigue model, and finally, this model is 
described by non-linear partial differential equations. The reverse 
procedure is often followed in science, and von Neumann was, of 
course, familiar with it. One takes a continuous system, such as a 
fluid with shock waves in it, and approximates this system by dividing 
it up into discrete cells, treating everything in a cell as if it were in the 
same state. In this way, the differential equations of the continuous 
system are replaced by the difference equations of the discrete system. 
One may then solve the difference equations on a digital computer, 
and under appropriate conditions the solution will approximate the 
solution of the differential equations. 

But whatever the order of inquiry, a system of differential equa- 
tions and the corresponding difference equations represents essentially 
the same phenomena. The transition rule for the cellular model (Ch. 
2 below) is the difference equation version of the system of partial 
differential equations of the continuous model. The design of the pri- 
mary automaton which reproduces itself corresponds to the boundary 
conditions on these partial differential equations. Another way to view 
the contrast between the continuous and cellular models is in terms 
of the difference between an analog and a digital computer. An analog 
J computer is a continuous system, and a digital computer is a discrete 
system. Thus von Neumann's continuous model of self -reproduction 
stands in the same relation to analog computers as his cellular model 
of self -reproduction stands to digital computers. In Section 12 of his 
"Probabilistic Logics and the Synthesis of Reliable Organisms from 



GENERAL CONSIDERATIONS 



99 



Unreliable Components," he proposed a scheme for representing and 
processing digital information in an analog device. His continuous 
model of self -reproduction should be compared with this scheme. 

Von Neumann's continuous model of self -reproduction should also 
be compared with some work of Turing. In "The Chemical Basis of 
Morphogenesis,' ' Turing analyzed morphogenesis by solving differ- 
ential equations which describe the interaction, generation, and 
diffusion of chemical substances. Turing confined himself almost 
entirely to linear differential equations, but he touched on non-linear 
differential equations. 

Von Neumann had been interested in the applications of probability 
theory throughout his career; his work on the foundations of quantum 
mechanics and his theory of games are examples. When he became 
interested in automata, it was natural for him to apply probability 
theory here also. The Third Lecture of Part I of the present work is 
devoted to this subject. His "Probabilistic Logics and the Synthesis 
of Reliable Organisms from Unreliable Components" is the first work 
on probabilistic automata, that is, automata in which the transitions 
between states are probabilistic rather than deterministic. Whenever 
he discussed self -reproduction, he mentioned mutations, which are 
random changes of elements (cf. p. 86 above and Sec. 1.7.4.2 below). 
In Section 1.1.2.1 above and Section 1.8 below he posed the problems 
of modeling evolutionary processes in the framework of automata 
theory, of quantizing natural selection, and of explaining how highly 
efficient, complex, powerful automata can evolve from inefficient, 
simple, weak automata. A complete solution to these problems would 
give us a probabilistic model of self -reproduction and evolution. 9 ] 

1.2 The Role of Logics — Question (A) 

1.2.1 The logical operations — neurons. In evaluating question (A), 
one must obviously consider automata which possess organs that can 
express the essential propositions of logics and which need not possess 
any other organs. This can be done by using organs each of which 
possesses two stable states, corresponding to the basic truth values 
of true and false in logics. It is convenient to use a plausible physiologi- 
cal analogy and to designate these organs (whatever they are or are 
thought to be in reality) as neurons, and the two above states as 
excited and quiescent, respectively. It is also convenient to attach to 
these states digital (arithmetical) symbols, namely 1 and 0, respec- 

9 [For some related work, see J. H. Holland, "Outline for a Logical Theory of 
Adaptive Systems, " and "Concerning Efficient Adaptive Systems."] 



100 



THEORY OF SELF-REPRODUCING AUTOMATA 



tively. 10 The familiar structure of logics can then be conveyed to an 
automaton built from such organs by connecting them with lines 
representing the logical implications, and by introducing a separate 
species of basic organs, i.e., of neurons, for each basic logical opera- 
tion. 11 In the usual propositional calculus these are and, or, and not, 
to be designated by •, +, and — , respectively. 12 The lines which 
control the neuron's behavior, i.e., which represent the logical varia- 
bles that enter into the basic logical operation or function to which 
the neuron corresponds, are its inputs; the lines through which this 
neuron expresses its resulting behavior, i.e., which represent the value 
of the logical function in question, are the outputs. These are usually 
the inputs of other neurons. Instead of attributing to a neuron several 
outputs, it is preferable to allow only one, and to split it afterwards 
into as many branches as necessary. The time-element in the func- 
tioning of a neuron is best expressed by stipulating that the state 
prescribed by the logical function corresponding to the neuron (i.e., 
the value of that function) is assumed a fixed delay time r after the 
neurons that control this behavior have assumed their relevant states. 
That is, the response of a neuron (on its output line) occurs a fixed 
delay time r after the stimuli (on its input lines). It is unnecessary to 
allow propagation delays along lines; i.e., an output may be instan- 
taneously active wherever it is an input. It is simplest to assume that 
all relevant events take place at times t that are integer multiples 
of r: t = ut, n = 0, zfcl, ±2, • • • . Next, r may be chosen as the unit 
of time: r = 1, and so always t = 0, zbl, ±2, • • • . 

The basic neurons referred to above are shown in Figure 3. Their 
behavior is described by the following rules: 
1. a, b are the input lines; c is the output line of this neuron. 

2.1. The + neuron is excited at time t if and only if either the neuron 
with output line a or the neuron with output line b is excited at 
time t — 1. 

2.2. The • neuron is excited at time t if and only if both the neuron 
with output line a and the neuron with output line 6 are excited 
at time t — 1. 

2.3. The — neuron ["minus neuron"] is excited at time t if and only 
if the neuron with output line a is not excited (i.e., is quiescent) 
at time t — 1. 

10 Boolean algebra is then applicable. 

11 McCulloch and Pitts, "A Logical Calculus of the Ideas Immanent in 
Nervous Activity." 

12 Von Neumann, ' 'Probabilistic Logics and the Synthesis of Reliable Or- 
ganisms from Unreliable Components," Sec. 4. 



GENERAL CONSIDERATIONS 



101 



The time delay caused by the operation of each neuron guarantees 
the effective and constructive character of the logical system arrived 
at in this manner. 13 It is easily seen — in fact it is essentially inherent 
in the correspondence between the species of neurons introduced and 
the basic operations of logics (cf . above and Fig. 3) — that automata 
built from these organs can express all propositional functions in 
logics. 14 Beyond this, the inclusion of inductive processes, and more 
generally, of all processes that are permissible in finitistic logics, re- 
quires a deeper analysis. 15 It brings in one substantively new element: 
the need for an arbitrarily large (finite, but freely adjustable in size) 
memory. This ties in with question (B) and will be considered sub- 
sequently. 

1.2.2 Neural vs. muscular functions. Question (A) involved merely 
logical determinations; therefore it required only (at least directly 
only; cf., however, the last remark in Sec. 1.2.1) organs with two 
states, true and false. These two states are adequately covered by the 
neural states of excitation and quiescence. Question (B), on the other 
hand, calls for the construction of automata by automata, and it 
necessitates therefore the introduction of organs with other than 
logical functions, namely with the kinematical or mechanical attri- 
butes that are necessary for the acquisition and combination of the 
organs that are to make up the automata under construction. To use 
a physiological simile, to the purely neural functions must be added 
at least the muscular functions. 

At this point several alternatives open up. 

1.3 The Basic Problems of Construction — Question (B) 

1 .3.1 .1 The immediate treatment, involving geometry, kinematics, etc. 
The most immediate approach is this. The constituent organs are the 
neurons and lines necessitated by (A), plus such additional organs as 
(B) (i.e., the present discussion) will require. These constituent organs 
are to be conceived of as physical objects in actual space. Their acqui- 
sition and combination (including the establishing of rigid connections 
between them) must accordingly take place in actual space, i.e., 
3-dimensional, Euclidean space. (Further variations on the dimen- 
sionality and geometrical character of the space are possible, but we 

13 McCulloch and Pitts, op. cit. Von Neumann, op. cit., Sees. 2.1 and 3.3. 

14 McCulloch and Pitts, op. cit. Von Neumann, op. cit., Sec. 3. 

15 McCulloch and Pitts, op. cit. Von Neumann, op. cit., Sees. 3.3 and 5.1. 
[Von Neumann also mentioned Kleene. He probably intended to refer to the 
Rand Corporation version of "Representation of Events in Nerve Nets and 
Finite Automata."] 



102 



THEORY OF SELF-REPRODUCING AUTOMATA 



will not consider them at this point. Cf., however, the crystal-lattice 
discussions later on.) The constituent organs that are needed for the 
automaton construction must thus be found and acquired in space, 
they must be moved and brought into contact and fastened together 
in space, and all automata must be planned as true geometrical (and 
kinematical and mechanical) entities. The functions that were de- 
scribed above, rather symbolically, as muscular, will now be very 
nearly truly muscular, etc. Different degrees of abstraction are still 
possible; for example, one may or may not pay attention to the truly 
mechanical aspects of the matter (the forces involved, the energy 
absorbed or dissipated, etc.). But even the simplest approach, which 
disregards the above-mentioned properly mechanical aspects entirely 
requires quite complicated geometrical-kinematical considerations. 16 
Yet, one cannot help feeling that these should be avoided in a first 
attempt like the present one: in this situation one ought to be able 
to concentrate all attention on the intrinsic, logical-combinatorial 
aspects of the study of automata. The use of the adjective formalistic 
at the beginning of Section 1.1.1.1 was intended to indicate such an 
approach — with, as far as feasible, an avoidance of the truly geomet- 
rical, kinematical, or mechanical complications. The propriety of this 
desideratum becomes even clearer if one continues the above list of 
avoidances, which progressed from geometry, to kinematics, to me- 
chanics. Indeed, it can be continued (in the same spirit) to physics, 
to chemistry, and finally to the analysis of the specific physiological, 
physico-chemical structures. All these should come in later, succes- 
sively, and about in the above order; but a first investigation might 
best avoid them all, even geometry and kinematics. (A certain amount 
of primitive geometry and vestigial kinematics will appear even so, 
as will be seen later.) 

1.3.1.2 The non-geometrical treatment — structure of the vacuum. A 
more sophisticated approach, which goes far towards meeting the 
desiderata expressed above, is this. 17 

The need to use geometry (and kinematics) merely expresses the 
fact that even the vacuum (the sites not now occupied, but poten- 
tially occupiable, by the constituent organs of automata) has a struc- 
ture. Now 3-dimensional Euclidean space represents (or, represents 
with an approximation that is sufficient in the situation envisaged) 
the actual "structure of the vacuum." Nevertheless, this structure 
involves a number of traits which are unnecessarily complicating. 

16 Von Neumann, "The General and Logical Theory of Automata," Collected 
Works 5.315-318. [See also the Fifth Lecture of Part I above.] 

17 [Von Neumann was going to refer to S. Ulam here. See Sec. 1.1.2.3 above.] 



GENERAL CONSIDERATIONS 



L03 



While these will have to be considered at a later stage, it is desirable 
to eliminate them in the first approach. We will accordingly try to 
do this. 

1.3.2 Stationarity — quiescent vs. active states. The main complication 
that we wish to remove is the influence of kinematics, that is, the 
necessity of moving objects around. It is preferable to have stationary 
objects only, normally in a quiescent state, and to postulate a system 
which will, under suitably and precisely defined conditions, transfer 
them from the quiescent state into an active state — or into a particular 
one of several possible active states. 

1.3.3.1 Discrete vs. continuous framework. Next, these stationary 
and normally quiescent objects could be thought of as discrete en- 
tities, or as (infinitesimal) elements of a continuously extended 
medium. In the first case we will have a granular or cellular structure, 
while in the second case we are led back to a continuous space, 
more or less like that of Euclidean geometry. 

1.3.3.2 Homogeneity: discrete (crystalline) and continuous (Eu- 
clidean). We now make the simplifying, but also very restrictive 
assumption, 18 that this spatial or quasi-spatial substratum be homoge- 
neous. That is, the granular structure of the first case must have 
crystalline symmetry, 19 while the continuous space of the second case 
must be Euclidean. 20 In both cases this degree of homogeneity falls 
short of absolute homogeneity. Indeed, we have only postulated the 
homogeneity of the spatial (or, more broadly, combinatorial) matrix 
which carries the (quiescent or active) objects referred to above — the 
basic organs — but not the homogeneity of the population of these 
objects. That is, in the first (discrete) case, we have not postulated 
that all the cells of the crystal behave according to the same rules; 
and in the second (continuous) case, we have not postulated that 
the continuous, space-filling medium be subject everywhere to the 
same rules. Depending on whether this is, or is not, postulated, we 
will say that the system possesses, or does not possess, intrinsic, or 
functional, homogeneity? 1 

18 Calling for stronger results. [That is, it is more difficult to construct a self- 
reproducing automaton in a homogeneous medium than in an inhomogeneous 
medium.] 

19 [A crystal is a solid body having a regular internal structure and bounded 
by symmetrically arranged plane surfaces intersecting at definite and charac- 
teristic angles. The regular internal structure consists of the rows and patterns 
of atoms of the crystal. The faces of the crystal express this regular internal 
structure externally.] 

20 [Apparently von Neumann was going to explain in this footnote why he 
was excluding the non-Euclidean spaces of Bolyai-Lobachevski and Riemann.] 

21 [In the discrete (crystalline, granular, cellular) case, functional homoge- 



104 



THEORY OF SELF-REPRODUCING AUTOMATA 



Beyond this, an even more complete homogeneity would exist if all 
(discrete or infinitesimal) elements were in the same state, for exam- 
ple, in the quiescent state (cf. above). In this case we will speak of 
total homogeneity or of total quiescence, respectively. This can obviously 
not be postulated in general, since it would exclude any positive and 
organized functioning of the automaton, for example, all moves in 
the sense of questions (B)-(E) of Section 1.1.2.1 above. It is, how- 
ever, quite reasonable to strive to assume total quiescence as the 
initial state of an automaton. This cannot be absolutely enforced, 
since with the usual systems of rules total quiescence is a self-per- 
petuating state (cf. later). It will, however, be practical to assume a 
totally quiescent initial state and to proceed from there with the 
injection of a minimum amount of external stimulation (cf. later). 

1.3.3.3 Questions of structure: (P)-(R). The point made at the 
beginning of Section 1.3.3.2, namely, that the homogeneity assump- 
tions of Section 1.3.3.2 are seriously restrictive, is worth elaborating 
somewhat further. Indeed, even the manner in which the basic organs 
of the discussion of question (A) (the neurons of Section 1.2.1) are 
ordinarily strung together (cf. later), violates the first principle of 
homogeneity formulated in Section 1.3.3.2, that of the underlying 
granular structure, i.e., its crystalline symmetry. This degree of 
homogeneity can, however, be attained by some rather obvious and 
simple tricks, as will be seen later. 

The systems that are thus obtained will, however, still violate 
the further, stricter, principle of homogeneity formulated in Section 
1.3.3.2, that of functional homogeneity. This is clearly so, as long as 
several species of neurons are used (cf. Sec. 1.2.1, particularly Fig. 
3) and these have to be distributed over the crystal lattice in an 
irregular (i.e., not crystalline-symmetric) manner. However, this 
use of several species of neurons and this distribution of them in an 
irregular manner are natural if one approaches the problem in the 
way which is the obvious one from the point of view of the ordinary 
logical and combinatorial techniques (cf. later). We will see that this 
difficulty, too, can be overcome and that functional homogeneity 
can be achieved. This, however, is considerably less easy, and it 
constitutes, in fact, one of the main results of this paper. 

The homogeneity problem also raises some ancillary questions, 
which will successively occupy us when their respective turns come. 
They are the following : 

neity means that each cell is occupied by the same finite automaton and each 
such automaton is connected to its neighbors in the same way. The particular 
cellular structure which von Neumann adopts in Ch. 2 is functionally homoge- 
neous; see Sec. 1.3.3.5 below.] 



GENERAL CONSIDERATIONS 



105 



(P) Which is the minimum usable dimensionality? (This question 
arises both in the first — discrete, crystalline — and in the second 
— continuous, Euclidean — cases of Sees. 1.3.3.1 and 1.3.3.2.) 

(Q) In connection with functional homogeneity, can we require 
isotropy 22 in addition to homogeneity? {In the crystalline case 
this is meaningful for the regular crystal class only. 23 Cf. also 
question (R).} 

(R) In the crystalline case, which crystal classes are usable? Or, to 
introduce our real aim explicitly, what is the highest degree of 
regularity that can be used? 23 

With respect to question (P), one would expect that the dimen- 
sionality 3 is usable and probably minimal. In fact even 2 is usable, 
while it seems unlikely that 1 should be, at least in combination 
with any otherwise plausible postulates. 24 These questions will be 
considered in some detail later. 

Question (Q) will be considered later; it will appear that isotropy 
can be achieved, although non-iso tropic models, too, can be of con- 
siderable interest. 

As to question (R), we will use the maximum regularity, which is 
reasonably interpreted as the (body-centered ) cubic class in 3 dimen- 
sions, and the corresponding quadratic class in 2 dimensions, the 
emphasis being, in view of what was said about question (P) above, 
on the latter. Some other classes, however, are also of interest, as will 
be seen later. 

1.3.3.4 Nature of results, crystalline vs. Euclidean: statements (X)~ 
(Z). To conclude this group of observations, we note this. We sur- 
mise that the comparative study of the two cases of Section 1.3.3.1 
{i.e., of the crystalline (discrete) and of the continuous (Euclidean) 
case} will prove very rewarding. The available indications point 
strongly towards these conclusions: 

22 [A substance or space is isotropic insofar as it has the same properties in 
all directions. In the crystalline (discrete) case, functional isotropy means that 
each cell is connected to each of its immediate neighbors in the same way. The 
particular cellular structure which von Neumann adopts in Ch. 2 is functionally 
isotropic; see Sec. 1.3.3.5 below.] 

23 Crystal classes in two and three dimensions. [Crystals are divided into 
six groups or systems according to the number and nature of the axes of sym- 
metry. Crystals belonging to the cubic system (also called "the isometric sys- 
tem" and "the regular system") are the most symmetrical of all crystals. In a 
crystal of the cubic system the three crystallographic axes of reference are at 
right angles to each other and are equal in length. Crystals having a cubic cell 
and crystals having an octahedral cell are the simplest of the forms of this 
class.] 

24 [Von Neumann was going to make a reference to Julian Bigelow and H. 
H. Goldstine here. They suggested modeling self -reproduction in 2 rather than 
3 dimensions.] 



106 



THEORY OF SELF-KEPRODUCINli AUTOMATA 



(X) The general possibilities are about the same in the two cases. 

(Y) The continuous case is mathematically much more difficult than 
the crystalline case. 

(Z) If and when the appropriate analytical methods to deal with the 
continuous case are developed, this case will be more satisfac- 
tory and more broadly and relevantly applicable than the 
crystalline case. 

These matters will be discussed in somewhat more detail later. We 
will make now only one more observation, which relates to the diffi- 
culties referred to in conclusions (Y) and (Z). These difficulties are 
due to the fact that, in the case in question, the mathematical problem 
becomes one of a system of non-linear, partial differential equations. 
It may be of some significance that non-linear partial differential 
equations, which in many directions define and limit our mathemati- 
cal horizon, should make their appearance in this context also. 

The difficulties referred to in (Y) and (Z) will cause us to direct 
our attention primarily to the crystalline case. In fact, we will from 
now on always have this case in mind, except where the opposite is 
expressly stated. 

[1.3.3.5 Homogeneity, quiescence, and self -reproduction. In writing 
the present Part II of this volume, von Neumann was going through 
a process of reasoning which was to terminate in his cellular model 
of self -reproduction. At these early stages of the process he was 
exploring various possibilities, leaving specific choices until later. As 
the inquiry proceeded his terminology necessarily changed somewhat. 
Thus the "quiescence" of Section 1.3.3.2 above divides into "unexcita- 
bility" and "quiescence of an excitable cell" in Sections 1.3.4.1 and 
1.3.4.2 below. A brief preview of the final outcome may help the 
reader to follow the development of von Neumann's thought. 

Von Neumann's cellular structure is described in detail in Chapter 
2 below. He chose an infinite 2-dimensional array of square cells. 
Each cell is occupied by the same 29-state finite automaton, and 
each such automaton is connected to its four immediate neighbors in 
exactly the same way; that is, the transition rule of Chapter 2 below 
is the same for each cell. Hence this cellular structure is "functionally 
homogeneous" in the sense of Section 1.3.3.2 above. Since each 29- 
state automaton is connected to each of its four neighbors in the same 
way, this structure is also isotropic in the sense of Section 1.3.3.3 
above. Functional homogeneity and isotropy have only to do with 
structure, however, and not with content or state. Consequently, if 
different cells of a region of the cellular structure are in different 
states, one part of the region may act in one way and send informa- 



GENERAL CONSIDERATIONS 



107 



tion in one direction, while another part of the region acts in a differ- 
ent way and sends information in a different direction. 

The 29 states each cell is capable of assuming fall into three cate- 
gories: unexcitable (1), excitable (20), and sensitized (8). These 
are listed later in Figure 9. 

The unexcitable state U is utterly quiescent. This state plays a 
fundamental role with respect to the information content of the 
cellular structure, for this structure is operated in such a way that at 
each moment of time only a finite number of cells of the structure are 
in some state other than the unexcitable state. In this respect, the 
unexcitable state of von Neumann's system is analogous to a blank 
square on the tape of a Turing machine. Indeed, von Neumann 
represented zero in his linear array L by the unexcitable state U; see 
Section 1.4.2.5 below. 

The 20 excitable states fall into three classes. There are 4 confluent 
states C CC ' , where e and e range over 0 and 1; the value "0" sym- 
bolizes quiescence, and the value "1" symbolizes excitation. There 
are 8 ordinary transmission states T 0ac and 8 special transmission 
states Ti ae , where a = 0, 1, 2, 3 and e = 0, 1 as before. Eight trans- 
mission states are quiescent and 8 are excited. The sensitized states are 
transient in nature, each lasting exactly 1 moment of time. 

The set of 10 states consisting of the unexcitable state U and the 
quiescent states Coo , T ua o (u = 0, 1; a = 0, 1, 2, 3) has this property: 
if every cell of the infinite cellular structure is in 1 of these 10 states, 
the system will never change (i.e., no cell will ever change state). 
The difference between the unexcitable state U and the 9 quiescent 
(but excitable) states Coo , T wa o is in the way they respond to stimuli 
(excitations). Stimuli entering a cell which is in the unexcitable state 
U convert that cell into 1 of the 9 quiescent states C 0 o , T ua o . This 
conversion is the direct process of Section 2.6 below. The direct process 
takes 4 or 5 units of time, the sensitized states serving as interme- 
diaries in the process. 

A stimulus entering a cell in 1 of the 20 excitable states C e6 ' , 
T Mac (e, e', u = 0, 1; a = 0, 1, 2, 3) does one of two things. It may 
change the cell back to the unexcitable state U; this is the reverse 
process of Section 2.5 below. Alternatively, this stimulus may be 
switched, combined with other stimuli, and delayed in the usual way. 
In particular, a quiescent finite automaton may be embedded in an 
area & of the cellular structure by putting each of the cells of & in 1 
of the 10 states U, Coo , and T ua o . If such a finite automaton is ap- 
propriately designed, it will compute in the usual way when it is 
stimulated (activated). 



108 



THEORY OF SELF-REPRODUCING AUTOMATA 



The temporal reference frame for the cellular structure consists 
of the times • • • — 3, — 2, — 1, 0, 1, 2, 3, • • • . Von Neumann did not 
say exactly how he planned to use this temporal reference frame, but 
the following is consistent with what he said. All cells are in the unex- 
citable state for negative times. An initial cell assignment is a finite 
list of cells together with an assignment of a state to each cell of the 
list. At time zero an initial cell assignment is imposed on the cellular 
structure from the "outside," all cells not in the assignment list being 
left in the unexcitable state. Thereafter the cellular system runs 
according to the transition rule of Chapter 2. Each initial cell arrange- 
ment determines a unique history of the infinite cellular structure. 
We will call the infinite cellular structure together with an initial 
cell assignment an infinite cellular automaton. 

An infinite cellular automaton which models self-reproduction 
operates as follows. The finite automaton E of Section 1.6.1.2 below 
constitutes an initial cell assignment. More specifically, the initial 
or starting state of this finite automaton E is an initial cell assignment. 
We impose this initial cell assignment on the cellular structure at 
time zero, thereby embedding the finite automaton E in the cellular 
structure. Let & be the area of cells affected, so that initially all cells 
outside & are unexcitable. The logical structure of E is such that at 
some later time r another copy of E will appear in another area o! 
of the cellular structure. That is, the state of each cell of &' at time 
r is identical to the state of the corresponding cell of & at time zero. 
Thus E reproduces itself in area d' . In summary, this infinite cellular 
automaton has only one copy of E embedded in it at time zero, but 
two copies of E embedded in it at time r. This is self -reproduction. 

Let us look at the temporal development of an infinite cellular 
automaton. At negative times it is totally homogeneous in the sense 
of Section 1.3.3.2 above, all cells being unexcitable. At time zero this 
total homogeneity is modified by the introduction of inhomogeneity 
in a finite area. This inhomogeneity will, in general, propagate into 
surrounding areas. In the case of self -reproduction, the inhomogeneity 
of area Ct spreads until area d' is organized in the same way as area d. 

The above treatment of infinite cellular automata makes no essen- 
tial use of negative times. Since all cells are unexcitable at these times 
we could use only the times 0, 1, 2, 3, • • • without any loss of general- 
ity. Von Neumann did not say why he introduced negative times. It 
is possible that he planned to use them in connection with a probabi- 
listic model of self-reproduction and evolution (see the end of Sec. 
1.1.2.3 above).] 

1.8.4-1 Simplification of the problems of construction by the treatment 



GENERAL CONSIDERATIONS 



109 



according to Section 1.3.1.2. We can now return to the original thought 
of Sections 1.2.2-1.3.3.1, that is, to the necessity of having organs 
that perform analogs of the muscular rather than of the neural func- 
tion. In other words, we need organs which are concerned with the 
acquiring, positioning, and connecting of the basic organs of the 
automata under construction, rather than with purely logical opera- 
tions in the sense of Section 1.2.1. Since the properly kinematic 
aspects of "acquiring" and "positioning" have been removed by the 
observations of Sections 1.3.1.2 and 1.3.2, the nature of the function, 
referred to above as an "analog of the muscular," must now be re- 
considered. 

The remark at the end of Section 1.3.2 makes it clear that this 
function now appears under the aspect of causing an object — or, to 
use a terminology suggested by Section 1.3.3.1, a cell — which is in a 
quiescent state, to go over into a suitable active state. Now the logical 
functions, as discussed in Section 1.2.1, also do this, but there is a 
difference here or, at least, a possibility of a difference. The nature of 
this difference can be best illustrated by a physiological simile. 

1.8.4.2 Quiescence vs. activity; excitability vs. unexcitability; ordinary 
and special stimuli. A neuron may be quiescent or active, but it is 
at any rate pote ntially active; that is, if is an excitable cell. Connective 
tissue, on the other hand, consists of unexcitable, truly passive cells. 
So far the difference between an excitable, but (momentarily) quies- 
cent cell, and a (permanently) passive cell is obvious. Now let us for 
a moment introduce the fiction that the growth of neurons (i.e., of 
excitable cells) occurs not by the formation of new cells, but by the 
transformation of existing, unexcitable cells into excitable ones. It 
should be noted that, while this is not so in reality, it is the arrange- 
ment that fits best into the picture of stationary cells introduced in 
Section 1.3.2. To reconcile this with reality, one may have to inter- 
pret the absence of a cell as the presence of one in a special, particu- 
larly unexcitable state. This concept is in reasonable harmony with 
the one relating to a "structure of the vacuum," as used in Section 
1.3.1.2. 

Such a transformation must itself be induced by some special 
stimuli, i.e., by some special active states of neighboring cells. The 
ordinary stimuli (i.e., the ordinary active states) which control the 
logical functions discussed in Section 1.2.1 cannot do this. These 
stimuli control transitions between quiescent and ordinary active 
states, but in excitable cells only, and without ever changing the 
species (in the sense of Sec. 1.2.1) of the cell (neuron) in question. 
Indeed it was just with respect to these ordinary stimulations that 



110 



THEORY OF SELF-REPRODUCING AUTOMATA 



unexcitability was defined above. In order to provide an equivalent 
of growth, the special stimulations referred to above must be able to 
cause transitions from unexcitability to excitability, and also to de- 
termine the specific species (in the sense of Sec. 1.2.1 ) of the excitable 
cell (neuron) thus created. 

These concepts permit us to define the difference between quies- 
cence (with excitability) and unexcitability; the former responds to 
(i.e., is removed by) ordinary stimuli, the latter only to special ones. 
This has, of course, only shifted the distinction into one between 
ordinary and special stimuli, that is, into a distinction between ordi- 
nary and special active states. Leaving the physiological simile, and 
returning to the mathematical problem at hand, the matter still 
presents some dubious aspects. 

1.3.4.3 Critique of the distinctions of Section 1.3.4.2. Indeed, one 
must now consider critically the usefulness of the distinction between 
ordinary and special stimuli. As outlined above, the underlying idea 
is this. Ordinary stimuli are to be used for logical operations, taking 
the species of the neurons that are involved as fixed; that is, ordinary 
stimuli are to be used for the control and utilization of already satis- 
factorily organized sub-assemblies. Special stimuli are to be used 
for growth operations, involving the introduction of excitability, to- 
gether with a new determination of the neuronic species, into pre- 
viously unexcitable, or otherwise different areas. In other words, 
special stimuli are used for the organization (according to some logi- 
cally determined plan) of hitherto unorganized (or differently organ- 
ized) areas. 

This distinction is certainly convenient for a first approach, since 
it permits one to keep conceptually fairly distinct functions quite 
sharply distinct in their actual embodiment and performance. We will 
therefore adhere to it strictly in our first constructions (cf. later). 
However, it is quite possible to relax it by various logical and combina- 
torial devices to varying degrees, up to (and including) complete 
obliteration. These turn out subsequently to be quite desirable for 
various mathematical and conceptual reasons, and we will therefore 
introduce them in our later constructions (cf. later). 

[ Logical functions and growth functions are fairly distinct con- 
ceptually. In his preliminary discussion von Neumann made a sharp 
distinction between their respective representations in his cellular 
system. Logical functions are performed by ordinary stimuli, and 
growth functions are performed by special stimuli. Later he relaxed 
the distinction somewhat by using both types of stimuli for both 
types of functions. 



(JEN ERA L CO N S 1 1) EH ATI ONS 



The final distinction between ordinary and special stimuli is shown 
in Figure 9. Both ordinary and special transmission stimuli bring 
about "growth" from the unexcitable state U to one of the nine 
quiescent states Coo > T M «o (u = 0, 1; a = 0, 1, 2, 3); this is the direct 
process of Section 2.6 and Figure 10 below. Special transmission 
stimuli change an ordinary transmission state T 0 « € (a = 0, 1, 2, 3; 
e = 0, 1) or a confluent state C €C ' (e, e = 0, 1) into the unexcitable 
state U, while ordinary transmission stimuli change a special trans- 
mission state Ti ac (a = 0, 1, 2, 3; c = 0, 1) into the unexcitable 
state U; this is the reverse process of Section 2.5 below. 

The logical functions of disjunction, conjunction, and delay will 
normally be performed by arrays of ordinary transmission and con- 
fluent states. The logical function of negation is not directly repre- 
sented in von Neumann's system. Instead, negation will be accom- 
plished by breaking a communication path and later restoring it. 
The breaking will be done by the reverse process and the restoring 
by the direct process. An example is given in Figure 17 of Section 3.2 
below.] 

1.4 General Construction Schemes — Question (B) Continued 

1.4.1.1 Construction of cell aggregates — the built-in plan. The discus- 
sion up to this point (i.e., in Sees. 1.2.2-1.3.4.3) dealt only with the 
first part of question (B), the immediate problems of the construc- 
tion of one automaton by another automaton. We can now pass to 
the second part of question (B), i.e., consider by what means a single 
automaton can be made to construct broad classes of other automata, 
and how variable, but essentially standard, attachments can be used 
to facilitate and extend this process. 

Our discussion dealt so far only with the question: By what means 
can a single cell of specified characteristics be created? In this respect 
we developed some orienting principles. There remains the question 
of how this operation is to be controlled in all its details. It is clear 
that this will have to be done by the logical section of the primary 
(parent) automaton, which was considered in Section 1.2.1. It is 
also natural that this logical section of the primary automaton must 
supervise and sequence the multiplicity of acts of single-cell creation, 
which are necessary to produce the complete secondary (constructed ) 
automaton. 

This "sequencing" of the single cell creations has to be controlled 
by a logical pattern which is already laid out in the logical section of 
the primary automaton. Such a "logical pattern" is obviously neither 
more nor less than the complete "plan" of the secondary automaton 



112 



THEORY OF SELF-REPRODUCING AUTOMATA 



— functionally laid out within the primary automaton in "terms" 
that the primary automaton can "understand" and act on. 

Thus the plan of the secondary automaton must be "built into" the 
primary automaton, presumably in terms of logical connections in the 
sense of Section 1.2.1. 

1.4.1.2 The three schemes for building in multiple plans — the para- 
metric form. The conclusion of Section 1.4.1.1 is that a primary autom- 
aton, constructed for this purpose, is prima facie (i.e., assuming 
the simplest type of design, which presents itself most immediately) 
suited to construct one and only one secondary automaton. Generali- 
zations beyond this level are, however, immediate. 

First, it is, of course, possible to have the plans of several (different) 
secondary automata built into the primary. Second, it is possible to 
incorporate the logical facilities that will make the primary autom- 
aton construct a specific secondary several times, e.g., a certain, 
preassigned number of times. Third, the plan of the secondary may 
contain a number of numerical parameters; and this plan can be 
built into the primary in this (variable) parametric form, together 
with facilities that make it possible to substitute any desired numeri- 
cal values for these parameters. 

The third scheme — or, rather, the combination of the second and 
the third schemes — is the most general of these. In their immediate 
form, however, these still contain a limitation, that is, a limitation of 
the numbers that can be used for the parameter values (third scheme ) 
and for the number of repetitions (second scheme). Indeed, these 
numbers must be present in some form in the interior of the primary 
automaton, say in a digital representation. Assume that p such num- 
bers are involved, and that they are all integers ^ 0, say, vi , • • • , 
v p . Let each one of those cells, which are to be used for their repre- 
sentation, have k states available for this purpose. It is best to in- 
terpret these states as the base k digits 0, 1, • • • , k — 1. Let 
such cells be available for vi , where i = 1, • • • , p; this requires a 
total of n = ni + • • • + n p cells. Vi is thus expressed by n t digits in a 
base k digital notation; hence it is limited to the k n i values 0, 1, • • • , 
k n i - 1. 

1.4.2.1 The descriptive statement L for numerical parameters. The 
limitation just described can be circumvented by a simple trick: let 
these cells lie "outside" (i.e., not within the area of the primary 
automaton, but next to it), in the external, otherwise quiescent, re- 
gion of the crystal. They might, for example, form a linear array L 
extending in the right hand (i.e., positive x) direction away from the 
area of the primary automaton. The k states used for these "nota- 
tional" purposes must, of course, also be of a quasi-quiescent charac- 



GENERA L ( JONS] I >EIt ATI ONR 



ter, i.e., such that they will normally not disturb (stimulate or other- 
wise transform) each other or the surrounding quiescent cells. This, 
however, is a desideratum that is easy to meet (cf. later). The pri- 
mary automaton must then be endowed with the ability to establish 
contact with all parts of this linear array L, and to have its operations 
controlled, in the desired sense, by the "notational" (i.e., base k 
digital) states of the cells of L. One might, at first sight, expect 
difficulties in trying to do this for all possible L (for all possible sizes 
of L, i.e., of n) with a fixed, limited primary automaton. All these 
difficulties can, however, be overcome by fairly straightforward 
methods, as will appear when this matter is considered in detail [See 
Sec. 1.4.2.5 below]. We will mention here only one of them. 

One might think that the "exploration" of L is not possible without 
specifying — i.e., expressing within the primary automaton — the size 
of L, i.e., n. In fact, it would seem natural that p and all the v% , • • • , 
v p , must be so specified. This would again limit p and U\ , • • • , n p , 
since the primary automaton is a fixed entity, and hence it would 
limit L and through it the numbers that it represents. 

This difficulty can be removed as follows. Let each cell in L have 
k states for notational purposes as before; i.e., the states corresponding 
to the digits 0, 1, • • • , k — 1, and two additional states to be called 
comma and period. (All of these states must be "quasi-quiescent" in 
the sense indicated above.) Within L, the numbers p and v\ , • • • , 
v p consist only of cells in digital states. Now let L be lined up as 
follows (proceeding from left to right, i.e., in the positive .t-direction). 
The digits of p, a comma, the digits of v\ , a comma, • • • , the digits 
of Vp , a period. The primary automaton, in "exploring" L, can sense 
the comma and the period and thereby ascertain the sizes of p and 
of the vi , • • • , Vp no matter what these are. 

14.2.2 Applications of L. The linear array L of Section 1.4.2.1 
is the variable, but essentially standard attachment mentioned in 
question (B). It is the simple addendum to the primary automaton, 
which, although essentially quiescent and possessed only of the most 
rudimentary structure, expands the active potentialities of that 
automaton substantially, as appeared in Section 1.4.2.1. The possi- 
bilities that are inherent in this device will, however, become really 
clear only after this. The main application in this sense will be de- 
scribed later [Sees. 1.5 and 1.6]. We will consider first a lesser applica- 
tion. 

1.4.2.8 Use of L as an unlimited memory for (A). The last men- 
tioned application relates to the attachments to a purely logical 
automaton, referred to in question (A). 

The setup for purely logical functions (in the sense of (A), as dis- 



114 



THEORY OF SELF-REPRODUCING AUTOMATA 



cussed in Section 1.2.1) fails to be universal because of the absence of 
one constituent: an arbitrarily large memory which is finite but of 
adjustable size (cf. the end of Sec. 1.2.1). The linear array L, as 
described in Section 1.4.2.1, is just that. Hence L, with its ancillary 
observation, exploration, and construction facilities, provides the 
(variable, but in the essential respects standard) attachment to the 
logical automaton, which bridges the gap to logical universality, as 
indicated in question (A). It should be noted that the need for the 
facilities ancillary to L, referred to above, means that componentry 
which is normally called for in the primary (construction) automata 
of (B) must also be introduced into the (logical) automata of (A), 
if logical universality is an aim. For the details of all this, cf. later 
[Chapters 4 and 5]. 

14-® 4 Use of base two for L. One more remark about the cells of 
L is in order: we can choose k = 2, i.e., let all representations of 
numbers be base 2. Then each cell in L must have k + 2 = 4 states 
for the purposes now under consideration (cf. the discussion in Sec. 
1.4.2.1). If it is now desired to keep the number of states for nota- 
tional purposes at 2, this can still be achieved. It suffices to replace 
each cell of L by 2 cells, since a pair of 2-valued states allows 2 2 = 4 
combinations. 

The digitalization and punctuation scheme for L meets all the re- 
quirements of Section 1.4.2.1. It is, however, not the only possible one. 
The following is an obvious variant. Keep the punctuation states 
(the comma and the period), as in Section 1.4.2.1. Instead of the two 
base 2 digital states, designated 0 and 1, use only one, designated 1. 
Designate a number v (an integer ^ 0), not by a sequence of 0's and 
l's, which express its base 2 digital expansion, but simply by a se- 
quence of v l's. This representation is a good deal longer than that 
of Section 1.4.2.1 (v symbols instead of n, where n is the smallest 
integer with 2 n > v, i.e., with n > 2 log v), but it is more simply 
defined, and more simply exploitable (in the sense of the ancillary 
functions referred to in Sec. 1.4.2.3). For a detailed consideration, cf. 
later. 

[14-®>5 The linear array L. It may be helpful at this point to 
anticipate von Neumann's design of the mechanism for reading and 
writing on an arbitrary cell of the unlimited linear array L. 

Let us begin with a Turing machine, which is a finite automaton 
connected to an indefinitely extendible or infinite tape. The tape is 
divided into squares, each of which contains any one of a finite num- 
ber of characters (i.e., is in any one of a finite number of states). 
Let the basic alphabet consist of two characters: zero and one, repre- 



GENERAL CONSIDERATIONS 



1 15 



sented by a blank and a mark, respectively. At any given time the 
finite automaton senses one square of the tape. It can change the 
contents of this square (make a mark or erase a mark already there) 
and move the tape one square to the left or right, so that at the next 
moment of time it senses an adjacent square. Thus the finite autom- 
aton can, in a finite amount of time, gain access to, read, and modify 
any square of the tape. 

It is clear that accessibility of an arbitrary tape square is the im- 
portant thing, and having the tape move is only a means to this end. 
Alternatively, we can have the tape stand still and the finite autom- 
aton move back and forth along it. Or, we can have both the finite 
automaton and the tape stand still, and let the finite automaton 
communicate to an arbitrary square x n by means of a contractable 
and indefinitely extendible "wire." The finite automaton can sense 
and modify the state of square x n through this wire. Then the finite 
automaton can extend the wire to square x n+ i , or contract it to 
square x n -i . 

This last procedure is the one von Neumann used in his cellular 
system. The details are given in Chapter 4 below. We will explain 
the basic idea in connection with Figure 37. The memory control 
MC is a finite cellular automaton occupying the area indicated in 
that figure. L is an infinite array of cells extending to the right. 
"Zero" is represented in cell x n by the unexcitable state U, and "one" 
is represented by the quiescent but excitable state T030 , which is an 
ordinary transmission state directed downward. 

To read cell x n , the memory control MC sends a sequence of 
stimuli around the connecting loop Ci in the direction of the arrows. 
This sequence passes through x n without affecting its neighbors 
x n -i and .r n+ i , is modified according to the state of x n , and returns 
to the memory control MC with a representation of the contents of 
x n . The memory control MC then writes on x n and either extends 
the loop Ci so that it passes through cell x n +i , or contracts the loop 
Ci so that it passes through cell x n -i . The timing loop C 2 is used 
in this extension-contraction process and is extended (or contracted ) 
along with loop Ci . 

There are finitely many basic characters to be represented on L, 
including the period and comma. These are represented by binary 
sequences of some length k, and each character is stored in k cells of 
L. Initially, we will place a finite sequence of characters on L, of 
which the last one, and only the last one, is a period. Now, the memory 
control MC can sense the period and move it to the right or left as 
it expands or contracts the information on L. Hence, though MC is 



110 



THEORY OF SELF-REPRODUCI NG AUTOMATA 



of finite, fixed capacity, there is no bound to the amount of informa- 
tion on L with which it can interact.] 

1.5 Universal Construction Schemes — Question (C) 

1.5.1 Use of L for non-numerical (universal) parametrization. The 
schemes of Section 1.4.2.1 (together with Sec. 1.4.1.2) introduced an 
important broadening of the class of secondary automata that arc 
constructible by one, suitably given, (fixed) primary automaton, in 
the sense of the second question of (B). They do not, however, 
achieve immediately the construction universality that is the aim 
of question (C). We will now get this, too, by introducing one further- 
variation into the methods of Section 1.4.2.1. 

The class of secondary automata which can be constructed accord- 
ing to Section 1.4.2.1 by a single primary automaton is limited in this 
sense. (We disregard for a moment the influence of Sec. 1.4.1.2.) 
These (secondary) automata may represent a broad class, but they 
must nevertheless all be particular specimens from a common species; 
that is, their individual (construction) plans all derive from a common 
master plan in which certain available parameters are specifically 
numerically substituted. In other words, even though the specific 
plan of the secondary automaton must no longer be built into the 
primary automaton, nevertheless the underlying, generic plan — the 
plan that controls all the subordinate plans — must be built in there. 

1.5.2 The universal type of plan. Consider an arbitrary (but 
specifically given) secondary automaton and the possible ways to 
describe it. The following is certainly an adequate one: 

(a) Specify the (two) x and the (two) y coordinates of the four 
sides of a rectangular area in which the entire secondary automaton 
is to be contained. Let these coordinates be X\ , 2/1 , x 2 , and 2/2 . These 
coordinates should be counted from an origin which is at a suitably 
designated point within the area of the primary automaton. 

It is actually better to introduce the side lengths a = x 2 — x\ + 1, 
0 = 2/2 — yi + 1 (assuming x\ S £2,2/1 ^ 2/2) of the rectangular 
area containing the secondary, and to use the numbers £1,2/1, «, 0. 

(b) According to (a) above, each cell within the rectangle covering 
the secondary can be characterized by two coordinates i (= 0, 1, • • 

a — 1), j (= 0, 1, " • j P — 1). (To be precise, with respect to the 
system of coordinates used in (a) above, the coordinates of the cell 
i y j are Xi + i, 2/1 + j*) This gives, as it should, a/3 cells in the rec- 
tangle covering the secondary. Let £ be the number of states that 
each one of these cells can assume, using accordingly an index X = 
0, 1, •••,<£ — 1 to enumerate these states. Designate by \ij the 



GENERAL CONSIDERATIONS 



117 



state of cell which is desired (on the basis of the plan of the 

secondary automaton in question) for the moment when the construc- 
tion of this automaton is just completed. 

It is clear from (a) and (b) that the secondary automaton is 
completely characterized by the specification of the numbers x x , 
7/1 , a, j8 and X»v for all pairs i = 0, 1, • • • , a — 1; j =* 0, 1, • • • , 
0-1- 

Note that these numbers have the following ranges: 

xi , y x = 0, ±1, ±2 - - • . 
a 9 fi - 1,2, . 
X t y 0, 1, • • • , <£ — 1 for i = 0, 1, • • • , a — 1; 

/ ~ 0,1, ,0 - 1. 

To conclude, x\ , y\ are better represented by specifying their 
absolute values \xi\, \yi\ and two numbers e, 77: 

{ = 0 for xi = 0 1 f= 0 for yi^0\ 

e {= 1 for x x < 0|, 77 \= 1 for 2/j < 0|. 

Thus the sequence (of integers = 0) 

(«, ^, l^il, | r/i |, a, /S, 
(*) fa for < = 0,1, 1; j = 0, 1, ,0 - 1 

[ (the X t y are to be thought of as lexicographically ordered by 

contains a complete description of the desired secondary automaton, 
in the condition — i.e., with the cell-states — actually desired for its 
initial moment, immediately after completion. 

This sequence of numbers may now be treated with the method 
described in Section 1.4.2.1 for the simpler sequence that occurred 
there (the sequence p, v\ , • • • , v p ). That is, we can form a linear 
array of cells L, extending in the right hand (i.e., positive x) direc- 
tion, and made up as follows: the numbers enumerated in formula (*) 
in the order in which they appear there, each represented by its base 
k digital expansion, any two consecutive ones separated by a comma, 
and the last one followed by a period. The general description above 
plays now precisely the role of the general plan of a class of secondary 
automata, which contains parameters, as described in connection 
with the third scheme in Section 1.4.1.2. In addition to this, the linear 
array L introduced above is the exact equivalent of the linear array 
L introduced in Section 1.4.2.1: it specifies the numerical values that 
have to be substituted for the parameters of the general description. 
Thus the present description of an arbitrary secondary automaton has 
been made to fit entirely into the parametrization pattern of the 



118 



THEORY OF SELF-REPRODUCING AUTOMATA 



third scheme of Section 1.4.1.2. Since it is entirely unrestricted, this 
means that the universality referred to in question (C) can be 
achieved in this way. 

1.6 Self -Reproduction — Question (D) 

1.6.1.1 The apparent difficulty of using L in the case of self-reproduc- 
tion. Let us now consider question (D), that is, the problem of self- 
reproduction. 

The a priori argument against the possibility of self-reproduction 
is that it is natural to expect the constructing automaton to be more 
complex than the constructed one — i.e., the primary will be more 
complex than the secondary. 25 This is confirmed by the results out- 
lined in Sections 1.2.2-1.4.1.1, i.e., those dealing with the first ques- 
tion in (B) : the primary must contain a complete plan of the second- 
ary (cf. Sec. 1.4.1.1), and in this sense the primary is more complex 
than the secondary. This limitation is somewhat transformed, but 
not removed, by the subsequent developments of Sections 1.4.1.1- 
1.5.2; even the strongest one among the results that are discussed 
there (the answer to question (C) in Sec. 1.5.2, securing universality) 
is subject to one form of it. Indeed, this result calls for a complete 
description of the secondary, expressed by the linear array of cells L, 
to be attached to the primary. 

If one tried to pass from here directly to self-reproduction, it would 
be necessary to have an automaton which can contain its own plan, for 
example, in the form L. If the second question of (D) is included, 
it would also have to contain the plan (i.e., the L) of another, pre- 
scribed automaton. 

With the scheme of Section 1.5.2, even the first is impossible: the 
(secondary) automaton considered there has no more than af3 cells, 
while L (according to formula (*) in Sec. 1.5.2) consists of a/3 + 6 
digitalized numbers, afi + 5 commas, and a period (i.e., 2a/3 + 12 
or more cells). Many variants on this theme are possible, but none 
has yet appeared which, when directly used, overcomes this diffi- 
culty. However, there is an indirect method that circumvents it. 

1.6.1.2 Circumvention of the difficulty — the types E and E F . This 
method is as follows. 26 

Designate the universal (primary) automaton of Section 1.5.2 by 
A. A constructs any secondary whose description L is attached to A, 
as described in Section 1.5.2. 

25 [See also pp. 79-80 above.] 

26 Von Neumann, "The General and Logical Theory of Automata." [See 
also pp. 84-87 above.] 



GENERAL CONSIDERATIONS 



119 



It is possible to design and to position at a definite place adjacent 
to A another automaton B with the following function. B explores L 
and produces an exact copy l/ of it, placing L' in exactly the same 
position with respect to the secondary that L is in with respect to 
the primary A. The information necessary for this positioning can 
be obtained by the investigation of L, since the latter contains the 
numbers #1,3/1, a, 0, which describe the position of the secondary 
in question with respect to the primary A. 

Consider finally an automaton C which controls the two previously 
mentioned automata A and B as follows: C first causes A, as primary, 
to build the secondary S described by L. C then causes B to make a 
copy L' of L and to attach it to the secondary S as described above. 
Now designate the total aggregate of all three automata A, B, C by D. 

Designate the description L of this automaton D by L D . Note that 
L D must contain the numbers X\ , ij\ (indirectly, by way of e, 77, 
|#i|, |?/i|; cf. formula (*) in Sec. 1.5.2), a, j8, which describe the posi- 
tioning of the desired secondary with respect to the primary. There 
need be no doubt about the values of a, /3 that are to be used here, 
since one can ascertain how large a rectangle is needed to cover D. 
With respect to x\ , y\ , however, there is a real choice; these two 
coordinates define the relative position of the desired secondary with 
respect to the primary. Let us assume first that this choice will be 
made in some definite manner; it need only guarantee that the second- 
ary and its attachment L' will lie wholly outside the primary and 
its attachment L. Later on we will have somewhat more to say about 
this. 

Now consider the complex E which results from attaching L D to 
D. By going over the description given above, it is easily verified 
that E will construct precisely D with L D , displaced as above. Thus 
E is self -reproducing. 

This answers the first question of (D ) . The second question of (D ) 
can now also be answered along the same lines. Indeed, assume that 
in addition to self -reproduction, the construction of a further autom- 
aton F is also wanted. In this case, form the L which describes D 
followed by F: L D+F . Now consider the complex E F which results 
from attaching L D+F to D. It is clear that this will construct D, at- 
tach L D+F to it, and also construct F. In other words, it self -reproduces 
and constructs F in addition. 

The following remarks serve to clarify somewhat further the nature 
of the procedure outlined for (D ) . 

1.6.2.1 First remark: shape o/L. The construction of an automaton 
was based on generating separately every correct state of every cell 



120 



THEORY OF SELF-REPRODUCING AUTOMATA 



in a suitable covering area {cf. Sec. 1.5.2 (b), where this modus 
procedendi is indicated}. The covering area is conceived in a sim- 
plified, and therefore presumably often overextended, form as a 
rectangle {cf. Sec. 1.5.2 (a)}. The external attachment L is a linear 
array (cf. the last part of Sec. 1.5.2). These two geometrical shapes 
will not always fit together perfectly: covering them simultaneously 
by a rectangle may force an inelegant overextension of the latter. It 
should be pointed out that there is nothing immutable about the 
linear shape of L, and that one might well decide to change it (cf. 
later). On the other hand, the linear shape has the virtue of easy 
overall accessibility (cf. later [Ch. 4]). 

1 .6.2.2 Second remark: avoidance of collision in a single reproduction. 
As pointed out toward the end of Section 1.6.1.2, xi , yi must be so 
large that the secondary (whose position relative to the primary is 
defined by the coordinates Xi , yi) and its attachment L' will lie 
wholly outside the primary and its attachment L. Hence they are 
affected by the size of L. (L,' is congruent to L.) This creates the 
danger of a vicious circle, since L contains \yi\. 

However, this danger is not serious, and any one of the following 
procedures will obviate it. 

L (both for the primary L itself and for the secondary L,') extends 
in one direction only (the positive x-direction; cf. the end of Sec. 
1.5.2), which implies that it is quite thin in the ^/-directions (especially 
if it is linear; cf. above and also later). Therefore, a fixed minimum 
value for \yi\ can be assigned, which guarantees that neither D nor 
L of the primary and of the secondary collide, by virtue of their 
separation in the ^/-direction. 

Alternatively, a base k notation for |#i|, \yi\ (cf. Sees. 1.4.1.2 and 
1.4.2.1 ) guarantees that the area used for their designation, and there- 
fore L also, increases only as the 2 log of these numbers (cf. Sec. 
1.4.2.4), whereas the separation that they provide is essentially that 
of their own size. Clearly for sufficiently large numbers these will 
overtake their own 2 log to any desired degree. 

Finally, if each number is to be designated, as alternatively sug- 
gested in Section 1.4.2.4, by a sequence of as many ones as it expresses, 
we can still avoid any difficulty by, for example, agreeing that \xi\, 
\yi\ are to be squares of integers and that the numbers to be desig- 
nated by the means indicated above will be their square roots. Thus 
the required size of L will go with the square root of the separation 
provided, which is, like the 2 log, a slowly increasing function, sure 
to be adequately overtaken when the numbers get sufficiently large. 

As mentioned above, any one of these three devices is workable, 



GENERAL CONSIDERATIONS 



121 



and they are not the only ones. The actual procedure will be devel- 
oped later. 

1.6 .2.3 Third remark: analysis of the method for overcoming the 
difficulty of Section 1.6.1.1 — the role of L. It is worth recapitulating 
how the a priori argument against the possibility of self-reproduction, 
as stated in Section 1.6.1.1, was overcome in Section 1.6.1.2. 

The essential step was that D contained a sub-assembly B which is 
able to copy (and re-position) any linear array L. B is a fixed entity, 
of fixed, finite size, and it is yet able to copy an L of any size. It is 
essentially this step of "copying" which transcends the otherwise 
seemingly valid rule of the primary's necessary superiority (in size, 
also in organization) over the secondary. 

Now L = L G is the description of the secondary G that is to be 
constructed, as discussed in Section 1.5.2. (In our actual applications 
in Sec. 1.6.1.2, D and D + F played the role of G.) One might ask 
why the description L G is preferable to the original G in controlling 
the copying device B. In other words, why can B not copy directly 
G itself, i.e., why must the intermediary L G be introduced? This 
question is clearly of considerable semantic importance for the area 
in which we are now working, i.e., for a theory of automata. Indeed, 
it touches at the base of the entire question of notations and represen- 
tations, i.e., of the significance and advantages of introducing "de- 
scriptions" in addition to the original objects. 

The reason is this. In order to copy a group of cells according to 
the ideas of Section 1.6.1.2 concerning B, it is necessary to "explore" 
that group to ascertain the state of each one of its cells and to induce 
the same state in the corresponding cell in the area where the copy 
is to be placed. This exploration implies, of course, affecting each cell 
of this group successively with suitable stimuli and observing the 
reactions. This is clearly the way in which the copying automaton B 
can be expected to operate, i.e., to take the appropriate actions on 
the basis of what is found in each case. If the object under observa- 
tion consists of "quasi-quiescent" cells (cf. the remarks made on this 
subject in Sec. 1.4.2.1), then these stimulations can be so arranged 
as to produce the reactions that B needs for its diagnostic purposes, 
but no reactions that will affect other parts of the area which has to 
be explored. If an assembly G, which may itself be an active 
automaton, were to be investigated by such methods, one would have 
to expect trouble. The stimulations conveyed to it, as discussed above, 
for "diagnostic" purposes, might actually stimulate various parts of 
G in such a manner that other regions could also get involved, i.e., 
have the states of their cells altered. Thus G would be disturbed; 



122 



THEORY OF SELF-REPRODUCING AUTOMATA 



it could change in ways that are difficult to foresee, and, in any case, 
likely to be incompatible with the purpose of observation; indeed, 
observing and copying presuppose an unchanging original. The 
virtue of L G (as compared to G) is that, since it consists of quasi- 
quiescent cells, no such complications (i.e., no spreading of the diag- 
nostic stimulations) need be expected. (For the details of all this, 
cf. later [Ch. 4].) 

The above requires one more qualification. Our choice actually did 
not lie between the copying of G and the copying of L G . It was rather 
the copying of G on the one hand, and the copying of L G , combined 
with the construction of G from its description L G , on the other 
hand. The last step in the second procedure, however, is feasible, 
since this is precisely what the universal constructing automaton in 
the sense of question (C) will do, according to Section 1.5.2. Note 
also that the quasi-quiescent character of L = L G is important in 
this construction step too; in fact, the observations of Section 1.4.2.1 
concerning quasi -quiescence in L were aimed directly at this applica- 
tion. 

1.6.3.1 Copying: use of descriptions vs. originals. It is worthwhile 
to observe at this point, too, why a third step, namely the construc- 
tion of L G , based on a direct exploration of the original G, cannot be 
carried out with these methods. Note that if this could be done, then a 
suitable primary automaton could copy a given automaton G with- 
out ever having been furnished with its description L G . Indeed, one 
would begin with the step mentioned above, the construction of L G 
from G, and then proceed with the two steps mentioned previously, 
the copying of L G and the construction of G from L G . The difficulty 
is that the two last mentioned steps require only the observation of 
the quasi-quiescent L G , while the first mentioned step would also 
call for the observation of the uncontrollably reactive G. If one 
considers the existing studies concerning the relationship of automata 
and logics, it appears very likely that any procedure for the direct 
copying of a given automaton G, without the possession of a descrip- 
tion L G , will fail; otherwise one would probably get involved in 
logical antinomies of the Richard type. 27 

To sum up, the reason to operate with "descriptions" L G instead of 
the "originals" G is that the former are quasi-quiescent (i.e., un- 
changing, not in an absolute sense, but for the purposes of the ex- 
ploration that has to be undertaken), while the latter are live and 
reactive. In the situation in which we are finding ourselves here, the 

27 [Von Neumann indicated that he was going to make a footnote reference 
to Turing at this point. See Sec. 1.6.3.2 below.] 



GENERAL CONSIDERATIONS 



123 



importance of descriptions is that they replace the varying and reac- 
tive originals by quiescent and (temporarily) unchanging semantic 
equivalents and thus permit copying. Copying, as we have seen 
above, is the decisive step which renders self -reproduction (or, more 
generally, reproduction without degeneration in size or level of 
organization) possible. 

[ 1 .6.3.2 The Richard paradox and Turing machines. As indicated 
above, von Neumann was going to make a footnote reference to 
Turing in connection with the Richard paradox. I do not know what 
he had in mind, but I think it likely that he was going to mention 
the parallelism between Richard 's paradox 28 and Turing's proof of 
the undecidability of the halting problem. In any case, this paral- 
lelism is illuminating in the present context. 

Richard's paradox may be generated in a suitable language <£ as 
follows. Let 6o,6i,6 2 , • • • be an enumeration of all the expressions of 
£ which define two-valued number-theoretic functions of one varia- 
ble, that is, functions from the natural numbers to the two values 
zero and one. The expression "x is odd" is such an expression; it 
defines a function which is true (has the value 1 ) for odd numbers 
and is false (has the value 0) for even numbers. Let fi(n) be the 
number-theoretic function denned by , and define —fi(n) by 

-/<(n) = 0if/<(n) = 1 
-fdn) = 1 if/«(n) = 0. 

Finally, let e be the expression "the function —f n (n)." 

We assume that e is expressible in «£, and derive a contradiction. 
(1) The enumeration e 0 , &i , , • • • contains all the expressions of £ 
which define two-valued number-theoretic functions of one variable. 
Expression e clearly defines a two-valued number-theoretic function 
of one variable. Therefore expression e is in the enumeration e 0 , 
ei , e 2 , • • • . (2) But e is an explicit definition of the function — f n (n), 
which differs from every function in the enumeration fo(n), fi(n), 
/ 2 (n), • • • . Therefore e does not define any of the functions /o(w), 
fi(n), • • • . For each i, fi(n) is defined by . Consequently, 

e is not in the enumeration e 0 , #i , 62 , • • • . 

Thus we have shown both that the expression e is in the enumera- 
tion 6 0 , 61 , 62 , • • • and that it is not in this enumeration. The appear- 
ance of this contradiction is surprising, because it would seem that 
expression e is a legitimate expression in a consistent language, 
namely, the English language enriched with some mathematical 

28 [Richard, "Les principes des mathematiques et le probleme des ensem- 
bles. " See also Kleene, Introduction to Metamathematics , pp. 38, 341.] 



124 



THEORY OF SELF-REPRODUCING AUTOMATA 



symbols. Actually, the contradiction shows that if a language £ is 
consistent then e cannot be expressed in it. 

Let us turn next to the halting problem for Turing machines. This 
problem was explained at the end of the Second Lecture of Part I 
above. A Turing machine is a finite automaton with an indefinitely 
extendible tape. A "concrete Turing machine ,, is a Turing machine 
which has a finite "program" or problem statement on its tape ini- 
tially. A concrete Turing machine is said to be "circular" if it prints 
a finite sequence of binary digits and halts, while it is said to be 
"circle-free" if it continues to print binary digits in alternate squares 
forever. Turing proved that there is no decision machine for halting, 
that is, no abstract Turing machine which can decide whether an 
arbitrary concrete Turing machine is circular (will halt sometime) 
or circle-free. 

Turing's proof that there is no decision machine for halting may 
be put in a form which closely parallels the preceding proof concern- 
ing Richard's paradox. Let fa , fa , fa , • • * , U , • • • be an enumeration 
of all the circle-free concrete Turing machines. Let s*(0), s»(l), 
Si (2), ••• , Si(n), ••• be the sequence computed by machine fa . 
Each Si(n) is either zero or one, so machine fa computes the two- 
valued function s t -(n) in the sense of enumerating its values in their 
natural order. Now consider the function — s n (n). This function is 
analogous to the function —f n (n) defined by expression e in the 
Richard paradox. 

To continue the parallelism, we assume that there is a circle-free 
concrete Turing machine t' which computes the function — s n (n) 
and derive a contradiction. (1) The enumeration to , fa , fa , • • • con- 
tains all circle-free concrete Turing machines. Machine t' is by hy- 
pothesis a circle-free concrete Turing machine. Consequently, machine 
t' is in the enumeration to , fa , fa , • • • . (2) By definition, { computes 
the function —s n (n), which clearly differs from every function in 
the enumeration So(ft), Si(ft), s 2 (n), ••• . For each i, the function 
s» (n) is computed by the machine fa . Consequently, machine t' is 
not in the enumeration fa , fa , fa , * • • . 

Thus we have shown both that machine { is in the enumeration 
U , t\ , fa ' ' ' and that it is not. The appearance of this contradiction 
is not surprising, however, for we had no reason to believe that 
machine t' exists. In other words, the contradiction shows that ma- 
chine { does not exist, and hence that the function — s„(ft) is not 
computed by any circle-free concrete Turing machine. 

We assume next that there is a decision machine t h for halting, and 
derive a contradiction. There is a concrete Turing machine which 



GENERAL CONSIDERATIONS 



can enumerate all the concrete Turing machines; call it t\ The output 
of t e can be fed into t h to produce a machine t e + t h which enumerates 
all the circle-free concrete Turing machines. There is an abstract 
Turing machine t u which can simulate each circle-free concrete 
machine in turn, find s n (n) for each machine n, and print — s n (n). 
Thus the machine f + t h + t u computes the function — s n (n), and 
is the machine t . But we know from the preceding paragraph that 
machine t' does not exist. Machines f and t u do exist. Therefore, 
machine t h does not exist. That is, there is no decision machine for 
halting. It follows also that machine f + t h does not exist, i.e., there 
is no machine which enumerates all the circle-free concrete Turing 
machines. 

The first part of the preceding proof that there is no decision ma- 
chine for halting establishes both that machine { is in the enumeration 
to , h , h , * * * and that it is not. This closely parallels the earlier 
proof, given in connection with Richard's paradox, that the expres- 
sion e is in the enumeration e 0 , #i , e 2 , • • • and that it is not. Both 
use Cantor's diagonal procedure to define a function which is not 
in a given enumeration. I suspect that it was because of this paral- 
lelism that von Neumann was going to refer to Turing at this point. 

It should be noted that the Richard paradox can be barred from a 
language by imposing a "theory of types" on that language. 29 For 
example, we can design the language so that every expression of the 
language has a type number, and so that an expression of given type 
can refer only to expressions of lower type. Suppose now that the 
expressions e 0 , #i , e 2 , • • • are of type m. Since expression e refers 
to all these expressions, it must be of higher type, and therefore 
cannot be in the list e 0 , e\ , e 2 , * • • . This being so, our earlier deriva- 
tion of Richard's paradox fails. See in this connection the letter 
from Kurt Godel quoted at the end of the Second Lecture of Part I 
above. 

These considerations about self -reference are relevant to the prob- 
lem of designing a self -reproducing automaton, since such an autom- 
aton must be able to obtain a description of itself. In Section 1.6.3.1 
(entitled "Copying: use of descriptions vs. originals") von Neumann 
considers two methods for accomplishing this, which I will call the 
"passive" and "active" methods. In the passive method the self- 
reproducing automaton contains within itself a passive description 
of itself and reads this description in such a way that the description 

29 [Russell, ' 'Mathematical Logic as Based on the Theory of Types." See 
also Kleene, Introduction to Metamathematics , pp. 44-46.] 



126 



THEORY OF SELF-REPRODUCING AUTOMATA 



cannot interfere with the automaton's operations. In the active 
method the self-reproducing automaton examines itself and thereby 
constructs a description of itself. Von Neumann suggests that this 
second method would probably lead to paradoxes of the Richard 
type, and for this reason he adopts the first method. See also Sections 
1.7.2.1, 2.3.3, 2.6.1, and 2.8.2 below. We will see by the end of Chapter 
5 below that a self -reproducing machine can indeed be constructed 
by means of the first method. This shows that it is possible for an 
automaton to contain a description of itself.] 30 

1.7 Various Problems of External Construction Intermediate 
Between Questions (D) and (E) 

1.7.1 Positioning of primary, secondary, ternary, etc. We pass now 
to an extension of question (D) which points the way towards 
question (E). This deals with the question of positioning the second- 
ary that the self -reproducing primary E or E F constructs, and the 
initiation, timing, and repetitions of the act of self-reproduction. 

Note that the positioning of F for E F need not create any new prob- 
lems: E F is D with L D+F attached (cf. the end of Sec. 1.6.1.2) and 
L D+F is a description of D followed by a description of F. In this 
joint description of D with F the latter must be unambiguously posi- 
tioned with respect to the former, and this takes care of what is 
needed in this respect. 

Returning to the main question of positioning the secondary by E 
or E F , we can argue as follows. Assume that this positioning is done 
by the first method of Section 1.6.2.2, i.e., by choosing a y such that 
I Vi | ^ y guarantees the separateness of the primary and the second- 
ary E or E F . (In the case of E F we think of both primary and second- 
ary as provided with an F positioned according to L D+F relatively 
to D; cf. above, although at the beginning of the process only the 
secondary need be accompanied by such an F.) Let the origin of the 
x, ^/-coordinate system referred to in Section 1.5.2 lie in the 
extreme lower left corner of the rectangle covering the primary, 
i.e., at the point that corresponds to the one designated by Xi , yi 
in the secondary. Thus the secondary is translated by x x , yi against 
the primary. 

Since the secondary is otherwise identical with the primary (except 
for the addition of F in the second case), it will again reproduce (and 
produce another F in the second case), constructing a ternary. This 

30 [There is an interesting parallel between Godel's undecidable formula 
(see p. 55 above), which refers to itself, and von Neumann's self -reproducing 
automaton, which contains a description of itself. See Burks, "Computation, 
Behavior, and Structure in Fixed and Growing Automata/' pp. 19-21.] 



GENERAL CONSIDERATIONS 



127 



will then produce a quaternary, followed by a quinary (each with 
its concomitant F in the second case; cf. above), etc. The shifts 
involved will be 2xi , 2yi , then Sxi , Syi , then 4xi , 4?/i , etc. 

Thus the shift between the p-ary and the g-ary is (q — p)x\ , 
(q — p)yi . Since p, q = 1,2, • • • , therefore p 9^ q implies 

| q - p | = 1, 2, • , 

and hence | (q — p)y x \ = \ q — p \ • 1 yi | ^ y. Hence, in view of our 
above observation relating to Section 1.5.2, these two will not 
intersect. That is, all the successively constructed descendants of 
the primary will be distinct and non-interfering entities in space. 
(To be more precise, these descendants will be distinct and non- 
interfering entities in the underlying crystal [crystalline structure].) 

Actually, this program of mutual avoidance among the primary and 
its descendants must be extended to the paths within the crystal 
through which each one of these entities, acting as a primary for 
its own reproduction, connects with the site and operates on the 
construction of its immediate successor in the line of descent, i.e., 
its secondary. 

This, however, presents no difficulties, and will be gone into in 
the course of the detailed discussion. 

1.7.2.1 Constructed automata: initial state and starting stimulus. 
The next point to be gone into is that of initiation and timing. 

Consider the state of the secondary automaton which the con- 
struction is designed to achieve, i.e., its so-called initial state {cf. 
Sec. 1.5.2 immediately after formula (*)}. In all states that lead up 
to this, and therefore conveniently in this state too, the automaton 
must be quasi-quiescent. This is clearly necessary for an orderly 
process of construction, since the already-constructed parts of the 
not yet completed secondary must not be reactive and changing, 
while the construction — in adjacent as well as in other areas — is 
still in progress. 

The problem that is encountered here is not unlike the one dis- 
cussed in Section 1.6.2.3 relative to the quasi-quiescence of L. How- 
ever, it is less severe here. The stimuli that have to be used in ex- 
ploring L must be able to induce responses in the primary, if they 
are to perform their function of inducing there appropriate actions 
that depend on the information acquired in inspecting L. (This is 
quite essential to the proper functioning of the primary, as was seen 
in the discussion of the operation of constructing under the control 
of instructions in Sec. 1.4.2.1, and again in the discussion of the 
operation of copying in Sec. 1.6.2.3 and in Sec. 1.6.3.) On the other 



128 



THEORY OF SELF-REPRODUCING AUTOMATA 



hand, the stimuli that create the desired cell states during the con- 
struction of the secondary need not have such effects on the class 
of automata that is involved here, secondary or primary. This will be 
verified in detail later. Thus it was necessary to keep the "descrip- 
tions" L of automata sharply apart from the "originals" (cf. Sec. 
1.6.3.1), while it will be possible to construct the automata which are 
relevant here so that they have quasi-quiescent initial states. For the 
details and precise definitions, cf. later. 

The crux of this matter is, of course, that once such a (secondary) 
automaton is completed, and hence present in its quasi-quiescent 
initial state, it can then be transferred by some appropriate process 
of stimulation into its normal (i.e., intended) mode of activity. 
This process of stimulation is most conveniently thought of as a 
single stimulus, delivered to the appropriate point of the secondary, 
at the appropriate time after completion, by the primary. This is 
the secondary's starting stimulus. This is, then, the concluding step 
of the primary in its construction of the secondary. For the details, 
cf. later. 

In the case of self-reproduction, i.e., for the E or E F discussed 
in Sections 1.6.1.2 and 1.7.1, the secondary (or one of the secondaries) 
is a shifted copy of the primary. The starting stimulus activates this 
secondary and makes it self -reproducing (as the primary had been 
originally). This, then, maintains the iterative process of self-re- 
production with which Section 1.7.1 dealt. 

1.7.2.2 Single-action vs. sequential self -reproduction. For the self- 
reproducing primary (E or E F ; cf. above) the next question is this: 
What does the primary do after it has completed the secondary 
and given it the starting stimulus? 

The simplest arrangement would be to let it return to a quasi- 
quiescent state which is identical with its original state. Implicitly 
this is the assumption which fits discussions like the one of continued 
reproduction in Section 1.7.1. 

An alternative possibility is to finish with the quasi-quiescent 
state, plus activity in a suitable terminal organ which imparts the 
starting stimulus again. This scheme leads to repeated self-reproduc- 
tions by the original primary, and of course similarly by all its 
descendants in the first, second, third, etc. degree. However, it is 
not operable without one more elaboration. 

Indeed, as it stands it would cause the primary to try to form all 
successive secondaries with the same Xi , iji , i.e., at the same location 
in the crystal. This is obviously conflicting; at best the second con- 
struction of a secondary would override and destroy the first speci- 



GENERAL CONSIDERATIONS 



129 



men. However, it is even more likely that the first secondary, which 
by then is reactive, will interfere with the second (attempted) 
construction, causing an unforeseeable class of malfunctions and 
corrupting all reproduction. It is therefore necessary to change 
Xi , 2/1 between the first and the second attempt to construct a second- 
ary, and similarly between the second and the third one, the third 
and the fourth one, etc., etc. This changing of X\ , yi must therefore 
take place during (i.e., as a part of) the activity of the terminal 
organ referred to above. The arithmetical rules that control these 
successive modifications of x\ , yi must be such that the whole se- 
quence of secondaries of the original primary do not conflict with 
each other, nor with the possible F which accompany them (cf. 
the first part of Sec. 1.7.1), nor with the paths which are required 
for their construction (cf. the end of Sec. 1.7.1). In addition, every 
secondary of the original primary, since it is a shifted copy of the 
latter, will behave in the same way. Thus a double sequence of 
ternaries will be constructed from these, then by the same mechanisms 
a triple sequence of quaternaries, then a quadruple sequence of 
quinaries, etc., etc. The rules for the successive modifications of the 
Xi , y\ must hence be such that no two in all the orders of this hier- 
archy ever interfere with each other, or with each other's possible F, 
or with each other's construction paths. 

This requirement sounds complicated, but it is not particularly 
difficult to implement by suitable arithmetical rules concerning the 
formation of the successive X\ , yi . This will be discussed later. 

The above discussion thus distinguishes between two types of self- 
reproduction : first, when each primary constructs only one second- 
ary, and second, when each primary keeps constructing secondaries 
sequentially without ever stopping. We will designate these as the 
single-action and the sequential type of self -reproduction, respectively. 

1.7.3 Construction, position, conflict. Some remarks about physio- 
logical analogs of the above constructions are now in order. 

Comparing these processes of construction and reproduction of 
automata, and those of actual growth and reproduction in nature, 
this difference is conspicuous; in our case the site plays a more criti- 
cal role than it does in reality. The reason is that by passing from 
continuous, Euclidean space to a discrete crystal, we have purposely 
bypassed as much as possible of kinematics. Hence the moving 
around of a structure which remains congruent to itself, but changes 
its position with respect to the crystal lattice, is no longer the simple 
and elementary operation it is in nature. In our case, it would be 
about as complex as genuine reproduction. This means that all 



130 



THEORY OF SELF-REPRODUCING AUTOMATA 



of our structures are rather rigidly tied to their original location, and 
all conflicts and collisions between them are primarily conflicts in 
location. 

It is true in the natural setting, too, that conflicts and collisions 
are due in the same way to location, but there the scheme has more 
elasticity because of the possibility of motion. The limitations of the 
pattern due to this circumstance are obviously the price one has to 
pay for the simplicity obtained by our elimination of kinematics (cf . 
the discussions of Sees. 1.3.1.1-1.3.3.1). 

An essential preliminary condition for the mechanisms of repro- 
duction that we have considered is the quiescence of the area in 
which they are to function (cf., e.g., the remarks in the first part of 
Sec. 1.7.2.1 and in the first part of Sec. 1.7.2.2). That is, the region 
of the crystal surrounding the primary must be free of all reactive 
organisms, and this must be true as far as the process of reproduction 
is expected to progress unhindered. It is quite clear that where the 
reproductive expansion of the area under the influence of the primary 
collides with other reactive organisms, the "unforeseeable malfunc- 
tions" referred to in Section 1.7.2.2 can set in. This is, of course, 
just another way to refer to the conflict situations involving several 
independent organisms that have come into contact and interaction. 

1.7.4.1 E F and the gene-function. Another physiological analog 
worth pointing out is the similarity of the behavior of automata of 
the E F type with the typical gene function. 31 Indeed, E F reproduces 
itself and also produces a prescribed F. The gene reproduces itself 
and also produces — or stimulates the production of — certain specific 
enzymes. 

1.1 ' .1+.2 E F and the mutation — types of mutation. A further property 
of E F that may be commented on is this. Assume that a cell of E F 
is arbitrarily changed. If this cell lies in the D-region of E F , it may 
inhibit or completely misdirect the process of reproduction. If, on 
the other hand, it lies in the L D+F region of E F , then E F will construct 
a secondary, but this may not be related to it (and to F) in the de- 
sired manner. If, finally, the altered cell lies in the L D+F region, and 
more particularly in the description of F within it, modifying F into, 
say F', then the production of E F > will take place, and in addition 
to it an F' will be produced. 

Such a change of a cell within E F is rather reminiscent of a mutation 
in nature. The first case would seem to have the essential traits of a 
lethal or sterilizing mutation. The second corresponds to one without 

31 Von Neumann, "The General and Logical Theory of Automata." Col- 
lected Works 5.317-318. 



GENERAL CONSIDERATIONS 



131 



these traits, but producing an essentially modified, presumably 
sterile, successor. The third one produces a successor which is viable 
and self-reproducing like the original but has a different by-product 
(F instead of F). This means a change of the hereditary strain. 

Thus the main classification of mutations turns out to be quite 
close to the one occurring in nature. 

] .8 Evolution — Question (E) 

The observations of Section 1.7 tend towards the transition from 
question (D) to question (E). On (E), itself, the question of evolu- 
tion, we will only make a few remarks at this point. 

There is no difficulty in incorporating logical devices into automata 
of the types E or E F which will modify the D, F areas in their L D , 
L D+F , respectively, depending on outside stimuli which they may 
have received previously. This would amount to a modification of the 
mass of heredity that they represent by the occurrences (experiences) 
of their active existence. It is clear that this is a step in the right di- 
rection, but it is also clear that it requires very considerable additional 
analyses and elaborations to become really relevant. We will make a 
few remarks on this subject later. 

In addition to this it must be remembered that conflicts between 
independent organisms lead to consequences which, according to the 
theory of "natural selection," are believed to furnish an important 
mechanism of evolution. As was seen at the end of Section 1.7.3, 
our models lead to such conflict situations. Hence this motive for 
evolution might also be considered within the framework of these 
models. The conditions under which it can be effective here may be 
quite complicated ones, but they deserve study. 



Chapter 2 



A SYSTEM OF 29 STATES WITH A GENERAL 
TRANSITION RULE 

2.1 Introduction 

2.1.1 The model: states and the transition rule. In this chapter we 
will develop the first model that possesses the potentialities of logical 
and constructive universality and of self-reproduction (cf. questions 
(A)-(E) in Sec. 1.1.2.1), as well as the other attributes evolved in 
the course of the discussion of these in Chapter 1. This model is 
based on a crystalline medium (cf. Sees. 1.3.3.1-1.3.3.3); we will be 
able to construct it in two dimensions and to use there the quadratic 1 
(regular) lattice {cf. the end of Sec. 1.3.3.3, in particular questions 
(P) and (R)}. Each lattice point of this crystal will be able to assume 
a finite number of different states (say N states) and its behavior 
will be described (or controlled) by an unambiguous transition rule, 
covering all transitions between these states, as affected by the states 
of the immediate neighbors. 

We will, then, perform the major constructions called for by ques- 
tions (A)-(E) in Section 1.1.2.1 (and the relevant subsequent 
discussions of Ch. 1) for a specific model defined along these lines. 

2.1.2 Formalization of the spatial and the temporal relations. At this 
point we introduce some rigorous concepts and notations. 

The lattice points of the quadratic crystal (cf. Sec. 2.1.1) are 
designated by two integer-valued coordinates, i, j. It is natural to 
treat the crystal as unlimited in all directions, at least as long as there 
does not emerge some definite reason for proceeding differently. This 
determines the ranges of i, j: 

(1) i,j = 0, ±1, ±2, ... . 

[It does not matter which lattice point is selected as the origin 
(0, 0).] The pair i,j thus represents a point in the plane, but it is also 
convenient to view it as a vector, i.e., to treat it as an additive quan- 

1 [The lattice points of a quadratic crystal lie at the corners of squares.] 



132 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 133 

tity. We write 

(2) - 

The nearest neighbors of (i,j) are the four points (i =b l,j)> (hj ± 1). 
The next neighbors are the four points (i db 1 , j ± 1 ) . In Figures 4a 
and 4c the nearest neighbors of X are marked with small circles 
(O), and the next nearest neighbors are marked with heavy dots 

(•). 

Put 

(3) 
and 

(4) 



= (1,0), v' = (0, 1), 

= -v° = (-1,0), v> = -v 1 = (0, -1), 



v e = —v* = ( — 1, —1), v = —v = (1, — 1). 



See Figure 4b. The nearest neighbors of & are the # + v a (a = 0, • • • , 
3), and the next neighbors of # are the # + v a (a = 4, • • • , 7). 
One might hesitate as to whether the immediate neighbors of # re- 
ferred to in Section 2.1.1 should be, among the $ + v a , the four with 
a = 0, • • • , 3 or the eight with a = 0, • • • , 7. We will choose the 
former, since it leads to a simpler set of tools. 

In Figures 4a and 4b the crystal lattice was shown in the usual 
manner, the lattice points being the intersections of the lines. In 
future figures we will use a different scheme: the lattice points will 
be shown as squares, and immediate neighbors (which in Figs. 4a and 
4b were connected by single edges) will now be squares in contact 
(i.e., with a common edge). Furthermore, we will always show only 
those squares (i.e., only those lattice points) which are needed to 
illustrate the point immediately at hand. Thus Figure 4a assumes 
the appearance of Figure 4c. 

As discussed in Section 1.2.1, the range of time t is 

(5) t = 0, d=l, ±2, ... . 

Each lattice point is a cell in the sense of Sections 1.3.3.1 and 
1.4.1.1. It is able to assume N states (cf. Sec. 2.1.1); let these be 
designated by an index 

(6) n = 0, 1, ••• , N - 1. 



The state of cell & = (£, j) at time t will therefore be written 
(7) n,'. 



134 



THEORY OF SELF-REPRODUCING AUTOMATA 



Also, the N numerals 0, 1, • • • , N — 1 used in expression (6) to 
designate n may be replaced by any other N symbols, according to 
convenience. 

The system is to be intrinsically homogeneous in the sense of 
Section 1.3.3.2; i.e., the same rule will govern its behavior at each 
lattice point This rule is the transition rule referred to in Section 
2.1.1, which defines the state of the cell # at time t in terms of its 
own state and of the states of its immediate neighbors at suitable 
previous times. We will limit and simplify our system by restricting 
these "suitable previous times" to precisely the immediate prede- 
cessor of t, i.e., t — 1. Thus n& will be a function of n#~ l and of the 
nf+ v a (a = 0, • • • , 3). That is, 

(8) n# = F(n# M .; 4+ x ,« | a = 0, • • • , 3). 

Let m take the place of n# and let m a take the place of n#+ v a . The 
function F then becomes F(m; m a \ a = 0, 1, 2, 3). This TV-valued 
function F of five Af-valued variables represents, therefore, the transi- 
tion rule. It is the sole and complete rule that governs the behavior 
of this (intrinsically homogeneous) system. 

Note that the range of F has N elements, while the domain of F 
(the set of all quintuplets) has N b elements. Hence there are 

(9) jV ( " 6> 

possible functions F, i.e., this many possible transition rules, or 
models of the class under consideration. 

2.1.3 Need for a pre-formalistic discussion of the states. Let us now 
discuss in a more heuristic way what the N states of a cell should be. 
The nature of these states is, of course, described not by their enumer- 
ation (6), but by the transition rule (8). The only relevant informa- 
tion contained in (6 ) is the number of states, N. In accord with this, 
the rigorous summation of these considerations will consist of a 
specification of the transition rule (8), i.e., of the function F. In the 
present, heuristic stage, however, it will be better to proceed with an 
enumeration (6), attaching to each n of (6) a name and a verbal 
description of the role that it is intended to play. In this connection 
we will also make use of the possibility of notational changes, referred 
to in the remark after (6) and (7) in Section 2.1.2. 

2.2 Logical Functions — Ordinary Transmission States 

2.2.1 Logical-neuronal functions. To begin with, states are needed 
to express the properly logical or neuronal functions, as discussed in 
Section 1.2.1. This calls for the equivalents of the neurons of Figure 
3 and of their connecting lines. 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 135 

2.2.2.1 Transmission states — connecting lines. We consider first 
the connecting lines. These must now be rows of cells, i.e., of lattice 
points. Since a line must be able to pass a (neural) stimulus, each one 
of its cells must possess, for this purpose alone, a quiescent and an 
excited state. The purpose that we consider here is to transmit a 
(neural) stimulus. We call these therefore the transmission states 
of the cell and designate them by the symbol T. We use an index 
e = 0, 1; i.e., we write T t to indicate quiescence and excitation. Let 
e = 0 designate the former and e = 1 the latter. 

This transmission must be a directed process, since the lines (that 
the cells in transmission states replace) were directed to connect 
definite points. Hence we must set up certain limitations. We may 
stipulate that a cell in a transmission state accepts a stimulus only 
from one, definite direction, its input direction. That is, an excited 
transmission cell brings an immediate neighbor (which is a quiescent 
transmission cell) into the excited transmission state (or, if the latter 
is found in that state, it keeps it there) only if the former lies in the 
latter's input direction. Alternatively, we may also stipulate that a 
cell in a transmission state emits a stimulus only in one, definite 
direction, its output direction. That is, an excited transmission cell 
brings an immediate neighbor (which is a quiescent transmission 
cell) into the excited transmission state (or, if the latter is found in 
that state, it keeps it there) only if the latter lies in the former's 
output direction. Finally, we may make both stipulations together. 

After trying various models along these lines, it appeared most 
convenient to stipulate a definite output direction. In order to avoid 
certain uncontrolled, and hence undesirable, return-stimulation 
phenomena, it seems desirable, while not prescribing any particular 
input direction, to specify that the output direction is insensitive to 
inputs. 

The v a (a = 0, • • • , 3) of Figure 4b enumerate all possible direc- 
tions for an immediate neighbor (cf. the remarks after expressions 
(3) and (4) in Sec. 2.1.2). Hence the T e will be given a further index 
a = 0, • • • , 3: T ac , so that T a e has the output direction v a . The 
above stipulations now assume this form: T a >i at induces T«i at 
* (from T a0 or T«i) if and only if & = &' + v a \ but + v a , 

i.e., if and only if # — &' = v a ^ — v a . 

Let us now use the symbols T ae (a = 0, • • • , 3; e = 0, 1) in place 
of some eight number values in expression (6) (cf. the remark after 
expression (6) and (7) in Sec. 2.1.2). Let us also recognize the unit 
time delay of the stimulus-response process, as discussed in Section 
2.1.2. Then the above rule becomes: 



136 



THEORY OF SELF-HEPUODIHM N( J AUTOMATA 



I Then n* = T al if n#7* = TV, 
for some with § — §' = v a ^ —v a . 
Otherwise n& = T a0 . 



Assume n& 1 = T«, 



(10) 



2.2.2.2 Delays, corners, and turns in connecting lines. Note that 
this model for the connecting lines differs from the one considered in 
Section 1.2.1 in that it introduces finite propagation delays. We have 
now a unit time delay between immediate neighbors. However, this 
deviation from the pattern of Section 1.2.1 will have no relevant 
undesirable consequences. 

Note also that this model serves equally to synthesize straight 
connecting lines from transmission cells, and connecting lines with 
corners or turns in them. Straight lines are shown in Figures 5a-5d; 
these represent the four possible "straight" directions in our lattice. 
"Corners" and "turns" are shown in Figures 5e and 5f. Figures 
5a-5f are drawn according to the rules stated in Figure 4c. Figures 
5a'-5f' are simplified (and more easily readable) versions of Figures 
5a-5f, respectively, in which each T a€ is replaced by the arrow of its 
v a (cf. Fig. 4b). 

We consider next the specific neurons of Figure 3. 



2.3.1 The + neuron. The + neuron merely calls for an excitable 
cell which has an output and two possible inputs. There is, of course, 
no harm done if it can accommodate more than two such inputs. We 
defined a transmission cell in Section 2.2.2.1 so that it has three 
possible inputs. Every one of its four sides, excepting the output side, 
is an input. Thus our transmission cells not only fulfill the function 
of (elements of) connecting lines between neurons (this being the 
function for which they were originally intended), but also fill the 
role of + neurons. 

The use of an ordinary transmission cell as a + neuron, i.e., as a 
connecting line junction, is shown in Figure 5g. This figure is drawn 
according to the scheme of Figures 5a'-5f'. 

2.3.2 Confluent states: the • neuron. The • neuron calls for an excita- 
ble cell that has an output and two inputs, which must be stimulated 
together in order to produce excitation. It would be quite practical 
to introduce a class of such states. However, a free choice of an output 
direction and two input directions (from the totality of four possible 
directions, as represented by the v a , a = 0, • • • , 3), would require 
(4 X 3 X 2)/2 = 12 kinds, and, since there must be a quiescent and 
an excited state for each kind, a total of 24 states. It is possible to 



2.3 Neurons — Confluent States 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 137 

achieve equally satisfactory results with more economy, namely 
with only one kind, and hence with two states. This can be done by 
prescribing no particular input or output directions at all, i.e., by 
stipulating that every direction is a possible input, as well as a possible 
output. In addition to this, one can then prescribe as the prerequisite 
of excitation a minimum of two stimulations, i.e., a minimum of two 
excited transmission cells that are immediate neighbors and in whose 
output direction our cell lies. However, it is still more convenient to 
frame this condition more elastically and to stipulate that the cell 
under consideration gets excited if every immediately neighboring 
transmission cell, whose output direction points at this cell, is itself 
excited. (This is to be taken with the exclusion of the subcase — which 
is strictly logically admissible, but obviously conflicting with the 
intention — that none of the immediate neighbors qualifies, i.e., is a 
transmission cell and has its output direction pointing at this cell.) 
This formulation of the rule has the effect that the cell under con- 
sideration can act as a neuron of threshold one (i.e., from the point 
of view of inputs, like an ordinary transmission cell), or two (i.e., 
like the desired • neuron), or three (i.e., like a combination of two • 
neurons), depending on whether one, two, or three, respectively, of its 
immediate neighbors are transmission cells with their output direc- 
tions pointing at it. (Since this cell should not be able to stimulate 
any transmission cell whose output direction is pointing at it — cf. 
rule (10) and its adaptation to the present situation in rule (12) 
below — it would be pointless to have all of its four immediate neigh- 
bors in such a state. This situation would preclude any results of an 
excitation of our cell.) 

We will call these states of a cell confluent states, and designate 
them by the symbol C. We again use the index e = 0, 1; that is, we 
write C € to indicate quiescence (e = 0) and excitation (e = 1). We 
proceed now similarly as we did at the end of Section 2.2.2.1. We 
use the symbols C e (e = 0, 1) in place of two number values in ex- 
pression (6) (cf. the remark after expressions (6) and (7) in Sec. 
2.1.2). The rule that we formulated above now becomes, inasmuch 
as it affects the inputs to C: 

Assume n&~ 1 = C € . 



(11) 



Then n* = d , if both (a), (b) hold: 

(a) nlv 1 = T«'i for some #' with # - / = v a ' . 

(b) Never n#~ l = T^ 0 for an with & - #' = v a ' . 
Otherwise n/ = C 0 . 



The portion of the rule that affects the outputs of C must be stated 



138 



THEORY OF SELF-REPRODUCING AUTOMATA 



as a modification of rule (10), since it provides for a new way to 
excite a transmission cell, i.e., to produce a T«i from a T ac . This is 
expressed by the following insertion between the second and third 
sentences of rule (10): 

s 19 v (Also n# = T«i if n^T 1 = Ci for some 

{ Z) \with 0 - y = / ^ -if 08 = 0, • • • , 3). 

Note that the system of rules (10), (11), and (12) provides for 
excitation of T by T, also of C by T, and T by C, but not for an excita- 
tion of C by C. This arrangement will have no relevant undesirable 
consequences. 

A • neuron with its close surroundings is shown in Figure 6a. This 
figure is drawn according to the scheme of Figures 5a'-5f' and 5g. 
The confluent state C here makes its first appearance. 

2.3.S The — neuron. The — neuron calls for an excitable cell in 
which the roles of quiescence and excitation are interchanged in 
comparison with the transmission states. It must be ordinarily excited 
(i.e., able to excite an immediately neighboring cell in a transmission 
state, at which its output direction points), but it must be made 
quiescent by an input stimulation (reverting to excitation when the 
stimulation ceases). We could introduce a class of such states — e.g., 
with a given output direction, all other directions being input direc- 
tions, just as in the transmission states. Since there are four possible 
directions, this would require four kinds, and with quiescence and 
excitation for each kind, eight states would be needed. However, we 
are reluctant to introduce a class of such states whose ordinary, 
unperturbed condition is not quiescence. This objection could be 
circumvented in various ways, with higher or lower degrees of econ- 
omy. 2 We shall find that a class of states which we will introduce 
later for other reasons can be used to synthesize the function of 
the — neuron. We can therefore forego altogether taking care of it 
at this stage. 

2.34 The split. Although all the neuron species of Figure 3, as 

2 [Von Neumann here referred to the double line trick of his "Probabilistic 
Logics and the Synthesis of Reliable Organisms from Unreliable Components," 
Collected Works 5.337. Using only -f neurons and • neurons, he synthesized a 
complete set of truth-functional primitives by using a pair of lines with the 
codings 01 (for "zero") and 10 (for "one"). In other words, each line of the 
pair is in the opposite state from the other line of the pair, so that negation may 
be realized by interchanging (crossing) the two lines of a pair. But in the pres- 
ent manuscript von Neumann synthesized negation from the destructive (re- 
verse) and constructive (direct) processes of Sees. 2.5 and 2.6 below. An exam- 
ple of this synthesis is given in Fig. 17 of Sec. 3.2 below.] 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 



139 



well as their connecting lines, have been disposed of, there remains 
one more entity in this category to consider. Logical (i.e., neuronal) 
networks in the sense of Section 1.2.1 must in most cases contain 
connecting lines that lead from one output to several inputs, i.e., 
output lines that have to be split. (This was mentioned in Sec. 1.2.1.) 3 
That is, transmission-like states with several outputs are needed. 

However, it suffices to note that our definition of the confluent 
states takes care of this need. (Cf. Sec. 2.3.2, in particular the discus- 
sion of the "threshold 1" geometry for this class of states, near the 
end of Sec. 2.3.2. In this condition the cell has one input, and there- 
fore up to three possible outputs.) 

A split that is achieved by using the confluent states is shown in 
Figure 6b. This figure is drawn according to the scheme of Figure 6a. 
This is also true of all subsequent figures, with such exceptions and 
modifications as are expressly stated. 

2.4 Growth Functions: Unexcitable State and Special 
Transmission States 

2.4-1 Muscular or growth functions — ordinary vs. special stimuli. 
Having finished with the logical (i.e., neuronal) functions in the sense 
of Section 1.2.1, we can now pass to the others. In Section 1.2.2 these 
were temporarily designated as muscular, but the discussion of Section 
1.3 showed that they are more properly viewed and treated as growth 
functions (cf. in particular Sees. 1.3.4.2 and 1.3.4.3). At any rate, 
we need states to express these functions. 

We know (cf. the above references) that this leads over into the 
problem of ordinary vs. special stimuli, i.e., of ordinary vs. special 
excited states. The ordinary class is the one used for logical purposes, 
i.e., the one considered up to now (specifically in Sees. 2.1.3-2.3.2). 
The special class is the one that we will have to introduce now, in 
order to take care of the functions referred to above. 

24.2 Unexcitable state. The purpose of the special class of excited 
states is to induce growth in the sense of Sections 1.3.4.2 and 1.3.4.3, 
i.e., to transfer cells from unexcitable to excitable states, and within 
the latter category also to determine the state's species. As for the 
last mentioned determination, we have already stipulated the exist- 
ence of several species: the transmission, or T states, which formed 
four species, the T a , a = 0, 1, • • • , 3; and the confluent species C 
(cf. Sees. 2.2.2.1 and 2.3.2). Note that each of these actually corre- 

3 [In his list of footnotes von Neumann wrote here "Degeneration (?)." I 
do not know what he intended. Possibly he was going to say that power ampli- 
fication is needed when one input line drives two or more output lines.] 



140 



THEORY OF SELF-REPRODUCING AUTOMATA 



sponds to two states, with e = 0, 1 (quiescent and excited) : T ac , C £ . 
We need nevertheless refer to the species T a , C only, or, to be more 
precise, to the states with e = 0 only: T a0 , C 0 . The reason is that it 
suffices to be able to create each excitable species in its quiescent 
state. If the excited state is wanted, it can be induced later by ordi- 
nary stimuli (for the latter, cf. Sees. 1.3.4.2 and 2.4.1). 

It is therefore necessary to introduce the unexcitable state (un- 
excitable by ordinary stimuli, cf. the above observations and refer- 
ences). We will designate it by U. Before we give a rigorous account 
of its properties, however, we must discuss some connected matters. 

2.4.3 Direct and reverse processes — special transmission states. It 
is desirable not only to be able to effect transfers from the unexcitable 
state U into excitable states (for example, T and C; cf. the discussion 
in Sec. 2.4.2), but also to have this process reversible, that is, to be 
able to effect transfers from the excitable states into U. Various uses 
of this two-way operability will appear later. Now the stimuli which 
induce all these metamorphoses must be transmitted by cell states 
which are not themselves affected by them in such a way. It is there- 
fore advisable to introduce a new class of transmission states, say 
T', which are in their relationship to each other analogs of the T. 
We will therefore have eight such states: T a£ (a = 0, • • • , 3; e = 
0, 1) — in analogy to the T a€ of Section 2.2.2.1. Accordingly, we pro- 
ceed the way we did for the T a€ , i.e., in rule (10) of Section 2.2.2.1. 

We use the symbols T ac (a = 0, • • • , 3; e = 0, 1 ) in place of some 
eight-number values in expression (6) (cf. the remark after expres- 
sions (6) and (7) in Sec. 2.1.2). The rule is: 

Assume n/" 1 = Tl e . 
(13) J Then n& = T a i if n<r l = T a >i for some 

with # — y = v a —v a . 
Otherwise n# = T a0 . 

2.5 The Reverse Process 

2.5.1.1 The reverse process for the ordinary states. We can now give 
a rigorous definition of the reverse process, that is, the transfer of an 
excitable state (T, C) into an unexcitable one (U) by the special 
stimuli (of T'). This takes the form of modifications of the T rules 
(10), (12) and of the C rule (11): 



(14) 



Assume n& 1 = T a€ or C 6 . 

Then the following rule overrides the rules of (10), (12) and 

of (11): f f 

[n# = U if n^/ 1 = T a a for some with # — &' = v a . 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 



141 



Note that # — ^ — v a {in the T case, cf. the similar stipulations 
in rules (10) and (12)} is not required; that is, the "kill" by a special 
stimulus (from a T') is efficacious even in the output direction (of T). 

2.5,1.2 The reverse process for the special states. The reasons which 
make the existence of the reverse process (from excitable to unexcita- 
ble; cf. Sec. 2.4.3 and later) desirable for the ordinary excitable 
states (T, C), operate in the case of the special ones (T') too. How- 
ever, rule (14) could not be extended to the T' states; such an effect 
of a T' on a T' is inadmissible for the same reason for which the 
corresponding effect of a T on a T is inadmissible (cf. Sec. 2.4.3) : it 
would destroy the character of the T as transmission states for 
special stimuli, just as the corresponding effect would have destroyed 
the character of the T as transmission states for ordinary stimuli. 
The latter circumstance caused us to introduce the T' (to transfer 
the T, as well as the C, into U) ; we can now make a similar use of the 
T (to transfer the T' into U). It is advisable, however, not to endow 
the C, too, with this property. Indeed, since every direction is an 
output direction for C, giving this faculty to C would make it much 
more difficult to direct and to control than is desirable. We introduce 
therefore this rule, which modifies (13): 

f Assume n#~ l = t1<= . 
(15) \ Then the following rule overrides the rules of (13) : 

[n/ = U if n^T 1 = T a 'i for some with # — # = v a . 

The remark at the end of Section 2.5.1.1, concerning the outputs, 
applies here, too. 

2.5.2 Origination of special stimuli. We have not provided so far 
for the origination of special pulses, i.e., for the excitation of the T , 
except by each other. 

It is not necessary to introduce a complete (logical) neuronic 
system for special stimuli, as we did for ordinary stimuli in Sections 
2.2.2.1-2.3.4. We can handle all of logics with ordinary stimuli, as 
was indeed intended in introducing them (cf. Sees. 1.3.4.2 and 
1.3.4.3), and use these to start chains of special stimuli, whenever 
necessary. (The physiological analog of this is that logics are re- 
stricted to neuronal activity, and that muscular activity is always 
induced and controlled by neuronal activity. See Sec. 1.2.2 for 
the simile.) Hence we need a class of states which can respond to an 
ordinary stimulus by emitting a special one, i.e., which can be excited 
by a T and can excite a T'. 

Before introducing a new class for this purpose, let us see whether 
we might not do it with the existing ones. T does not excite T' {it 



142 



THEORY OF SELF-REPRODUCING AUTOMATA 



"kills" it; cf. rule (15)} ; hence we can use neither T nor T' for this 
purpose. This leaves only C. Now C is excited by a T; hence we need 
only endow it with the ability to excite a T'. This can be done. (That 
T' does not excite a C — but "kills" it; cf. Sec. 2.5.1.1 — is irrelevant 
in this connection.) All we need is to stipulate the analog of rule (12) 
for T'. Thus the duality of T and T , already expressed by the duality 
of rules (10) and (13), and by the duality of rules (14) and (15), is 
further completed by the duality of (12) and of (16) that we will now 
state. This is to be viewed as an insertion between the second and 
third sentences of (13), and it is therefore, like the original (13), 
overridden by (15). The rule follows: 

( . fAlso n# = T„i , if ni' 1 = Q , for 

U ; \some y with 0 - tf' = / ^ -v* (0 = 0, • • • , 3). 

2.6 The Direct Process — Sensitized States 

2.6.1 The direct process. The reverse process of Section 2.4.3 (trans- 
fer from excitable to unexcitable) having been taken care of, we pass 
now to considering the (primarily required) direct process of Section 
2.4.2 (transfer from unexcitable to excitable). 

The list of species that this process must be able to create has been 
extended. We had the T a and C in Section 2.4.2; to these we must 
add, since Section 2.4.3, the T a . In other words, we must be able 
to create the following states (for the role of e = 0 compare the 
discussion in Sec. 2.4.2) : 

(17) 

This is a total of nine states. 

Thus we need a mechanism to transfer U into any one of the nine 
states of (17). Regarding this mechanism two remarks are in order. 

2.6.2.1 First remark: duality of ordinary and special states. We have 
two kinds of stimuli at our disposal : ordinary and special, correspond- 
ing to the excitation of the T (possibly together with the C) and of 
the T' states, respectively. Our original intention had been to use 
only the special stimuli, i.e., the T' states, for transforming U into 
any one of the nine states of (17) (cf. Sees. 2.4.1 and 2.4.2). Subse- 
quently however, in dealing with the reverse process (cf. Sees. 2.4.3- 
2.5.2), we let the T (actually without the C) and the T states play 
rather complementary and symmetric roles with respect to each 
other (cf. the references to "duality" at the end of Sec. 2.5.2). It is 
therefore tempting to assign to them similarly symmetric roles in 
connection with the direct process. 

This might again be done in a dual way, i.e., by using the T for 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 143 

transfers from U to T (and to C) and the T for transfers from U to 
T'. However, even this limitation will prove to be unnecessary, and 
a satisfactory system can be built by stipulating that T and T' have 
identical and interchangable effects on U for transfers into all states 
T, C, and T' {i.e., all nine states of (17)}. 

This raises the question of why T' had to be introduced at all, if T 
alone can induce and control all transfers from U. The answer is that 
the reverse process called for T' because of the contrast between 
transmission and causing transfer into U; compare rules (10) and 
(12) with rule (14), and rules (13) and (16) with rule (15). More- 
over, the reverse process (transfer from T, T', or C into U) is a 
prerequisite for a proper control of the direct process (transfer from 
U into T, T', or C — or even into T or C only). This point deserves 
somewhat closer consideration. 

2.6,2.2 The need for the reverse process. In Figure 7, cells 1, • • • , 9 
constitute an area of 3 X 3 cells (i.e., lattice points; cf. the explana- 
tion of Fig. 4c) which is to be organized, that is, transferred from the 
U states into various (prescribed) states, e.g., T states. This organiza- 
tion of cells 1, • • • , 9 is to originate from and be controlled by area 0. 

Consider the transfer of the middle cell, number 5. Whether this is 
done by ordinary or by special pulses (i.e., by T or by T' excitations), 
an unbroken chain of (T or T') transmission cells must be laid down, 
from the area of the origination and logical control of these excita- 
tions to the cell to be operated on, in this case number 5. In Figure 7 
the cells marked with arrows form the chain; they may be T a accord- 
ing to Figure 5 or their corresponding T a . This chain must cross the 
ring of cells that surround the target cell number 5, i.e., the ring of 
cells numbered 1, • • • , 4, 6, • • • , 9. In Figure 7 the chain crosses at 
cell number 8. 

Now the desired organization of the area 1, • • • , 9 may provide 
for the cell of the cross-over (in this case number 8; cf. above) another 
(quiescent ) state than that of the chain. (The latter must be a T a 
or a T a state, with v a in the direction of the chain; this is a = 1 in 
the present case; cf. Figs. 4b and 7.) Hence the organization of this 
cell cannot have occurred before that of cell number 5. If it is to occur 
after that of cell number 5, then the (T or T') cell of the chain that 
is involved must be transferred into the desired state. Since the direct 
process allows the transfer from U into any desired state, it is simplest 
to provide for a way to transfer from the present (T or T') state into 
U. Hence the reverse process is indeed necessary. 

2.6.3.1 Second remark: the need for fixed stimulus sequences to con- 
trol the direct process. We observed near the end of Section 2.6.2.1 



144 



THEORY OF SELF-REPRODUCING AUTOMATA 



that the direct process {i.e., the transfer from U into the states of list 
(17)} should be effected by the stimuli due to T and T' excitations, 
and that it will prove possible to let both (i.e., T and T') have 
exactly the same effects in this respect. It will also appear later that 
this arrangement is more economical in some relevant ways than its 
obvious alternatives. 

It is again advisable to exclude the C from this process. The reasons 
for using the direct process here are the same as those given (in Sec. 
2.5.1.2) for using the reverse process. Note in addition that, just 
because every direction is an output direction for C, it is often neces- 
sary to protect certain sides of a C, and for this function a U is 
natural (cf. later). Hence it would be most inconvenient if a C had 
any effect on a U. 

The direct process must provide for transfers from U into every one 
of the nine states of list (17). These nine alternatives are too many to 
be handled by a single stimulus, even if T and T' excitations had 
different effects. Besides, we stipulated that T and T' have the same 
effects in this process. Hence the nine alternatives which are now 
needed must be expressed by some binary coded stimulus sequence. 
In this binary code a digit 1 is expressed by a stimulus (T or T'), 
while a digit 0 is expressed by the absence of a stimulus. Coded 
sequences of length three can express eight alternatives: 000, 001, • • • , 
111. Since nine alternatives are wanted, one of these — e.g., the first 
one, 000 — must be made to express two further alternatives; i.e., it 
must be continued to 0000 and 0001. Thus we end up with the nine 
coded sequences 



This must be made to correspond to the nine states of list (17). 

The nine coded sequences of list (18) will have to be built up 
successively from their constituent digits 0 and 1. That is, in the 
(direct) process of transfer, U will have to go through intermediate 
states corresponding to the subsequences encountered in building 
up the nine coded sequences of list (18). These coded subsequences 
are 



i.e., their number is seven. Finally, there must be a state correspond- 
ing to the beginning of the process, where the formation of the coded 
sequence {going through subsequences of list (19) to a sequence of 
list (18)} has not yet begun. Here the coded subsequence is best 



(18) 



0000, 0001, 001, 010, 011 
100, 101, 110, 111. 



(19) 




SYSTEM OF 29 STATES W ITH A TRANSITION RULE 



interpreted as being (in its initial condition) the empty set, to he 
designated 

(20) e. 

2.6.3.2 Additional states required. Let us use a common symbol 2 
for the 17 coded sequences of lists (20), (19), and (18). These must 
correspond to 17 states to be designated Sz . However, the with 
the 9 2 of list (18) must be the 9 states of list (17); i.e., these are 
not new states. In addition, we must consider whether it is not natural 
and proper to make Ss with the 2 of (20) (i.e., Se) coincide with U. 

The direct process goes from Se through the S 2 of list (19) to the 
Ss of list (18) {i.e., to the states of list (17)}, by adding a digit 1 to 
2 when a (T or T') stimulus occurs, and by adding a digit 0 to 2 
when no stimulus occurs. { Of course, this process of increasing 2 ceases 
to operate when 2 has reached its maximum size (18). Then we deal 
with the states of list (17), and these are governed by the rules (10)- 
(16).} This means that the evolution from Se to the final Ss {with-2 
maximal, i.e., according to list (18)} is rigidly timed. The absence of 
the stimulus has as definite an effect as the presence of one, and there- 
fore the stimuli that are required must be delivered at definite times, 
without any delays other than the expressly required ones. On the 
other hand, U was always conceived of as a quiescent state; it should 
not change unless it receives a stimulus. Combining these two facts, 
we see therefore that U and Se must not be identified. Thus we come 
out with eight new states, namely the Ss with the 2 of lists (20) and 
(19). 

2.6.4 Sensitized states. We call the new states, to which Sections 
2.6.3.1 and 2.6.3.2 lead us, sensitized states. To restate, these are the 
Ss , with 2 according to lists (20) and (19), but not according to list 
(18) {the latter being the old states of list (17)}. 

The rigorous rules that control the behavior of U and of the sensi- 
tized states can now be extracted from the discussions of Sections 
2.6.3.1 and 2.6.3.2. These rules are: 

Assume n/ _1 = U. 
Thenn^ = S e if n#~ l = TViorTU 
for some with # — &' = v a . 
^ Otherwise n# = U. 

f Assume n/" 1 = S s with a 2 of (20) or (19). 
j Then n/ = S 21 if n^T 1 = TVi or 
] for some with # — &' — v a ' . 
Otherwise n& = Sso. 

[Note that Sao = So , S01O = S10 , Sam = Sm , etc. ] 



(21) 



14G 



THEORY OF SELF-REPRODUCING AUTOMATA 



2.7 Even and Odd Delays 

2.7.1 Even delays by path differences. We have completed the de- 
scription of the direct process of Section 2.4.2 (cf. Sec. 2.6), i.e., of 
the process that operates the transfers from U into the states of list 

(17) . We noted in Section 2.6.3.2 that the direct process is rigidly 
timed, that is, that each state of (17) calls for certain stimuli delivered 
at definite distances in time from each other. To be more specific, 
to get from S$ to the states of list (17), i.e., to the S 2 with 2 from 
list (18), the uninterrupted stimulus-no-stimulus sequences of list 

(18) are needed, according to rule (22). These are certain sequences 
of lengths three and four. Since we actually want to start from U, 
we must also apply rule (21); that is, we must have one stimulus 
immediately before this sequence. Hence the complete requirement 
prescribes certain uninterrupted stimulus-no-stimulus sequences of 
lengths four and five. 

In a properly organized control system such a sequence will be 
induced by a suitable (single) control stimulus. To get a prescribed 
train of stimuli during four or five consecutive moments t {t is integer 
valued; cf. expression (5)} from a (single) control stimulus, some 
multiple delay system is needed. See Figure 8. If the (single) control 
stimulus appears at a lattice point A, and if the prescribed stimulus- 
no-stimulus sequence is wanted at a lattice point B, then it is neces- 
sary to route the excitation from A to B over several paths, which 
produce, relatively to each other, definite delay differences. Indeed 
let (P be the path from A to B, over which the excitation arrives first 
to B. Then the other paths from A to B, say (Pi , (P 2 , • • • , must 
produce delays against (P that are equal to the distances of the stimuli 
prescribed in the desired stimulus-no-stimulus sequence from its 
first stimulus (the one required by rule (21); cf. above). Figure 8 
exemplifies such a situation. [The delay on the output side of cell A 
to an input side of cell B is 7 along path (P, 17 along path (Pi , and 37 
along path (P 2 . If these paths are stimulated at time zero, stimuli will 
enter cell B at times 7, 17, and 37.] 

2.7.2.1 Odd delays and single delays. It is clear, however, that only 
even delays can be produced in this manner: the lengths of any two 
paths connecting two given points A and B differ by an even number. 
Yet the sequences of list (18) {as required by list (22)} preceded by a 
(stimulus) digit 1 {as required by list (21)} may contain digits 1 
(i.e., stimuli) at odd distances. This conflict must be resolved. The 
resolution can be effected in various ways. 

First, the principle of even difference of lengths between paths 
connecting the same two points depends for its validity on the crystal 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 147 

lattice used. It holds for the quadratic lattice that we use here, but 
it fails for some other lattices. We might therefore consider changing 
the crystal lattice. 

Second, we could lengthen the sequences of list (18) by inserting 
(no stimulus) digits 0, so as to make all the distances between the 
(stimulus) digits 1 {including the prefatory 1 required by rule (21)} 
even. This would increase the number of the subsequences encoun- 
tered in building up the sequences of list (18), i.e., the number of 
the subsequences according to list (19). Thus there would have to be 
more sensitized states. 

Third, we can introduce the odd delays directly. It is clearly suffi- 
cient to introduce a single delay. This means that we need an excited 
state, which occurs at time t, not after a stimulation at time t — 1 
(as in all cases so far considered, i.e., for T, T', C), but after a stimula- 
tion at time t — 2. 

2.7.2,2 Single delays through the confluent states. A closer inspection 
indicates that, of the three alternatives described in Section 2.7.2.1, 
the last one is most convenient, and particularly most economical 
with respect to the number of new states required. In this connection 
two additional remarks are in order. 

First, it suffices to supply a single delay mechanism for ordinary 
(T) stimuli. Indeed, special (T') stimuli can be obtained from the 
ordinary ones by a fixed delay conversion process {namely (16)}. 
That is, if a rigidly timed sequence (in the sense of Sec. 2.7.1) of 
special (T') stimuli is wanted, it is practical to produce it first (with 
a delay system according to Sec. 2.7.2.1) with ordinary (T) stimuli, 
and then convert from ordinary to special {with the help of rule (16)} 

Second, in order to introduce a single delay for ordinary stimuli, 
it is not necessary to introduce a new kind of state; it suffices to 
utilize and expand an existing kind. Indeed, it is quite practical to 
attribute this trait to C, because the other necessary functions of C 
need not be impaired by such a change. 

That this is so will become clear in the course of our actual uses of 
C. For the present we will limit ourselves to formulating those changes 
in the rules that govern the behavior of C which are needed to 
introduce the desired (single) delay. 

At present the excitation of C is described by rule (11). The 
stimulating effect of C is described by rules (12) and (16); the 
"killing" of C (i.e., its transfer into U) is described by rule (14). 

Under these rules C has the states C £ , where the index e = 0, 1 
describes the present state of excitation. If the excitation is to be 
delayed by a single time unit, then C must remember for that length 



148 



THEORY OF SELF-REPRODUCING AUTOMATA 



of time what its state of excitation will be next. Hence two indices, 
e.g., €, e', will be needed. The states will be C €£ > , where the index 
€ = 0, 1 describes the present state of excitation, and the index 
e = 0, 1 describes the next state of excitation. The effects on rule 
(11), and on rules (12), (14), and (16) are therefore the following. 

In rule (11): where C £ (for t — 1) was transferred into C e >> (for 
t), now C eC ' , (for t — 1) should be transferred into C e ' e >> (for t). 

In rules (12), (14), and (16): the role of C t (for / - 1) is taken 
over by C tc > , (for I — 1 ) . 

Furthermore, it is natural that in the list (17) the (quiescent) 
slate Co should be replaced by the (completely quiescent) state Coo . 

The rigorous statements of the necessary modifications are accord- 
ingly these: 

In rule (11) replace n#~ l = C 6 by n#~ l = C €C ' , 
replace n# = Ci by n# = C e 'i , and 

(2°) < re P lace n » = Cobyn/ = C c > 0 . 

I In rules (12) and (16) replace n^ 1 = Ci by n#~ l = Cw ; 
in rule (14) replace n&~ 1 = C c by n#~ l = C CC ' . 
In list (17) replace C 0 by Coo . 

To conclude, we observe that with this method of introducing a 
(single) delay we replace the two states C 6 by the four states C ce ' ; 
that is, we introduced two new states. 

2.8 Summary 

2.8.1 Rigorous description of the states and of the transition rule. 
We can now give a rigorous summary, i.e., a complete list of states 
and an exhaustive transition rule. 

Let us write T u , u = 0, 1, in place of T, T', respectively. Corres- 
pondingly, let us call the ordinary stimuli and the special stimuli 
stimuli [0] and stimuli [1], respectively. 

The enumeration of states becomes this : 
(S) The states: 

The states are the following ones : 

The transmission states T uai , where u = 0, 1 correspond to ordinary 
and special; a = 0, 1, 2, 3 to right, up, left, down; e = 0, 1 to quiescent 
and excited. 

The confluent states C ee ' , where e = 0, 1 correspond to quiescent 
and excited; e = 0, 1 to next quiescent and next excited. 
The unexcitable state U. 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 



149 



The sensitized states Ss — where 2 has the range 

(5.1) 2 = 0,0, 1,00,01, 10, 11,000. 
In addition, the Ss with 

(5.2) 2 = 0000, 0001, 001, 010, Oil, 100, 101, 110, 111, 
in this order, are identified with 

(5.3) T ua0 (u = 0, 1; a = 0, 1, 2, 3) and C 00 

in the order Tooo , Toio , T020 , T030 , T100 , Tno , T120 , T130 , Coo . This is 
a total of 16 (transmission) + 4 (confluent) + 1 (unexcitable) + 
8 (sensitized) = 29 states. Hence 

(24) N = 29, 

and the symbols T wa£ (u = 0, l;a = 0, 1,2,3; e = 0,1), C e€ , (e = 0, 1; 
e = 0, 1), U, Ss {2 according to (S.l)} will be used in place of the 29 
number values in expression (6) (cf. the remark after expressions (6) 
and (7) in Sec. 2.1.2). 

Let us now consider the transition rule. First, note that the num- 
ber of possibilities for this rule (i.e., for the function F in the sense 
of Sec. 2.1.2) is, according to expression (9) with N = 29, 

(25) 29 (295) & io 30 ' 000 ' 000 

(with three significant figures in the exponent). Second, the rules 
(10)- (16) and (21)- (23) constitute together the transition rule, 
and they can be summarized as follows. 
(T) The transition rule: 

Assume n#~ l = T uae . 

(a) n$ = U if and only if n^T 1 = T u > a 'i , 
for some with $ — §' = v a \ and also 11 ^ 
u . 

((3) n/ = Tuai if and only if (a) does not hold 
™ jv ^ and either (a) or (b) holds: 

J (a) n^T 1 = T Ma 'i for some with 

# - y = v a ' ^ - v a . 

(b) n^T 1 = C u > for some with 

# - y = / ^ — tF*o = 0, ••• ,3). 

(7) n$ = T ua o if and only if neither (a) nor (/3) 
holds. 



150 



THEORY OF SELF-REPRODUCING AUTOMATA 



Assume n\ 1 = C«' 



t—i 



Ti a 'i , 



(T.3) 



(T.4) 



(a) n& = U if and only if 

for some &' with # — = v a ' . 
(P) n» = C C 'i if and only if (a) does not hold 
and both (a) and (b) hold: 
(T.2) \ (a) n l i> 1 = T 0 «'i for some 0 with 

# - y = v a \ 

(b) Never n^T 1 = T 0a 'o for an &' with 

# - y = 

(7) = C € 'o if and only if neither (a) nor (0) 
holds. 

Assume n# _1 = U. 

(a) n& = Se if and only if n^T 1 = T u «'i , 

for some & with # — $ = 
(iS) n^' = U if and only if (a) does not hold. 

Assume ni' 1 = Ss {with 2 according to list (S.l)}. 
(a) n# = Ssi if and only if n^T 1 = T tttt 'i , 

for some with & — ft' = v a ' . 
(/3) n# = Sso if and only if (a) does not hold. 

2.8.2 Verbal summary. The rigorous summary of Section 2.8.1 is 
a strict restatement of the verbal formulations and conclusions 
arrived at in Sections 2.2-2.7. In this sense, a verbal statement of 
the strict and formalistic contents of Section 2.8.1 is available in 
those sections. However, it seems desirable to give at this point a 
verbal restatement of the contents of Section 2.8.1, i.e., of its descrip- 
tions of the states and of the transition rule. Indeed, the formalism 
of Section 2.8.1 is not easy to follow without the verbal motivations 
of Sections 2.2-2.7. On the other hand, the verbal elaborations of 
those sections are lengthy and were arrived at stepwise. A direct 
verbal restatement is therefore indicated. 

Such a restatement, covering the states and the transition rule 
together, is given in what follows. 

There exist 16 transmission states T ua e . The index u indicates the 
class of the state: u = 0 for ordinary and u = 1 for special. The index 
a indicates the orientation of the state: a = 0 for right, a = 1 for 
up, a = 2 for left, and a = 3 for down. The index e indicates the pres- 
ent state of excitation: e = 0 for quiescent, and e = 1 for excited. 
A transmission state has one output direction and three input direc- 
tions: the former is the direction defined by its orientation; the latter 
are all the others. A transmission state can be excited with a delay 1, 
by any immediately neighboring excited transmission state of the 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 151 

same class, provided that the former lies in the output direction of 
the latter, which in turn must be lying in one of the former's input 
directions. 

There exist four confluent states C €C ' . The index e indicates the 
present state of excitation, the index e indicates the next state of 
excitation: e or e = 0 for quiescent, and e or e = 1 for excited. The 
confluent states are viewed as being of class 0. For a confluent state, 
all directions are available both for inputs and for outputs. [But at 
any given time a direction cannot be used for both an input and an 
output.] A confluent state can be excited with a delay 2 by those 
immediately neighboring transmission states of its own class (i.e., 0) 
in whose output direction it lies. The excitation will take place if 
there exists at least one such immediate neighbor, and if all such 
immediate neighbors (whatever their number) are excited. 

A transmission state (of either class) can also be excited with a 
delay 1 by any immediately neighboring excited confluent state, 
provided that the latter lies in one of the former's input directions. 

There exists an unexcitable state U. This state is viewed as quies- 
cent. Any transmission or confluent state is killed (i.e., transferred 
into the unexcitable state) by any immediately neighboring excited 
transmission state of the opposite class, provided that the former 
lies in the latter 's output direction. 

All the above states (transmission and confluent) go into their 
quiescent forms when no excitation or kill, according to the above 
rules, is provided for. 

There exist eight sensitized states Sz , with 2 according to list (S.l). 
We will also use the symbol Ss with 2 according to list (S.2), but 
these latter states are not considered to be sensitized ones. They are 
identified with the quiescent transmission and quiescent confluent 
states according to list (S.3). (For the references to lists (S.1)-(S.3), 
cf. Sec. 2.8.1.) A sensitized state Ss will in any case undergo a change 
(immediately, i.e., with a delay 1 ), namely into Sso or S21 . The change 
into S21 will take place under the influence of any immediately 
neighboring excited transmission state (of either class), provided 
that the sensitized state lies in the output direction of the latter. 
Otherwise, a change into S20 will take place. 

We re-emphasize that this last mentioned rule applies only as long 
as the state is sensitized, i.e., as long as 2 is according to list (S.l), 
and not according to list (S.2) (i.e., as long as it has not reached a 
maximum size). 

[ 2.8.3 Illustrations of the transition rule. Each cell of von Neumann's 
infinite cellular structure is occupied by the same 29-state finite 



152 



THEORY OF SELF-REPRODUCING AUTOMATA 



automaton. As explained in Section 1.3.3.5, an "infinite cellular 
automaton" consists of this infinite cellular structure together with 
an initial cell assignment. An "initial cell assignment" is a finite list 
of cells together with an assignment of a state to each cell of the list; 
all cells not in the list are assigned the unexcitable state U. An initial 
cell assignment determines the state of an infinite cellular automaton 
at time zero. The history of the infinite cellular automaton is then 
determined by the transition rule, which gives the state of each 29- 
state finite automaton at time t + 1 as a function of its own state 
and the states of its four immediate neighbors at time t. 

The 29 states and their transition rule are summarized in Figures 
9 and 10. Von Neumann also symbolized pairs of transmission states 
by a zero (for ordinary) or a one (for special), together with an arrow 
indicating the output direction. For example, the pair of states T 0 oe 
is represented by °> , while Tn € is represented by 1 1 . 

There are many ways of viewing the 29-state automaton which 
occupies a single cell of von Neumann's cellular structure. It can be 
viewed as a finite automaton which is constructed from switch ele- 
ments and delay elements and which is connected to its four immediate 
neighbors by wires crossing its four boundaries. In what follows it is 
more fruitful to view it as a set of primitive elements (sub-automata) 
together with control apparatus for switching back and forth among 
these elements. This amounts to partitioning the 29 -states of a cell 
into subsets which correspond to certain functions. 

Consider as an example the pair of transmission states T 0 oo and 
Tooi , i.e., Too € for e = 0, 1. This pair of states functions as a disjunc- 
tion (+ neuron) feeding a unit delay whose output is directed to the 
right. The potential inputs of Too* are from its immediate neighbors 
above it, to its left, and below it. These inputs come from confluent 
states or from other ordinary transmission states which are directed 
toward it. Thus in Figure 11a, the cell T 0 oe behaves as follows. If 
either Ci e ' or Ton at t, then Tooi at t + 1; if both Co e ' and T 0 io at t, 
then Tooo at t + 1. In other words, as long as Figure 11a is limited 
to the states shown there, the two states Too* can be thought of as 
constituting a disjunctive element with inputs from the left and be- 
low and an output (after unit delay) to the right (at/). 

The set of four confluent states C t€ ' (e, e = 0, 1) perform the func- 
tions of conjunction ("and", "•"), double delay, wire branching 
(splitting), and conversion of ordinary stimuli into special stimuli; 
von Neumann also symbolized these four states by C. These states 
have no direction, the direction of their functioning being determined 
by the directions of the transmission states (both ordinary and 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 153 

special) in the four immediately neighboring cells. Consider a set of 
states C £t ' occupying a given cell. The inputs to this set C eC ' are from 
ordinary transmission states directed toward it. The outputs from 
this set C CC ' are to both ordinary and special transmission states not 
directed toward it. See Figures lib and 11c. In Figure lib the cell 
C C€ ' behaves as follows. If both Tooi and Ton at t, then C e 'i at t + 1 
and Cie" at t + 2; if either Tooo or Toio at t, then C C ' 0 at t + 1 and 
Coe" at t + 2. In other words, in the context of Figure lib, the four 
states C et ' can be thought of as constituting a conjunctive element 
with inputs from the left and below and with output (after two units 
of delay) to the right (at g). 

It is convenient to think of the state e of a T wa€ cell or a C cc > cell 
as its output state at a given time, and to think of the composite 
state of its immediate neighbors as its input state at that time. It is 
also convenient to use short bars to indicate an input to a cell (a, 6, c, 
d, and e of Fig. 11) or as output from a cell (/, g, h, i, and j of Fig. 
11). Under this convention, we have for Figure 1 1 : 

f(t + 3) = [a(t) + b(t + 1)] 

g(t + S) = [c(t)-d(t)] 

h(t + 4) = e(t) 

i(t + 5) = e(t) 

j(« + 4) = e(t). 

All the outputs are ordinary stimuli except that the output j is a 
special stimulus. 

Each switching function (truth function) of disjunction (+) and 
conjunction (•) can be realized, with suitable delay, by a cellular 
network of ordinary states (To ac and C«')« Storage loops may be 
made of ordinary states. In Figure 12 the loop Bl, B2, A2, Al stores 
the sequence 10000, which will cycle around the square ad infinitum, 
continually feeding into cell Cl . Negation is not represented directly 
in the 29 states but is synthesized from the destructive (reverse ) and 
constructive (direct) processes. See Section 3.2.2 and Figure 17 below. 

The direct process changes a cell from the unexcitable state U into 
one of the nine quiescent states T Ma0 (u = 0, 1; a = 0, 1, 2, 3) or 
Coo . Transmission states (ordinary or special) directed toward U 
initiate and control the direct process, and sensitized states serve as 
intermediaries. Any (or several) T ua i directed toward U converts it 
to Se . Thereafter Ss is followed by (a) Ssi if some T wai is directed 
toward the cell, (b) Sso otherwise, until the direct process terminates 
in a T wa0 or Coo in accordance with Figure 10. For example, when 
sent into a cell which is in state U, the sequence 10000 produces 



154 



THEORY OF SELF-REPRODUCING AUTOMATA 



Tooo, the sequence 1011 produces Tioo , and the sequence 1111 pro- 
duces Coo . 

The direct process is illustrated in Figure 12. As indicated by the 
subscripts (values of e and e'), the loop Bl, B2, A2, Al, stores the 
sequence 10000 in that order at time zero. This sequence will cycle 
around the loop ad infinitum, repeatedly feeding into cell Cl. It 
will pass through cell Cl with a unit delay. Its effect on cell Dl through 
time is as follows: 

Time: 0 12 3 4 5 6 7 

Input to Dl: 0 1 0 0 0 0 1 0 
State of Dl: U U Se So Soo Sooo Tooo Tooi • • • . 

The process will now repeat, with cell Dl feeding the sequence 10000 
into the cell on its right, causing it to pass through the sequence 
S* , So , Soo , Sooo , Tooo , Tooi , • • • . This process will again repeat, 
ad infinitum. Thus an infinite cellular automaton which has the first 
pattern of Figure 12 as its initial cell assignment will grow an ever 
lengthening communication channel 0> Z t • • • to the right. 

The reverse process (destruction, killing) changes any transmission 
state Tuat and any confluent state C ce > into U. An excited ordinary 
transmission state T 0a i kills a special transmission state Ti ae toward 
which it is directed, and an excited special transmission state T ia i 
kills an ordinary transmission state To a€ or confluent state C cc > 
toward which it is directed. Examples are given in Figure 13. In 
Figures 13a and 13b cells shown with question marks at time 1 are 
so marked because their states at this time depend on the states of 
the cells in the surrounding environment. Except for Figures 13a 
and 13b, we always assume that the finite cellular arrays of our figures 
are not affected by the direct or reverse process operating on them 
from the environment. Note that the special and ordinary transmis- 
sion states of Figure 13b kill each other. 

The reverse process dominates reception. If a "kill" stimulus enters 
a cell at time t, that cell will be in state U at time t + 1, no matter 
what other stimuli may have entered the cell at time t. On the other 
hand, killing does not dominate emission, in the sense that if a cell is 
prepared to emit a stimulus at time t it will in fact emit that stimulus, 
even though it also receives a kill stimulus at time t. We will illustrate 
these facts with Figure 13c. 

Assume in Figure 13c that ordinary stimuli enter input a at times 0 
and 1 , a special stimulus enters input c at time 1 , and no other stimuli 
enter the figure at any other time. The ordinary stimulus entering a 
at time 0 enters cell Bl at time 1, leaves cell Bl at time 2, and leaves 



SYSTEM OF 29 STATES WITH A TRANSITION RULE 



155 



cell Cl (output b) at time 3. The special stimulus which enters input 
c at time 1 enters cell Bl at time 2, causing cell Bl to be in state U 
at time 3. The ordinary stimulus entering a at time 1 enters cell Bl 
at time 2, but, since the special kill stimulus also enters cell Bl at 
time 1, the ordinary stimulus has no effect. Hence the first ordinary 
stimulus into input a is transmitted through cells Al, Bl, Cl and is 
emitted from output 6, but the second ordinary stimulus into input a 
is dominated by the kill stimulus from input c and is lost. 

One fundamental construction in the cellular system is to change 
the state of a remote cell by the direct process and then wipe out the 
constructing path used for the construction. The technique is illus- 
trated in Figure 14. Ordinary stimuli are fed alternately into inputs 
i and j, they travel along the path B2-D2, and cell D3 is left in the 
quiescent state. The sequences required are as follows (cf. Fig. 10). 
(a) Input i is supplied with 10000, 10000, 1010, and 1111. The path 
B2-D2 becomes an ordinary transmission channel, and cell D3 is 
left in the desired state C, as in Figure 14b. (b) Input j is supplied 
with 1,1011; 1,1011; and 1. The cells B2 and C2 become U, S e , S 0 , Soi 
and Son (i.e., Tioo) in turn. The cell D2 becomes U, as in Figure 14c. 
(c) The sequence 1,10000, and 1 into i produces Figure 14d. (d) 
Finally, the single stimulus 1 into j produces Figure 14e. Thus we 
are left with cell D3 in the desired state and with the constructing 
path B2-D2 in the unexcitable state. 

The transformations of Figure 14 require 37 time steps, measuring 
time from the input sides of cell B2. As described above, stimuli come 
from inputs i and j alternately, but, since the absence of a stimulus is 
represented by "0," we can think of a sequence of length 37 going 
into cell B2 via input i and a simultaneous sequence of length 37 
going into cell B2 via input j. The path from input j to cell B2 re- 
quires 2 more units of time than the path from input i to cell B2; 
hence the input to j must lead the input to i by 2 units of time. Con- 
sequently, the transformation from Figure 14a to Figure 14e will be 
brought about by the two 36-bit sequences of ordinary stimuli shown 
at the bottom of Figure 14; in these sequences time goes from left to 
right. The sequence for input i may be obtained by prefixing i with a 
sequence of 36 cells in states Too* , each e being chosen according to 
the needed input into i. Likewise, the sequence for input j may be 
obtained by prefixing j with a sequence of 36 cells in the appropriate 
Tooc states. 

The above technique may be used to construct an arbitrary finite 
array d of quiescent cells, i.e., cells in the states U, T wa0 (u = 0, 1; 
a = 0, 1, 2, 3), or Coo . For each such array (2, there are two binary 



156 



THEORY OF SELF-REPRODUCING AUTOMATA 



sequences of stimuli which, when fed into inputs i and j of Figure 14, 
will produce the array & to the right. Moreover, these two sequences 
will emanate from two linear arrays of cells in the states T 0 o € , with 
the e's properly chosen. These two linear arrays, together with cells 
Al, A4, B1-B4 of Figure 14 constitute an array (B. Thus we have the 
following result about constructions in von Neumann's cellular 
structure: for each quiescent finite array Ct, there is a finite array (B 
and a time r such that array Ct will appear at time r in the infinite 
cellular automaton which has (B as its initial cell assignment. More- 
over, the limitation that the cells of array & be quiescent, while 
important (cf. Sec. 1.6.3.2 above), is not serious, for the array (B 
can impart a starting stimulus to array & before returning the con- 
structing path to the unexcitable state. 

It is clear from the previous construction that area (B is always 
larger (contains more cells) than CL Compare Section 1.6.1.1 above. 
Von Neumann circumvents this difficulty by designing a universal 
constructing automaton D and attaching to it a tape description of 
itself L D ; see Section 1.6.1.2 above. This universal constructing 
automaton will have, among other powers, the power of a universal 
Turing machine. The technique of Figure 14 will play an essential 
role in the operation of this universal constructing automaton C. 
See Chapters 4 and 5 below.] 



Chapter 



DESIGN OF SOME BASIC ORGANS 



3.1 Introduction 

3.1.1 Free and rigid timing, periodic repetition, and phase stop. 
The organs that will be constructed in the sections that follow will 
exhibit two relevant traits (separately or jointly): free timing and 
rigid timing. 

We have free timing when the emission of certain stimuli (at a 
specified point and in a certain order) is provided for, but the time 
intervals or delays from one such emission to the next are not pre- 
scribed. 

We have rigid timing when the delays of the above pattern are 
numerically specified. We may alternatively express this by stating 
that these delay periods are filled with "no stimuli." (Clearly there 
will be d — 1 of these for a delay period between a prescribed stimu- 
lus and its successor, if the specified delay between these two is d.) 
The rigidly timed sequence of stimuli can therefore also be described 
as a "stimulus-no-simulus sequence" of uninterruptedly consecutive 
events. In this form it will be designated by a sequence of O's and l's, 
each 1 standing for a "stimulus," and each 0 for a "no stimulus": 
ft ... jn (each i v = 0, 1). Whenever this notation i 1 • • • i n is used, 
rigid timing is implied. 

[The direct process of Section 2.6.3.2 above involves rigid timing. 
As von Neumann said there, in rigid timing "the absence of a stimu- 
lus has as definite an effect as the presence of one, and therefore the 
stimuli that are required must be delivered at definite times, without 
any delays other than the expressly required ones." 

As we saw in Section 1.3.3.5, there is no bound on the number of 
cells which can be in the excited state at any given time, though of 
course this number is always finite. Thus von Neumann's cellular 
structure allows for an indefinite amount of parallelism. But in de- 
signing his self -reproducing automaton von Neumann did not make 
much use of the potential parallelism of his cellular structure. Rather, 
his self -reproducing automaton works like a serial digital computer, 



157 



158 



THEORY OF SELF-REPRODUCING AUTOMATA 



with most organs normally quiescent. In this respect it is similar to 
the ED VAC (see pp. 9-11 above). 

When an organ of the self -reproducing automaton is stimulated by 
a sequence containing one or more stimuli, the organ will produce an 
output (response) after a delay. Usually the delay between input 
and output doesn't matter, because no other construction or com- 
putation is going on simultaneously. Likewise, the delay between 
one input to an organ and the next input to that organ does not usually 
matter, provided that this delay is sufficiently great for the organ 
to finish one action before starting the next. The main exception to 
these statements concerns the tape unit of Chapter 4 below. The 
method von Neumann used for lengthening and shortening the tape 
requires simultaneous (parallel) action in two connecting loops.] 

By a periodic repetition of such a sequence i l • • • i n a rigidly timed 
periodic repetition is meant, i.e., one with uninterruptedly consecutive 
periods i l • • • i n , unless the opposite is expressly stated. Thus a periodi- 
cally repeated i l • • • i n , to be designated i 1 • • • i n , means i l • • • i n i l • • • 

i n i l - - - i n Such periodic repetitions are never intended to 

go on indefinitely. By a stop in phase k (= 1, • • • , n) in period s 
(= 1,2, • • • ), we mean that the sequence is interrupted immediately 
before the i k in the s-th period (i.e., no stimuli are emitted at this 
time or later). We may also call this a stop at step I with I = nx + k 
(I = 1,2, • • • ). By a stop in period s (without stating a phase) we 
mean one in phase 1 (i.e., at step ns + 1 ). 

8.1.2 Construction of organs, simple and composite. We will now 
construct successively certain organs, going from the simpler to the 
more complicated. Most of these organs will be composites of previ- 
ously defined organs with each other and with suitable controlling 
and connecting networks. Each one of the organs to be defined will 
be given a name and a symbol, so that it can be identified when it 
occurs as a part of a subsequent larger, composite organ. 

[In some cases von Neumann gave algorithms for designing any 
organ of a given class; this is so for pulsers (Sec. 3.2), decoding 
organs (Sec. 3.3), and coded channels (Sec. 3.6). In the case of the 
triple-return counter (Sec. 3.4) and the 1 vs. 10101 discriminator 
(Sec. 3.5), he designed specific organs. To make von Neumann's 
algorithms and constructions easier to follow, we discuss a completed 
organ at the beginning of each section. 

Since von Neumann was interested in an existence proof of self- 
reproduction he did not, in general, attempt to minimize his design.] 



DESIGN OF SOME BASIC ORGANS 



159 



3.2 Pulsers 



3.2.1 The pulser: structure, dimensions, and timing. 

[ Figure 15 shows two pulsers designed according to von Neumann's 
algorithm. We will explain how they work. 

A stimulus (e.g., Tooi) into input a of pulser P(lll) at time t is 
split at each confluent element and produces a sequence 111 from 
output b at times t + 9, t + 10, and 2+11. The cross-hatched 
cell is in the unexcitable state U. 

The characteristic 10010001 contains three ones, so three paths (B, 
D, and F) are needed for P (10010001), with relative delays 0, 3, and 
7, respectively. The confluent cells along the bottom produce relative 
delays of 0, 1, and 2, respectively. Additional delays are achieved 
in two ways. Replacing two ordinary-up transmission states by a 
block like C5, C6, D5, and D6 adds two units of delay to a path; 
by this means we get relative delays of 0, 3, and 6 in the three paths. 
Replacing an ordinary transmission state in row 2 by a confluent 
state (e.g., F2) adds a unit of delay to a path; in this way we ob- 
tain relative delays of 0, 3, and 7 in the three paths B, D, and F, 
respectively.] 

We begin with the pulser. This organ has an input a and an output 
b. Upon a stimulus at a it will emit a prescribed sequence i • • • i n at b. 

The relation between the stimulus at a and the response at b is 
freely timed; i.e., the delay between these is not prescribed at this 
point (cf., however, the remarks at the end of this subsection). 
This pulser has the symbol J*(i l ••• i n ). The sequence i l ••• i n , 
which can be prescribed at will, is its characteristic; n is its order. 

The principles involved in constructing a pulser are quite simple. 
The actual network is successively developed in Figure 16. This 
construction and the network that results from it will be discussed 
in the balance of this section. 

Let vi , • • • , v k be the v for which i v = 1, i.e., the positions of 
the stimuli in the sequence i 1 • • • i n . Then 1 <> v\ < V2 < • • • < pjc S n> 
Hence v h ^ h. Write 



The input stimulus at a must be broken up, or rather multiplexed 
into k distinct stimuli, numbers h = 1, • • • , k, arriving at b with the 
relative delays ¥h (h = 1, • • • , k), respectively. 

Consider first the network of Figure 16a. A stimulus arriving at a 





where pn = 0, 1, 2, • • • ; rn = 0, 1. 



100 



THEORY OF SELF-REPRODUCING AUTOMATA 



has k paths available to reach b: on path number h(h = 1, • • • , k) 
it goes horizontally from a to the C number h, then it turns vertically 
up, ascends to the top line, and continues there horizontally to b. 
There are (2h - 1) + u + (2(k - h) + 1) = 2k + u steps, but, 
since h of them are C's, the entire delay involved is (2k + u) + h. 
That is, path number h has the relative delay (relatively to the 
other paths!) h. 

Note that this network has the width 2k — 1, but that its height 
u + 2 is still undetermined, along with u. 

We must now replace the relative delay h of the path number h 
in Figure 16a by a relative delay v h (for each h = 1, • • • , k) \ i.e., 
we must increase it by v h — h = 2\x h + r h {cf. formula (l')}. We 
replace therefore each one of the k vertical branches in Figure 16a 
by a suitable delay network, say branch number h by a network 
Nh , as shown in Figure 16b. (We have also moved input a one step 
left, by placing a ° before the first C.) Hence Nh must produce a 
delay 2fx h + r h . 

Nh is show T n again in Figure 16c, indicating its input Ch and its 
output dh . (ch is immediately above the C number h, dh above this, 
under the top line, i.e., immediately below the !t number 2h — 1 
in the top line.) To achieve the desired delay 2/u + rh , Nh may be 
built up by stacking ^ delay-two blocks vertically, plus a single 
delay-one block if = 1. The former are shown in Figure 16d; their 
height is 2. The latter is shown in Figure 16e; its height is 1. The 
total height of Nh is then adjusted to the uniform value u by in- 
serting an ordinary vertical ascent of length u — 2\ih — rh = u — 
(v h — h) = Sh {cf. formula (l')}. This is shown in Figure 16f. (The 
delays referred to above are, of course, always relative ones, compared 
to an ordinary vertical ascent.) It is best to put the ix h blocks of 
Figure 16d at the bottom of Nh, the (possible, single) block of 
Figure 16e at the top of Nh , and the vertical insertion of Figure 16f 
in between. The reason for this last precaution is that a C of Figure 
16e must never be in contact with a transmission state of Figure 
16d, since this might produce unwanted stimulations. The U-padding 
in Figures 16e and 16f is such that the contact in question could 
occur only between Figures 16e and 16d, and then only with the 
latter to the right of the former. Hence the top line of the Nh (h = 1 , 
• • • , k) must contain no Figure 16d block immediately to the right 
of a Figure 16e block. 

In view of all this we have, therefore, these conditions: always 
u ^ Vh — h. If some n = 1 (i.e., Vh — h odd, i.e., some Fig. 16e 



DESIGN OF SOME BASIC ORGANS 



161 



block in the top line), and the maximum of vh — h is even, then 
u > vh — h for every h. In other words: u ^ u, where 



(2') 



u = Max (vn — h) + €°, 

h=l, • • •, 

where e° = 1 if the Max is even but some v h — h is odd, 
and e° = 0 otherwise. 



It is, of course, simplest to put u = u°. 

[This rule is wrong when v k — k = 1, as in P(1010), for a con- 
fluent state in the bottom row cannot directly drive a confluent 
state above it. This oversight may be corrected in different ways. 
We correct it by adding the following restriction to von Neumann's 
rule (2'): 

If v k — k = 1, then u = 2. 

Note that Max (v h — h) = v k — k. 

Von Neumann introduced e° because he did not want a Figure 16d 
to be immediately to the right of a Figure 16e, since the confluent 
state of the latter figure would affect an ordinary transmission state 
of the former figure. However, if Figure 16e is in the next to the top 
row of the pulser, and Figure 16d occupies this row and the row 
below it, this effect does not in fact change the output of the pulser. 
We could therefore replace von Neumann's design algorithm by a 
new algorithm with this simpler rule for determining u: 

If Vk — k = 1, then u = 2 

Otherwise, u = v k — k. 

We will not make this replacement, however, since we wish to keep 
as close to von Neumann's original design as possible.] 

This completes the construction. As Figure 16b shows, the area of 
this network has the width 2k and the height u + 2. An abbreviated 
representation of this network is given in Figure 16g. 

Note that the sequence i l • • • i n , i.e., the relative delays v h (h = 
1, • • • , k), is achieved with a certain preliminary absolute delay. 
We saw that this was 2k + u under the conditions of Figure 16a. The 
insertion of the *L before the first C in Figure 16b raises this to 2k + 
u + 1. Thus in the final arrangement, represented by Figure 16g, 
a stimulus at a starts the sequence i l • • • i n after a preliminary 
[absolute] delay 2k + u + 1 at b. (This means that the first stimulus 
or no-stimulus position of that sequence, i 1 , has a delay 2k + u + 2.) 

If input a is stimulated several times, the emissions at b take place 
concurrently, irrespectively of whether the sequences i 1 • • • i n that 



162 



THEORY OF SELF-REPRODUCING AUTOMATA 



are so induced overlap or not. That is, there is no corruption by 
interference in this network. 

[We will summarize the external characteristics of the pulser 
P(i l ~ i n ). See Figure 16g. 

The width of the pulser is 2k, where k is the number of ones in the 
characteristic i l • • • i n . Von Neumann implicitly assumed that 
k ^ 2; for k = 0 or k = 1 no organ is needed. 

The height of the pulser is u + 2, where u is defined as follows: 
vi , • • • , v k are the v for which f = 1. Von Neumann's rule for u, 
as corrected, is 

If Vk — k = 1, then u = 2. 

Otherwise, u = (v k — k) + e, where 

e° = 1 if (v/c — k) is even but some vk — h (h = 1, • • • , k) is 
odd, 

e° = 0 otherwise. 

Note that k is the number of ones in the characteristic and that 
v k is the superscript of the rightmost one; hence ^ — A; is the total 
number of zeros in the sequence preceding the rightmost one of the 
characteristic. 

The delay between the input pulse going into a and the first 
output i emerging from b is 2k + u + 2.] 

3.2.2 The periodic pulser: structure, dimensions, timing, and the 
PP (I) form. 

[ Figure 17 shows two pulsers designed according to the algorithm 
von Neumann developed next. 

The periodic pulser PP (10010001 ) is constructed from a pulser 
P (10010001 ) which produces one occurrence of the desired sequence, 
a periodic repeater G4--G6 and H^-H6 of period eight, a mechanism 
H1-H3 for turning the repeater off, and a pulser P (1111111111) 
which produces the signals needed for turning the repeater off. 

A "start" stimulus into input a+ at time t will produce 10010001 
from an outpu t b' of P ( 10010001 ) at times t + 29 through t + 36. 
The sequence 10010001 is emitted repeatedly from output b until 
the repeater is turned off. A "stop" stimulus into input a_ will cause 
the top pulser to emit a sequence of 10 ones. These will pass through 
the confluent state H2, the special transmission state H3, and into 
cell m. The first five pulses will induce the transformation C €e ' — ► 
U — -> % — > Si — > Sn — > Sin (= Coo), and the last five pulses will 
repeat this transformation. No matter what the contents of the 
periodic repeater are, they will be wiped out. Note in this connection 
that since a sequence of ones is used to turn the repeater off, any 
ones entering cell H4 from the left (i.e., from the repeater itself) are 



DESIGN OF SOME BASIC ORGANS 



163 



ineffectual. This is a motive for putting the confluent state at the 
bottom of the periphery of the tree of Figure 10. 

Since the periodic pulser PP(T) is used very often, von Neumann 
elected to simplify the design for this special case. Here he used the 
fact that, when Figure 17b is operating, there is a sequence of six 
ones in its periodic repeater. Once the transformation of the confluent 
state E3 is begun by a pulse from E2 the transformation is completed 
by the four ones remaining in the periodic repeater (two ones being 
lost when E3 is killed). This periodic pulser does not, however, 
work in all contexts. See the editorial discussion at the end of this 
subsection.] 

The next organ that we construct is the periodic pulser. This organ 
has two inputs a+ , a_ , and an output b. Upon a stimulus at a+ it 
will begin to emit a prescribed, periodically repeated sequence, say 
i 1 • • • i n at b. Upon a stimulus at a_ this emission at b will be stopped. 

The relation between the stimuli at a + and a_ and the (starting or 
stopping of the) response at b is freely timed; i.e., the two delays 
between these are not prescribed at this point (cf., however, the re- 
marks at the end of this section) . 

This periodic pulser has the symbol PP^' 1 • • • i n ). The sequence 
i l • • • i n , which can be prescribed at will, is its characteristic; n is its 
order. 

The required network is successively developed in Figure 18. This 
construction and the network that results from it are discussed in 
the balance of this section. 

The operation of producing the periodically repeated sequence 
i l • • • i n is best decomposed into two suboperations. First, we produce 
a single sequence i l • • • i n ; second, we repeat it periodically. The 
first task calls for the pulser P(i x • • • i n ) of Section 3.2.1, according 
to Figure 16g. We write a\ b' for a, b there. Then our present input 
a + must either feed into or be the input a \ The second task requires 
attaching to the output b' an organ that will repeat any period of 
length n to the output b. 

The simplest periodic repeater is a (closed) cycle of transmission 
states. The shortest cycle of this kind has a period 4, as shown in 
Figure 18a. Clearly any [such] cycle must have an even period; hence 
the general cycle has the period 2f, I = 2, 3, • • • .A cycle of this pe- 
riod is shown in Figure 18b. 

Our present output b must issue from this repeater. Hence at least 
one of its cells must be able to stimulate in two directions (6, as 
well as its own successor in the cycle) ; i.e., it must be a C. The output 
b has two possible positions on this C : bi , 6 2 . 



1G4 



THEORY OF SELF-REPRODUCING AUTOMATA 



This is shown in Figure 18c. The C raises the length of the period 
to 2t + 1. This is an odd period. If an even period is wanted, a second 
C must be inserted (since the first one cannot be removed). This is 
shown in Figure 18d; the period length is here 21 + 2. (Note that 
the two C are not immediate neighbors, since a C cannot stimulate 
a C.) Both alternatives are jointly represented by Figure 18e, where 

( = o for r = 1. 

X 

( = C for r = 2. 

Here the period is n = 21 + r, where I = 2, 3, • • • ; r = 1, 2. That 
is, we can handle precisely the orders (periods) n = 5, 6, • • • . 
Thus the order n is subject to the condition 

(3 r ) n ^ 5. 

If the condition (3') is violated, i.e., if the order n is <5, then 
the period i 1 • • • i n may be repeated 0 times. This replaces n by 0n, 
and we need only choose 0 so that On fulfills (3'), i.e., On ^ 5. 

Let us now return to Figure 18e: The output b must be on the C, 
as shown there. The input of the cycle, i.e., the point fed by b', 
should be as close as possible to this C, to minimize the delay. It 
cannot be on the C itself, because C's remaining free side (which- 
ever of 6i , 6 2 is not used for b) will be needed for another purpose. 
We place it therefore immediately before C, at bi or 6 2 '. (The sense 
of the cycle has been arranged to maximize this accessibility.) 

Now the pulser P^' 1 • • • i n ) of Figure 16g and the cycle of Figure 
18e can be placed into actual contact. This is shown in Figure 18f, 
where 

n = 21 + r; 

I = 2,3, ;r - 1,2. 




= TO forr = 1. 
= C for r = 2. 



Figure 18f is drawn as if I < u + 2, but I ^ u + 2 is equally possible. 
Note that the cycle is rotated against its previous form (Fig. 18e vs. 
Fig. 18f), to have a convenient fit with P^' 1 • • • i n ). In addition the 
two organs are separated by a column of IPs, to avoid the possibility 
of unwanted stimulations due to contacts between a C of one organ 
and a transmission state of the other. 

The network of Figure 18f takes care of starting j 1 ■ — j n ; there 
remains the task of providing a network for stopping i l • • • i n . 



DESIGN OF SOME BASIC ORGANS 



Since we have not introduced any states (or stimuli) for expressing 
the negation or inhibition directly, this must now be done by indirect 
means. The obvious way to achieve this is by using special stimuli, 
i.e., by changing the character of one of the transmission and con- 
fluent states that make up the cycle of Figure 18e (which is, after a 
rotation, part of Fig. 18f). This means that a_ should control a 
stimulus [1] which is directed against one of the exposed sides of the 
cycle. 1 Figure 18g shows the organ that will do this. Since it will be 
convenient to stimulate this organ (from a_ via a") from above 
and to attach it to the upper side of the cycle, Figure 18g shows it 
in this position. The output b" of this organ may be attached to any 
cell of the cycle. In the position shown (cf. Figs. 18f and 18g) this 
can be the \ or the C; in the two other possible positions (on Fig. 
18f, i.e., from the right or from below) it could be the C, any lo ? 
the Z. (from the right), the L or the X (from below). 

With this arrangement, a stimulus [0] from a_ , arriving at a , would 
excite C and then i 1 , and deliver a stimulus [1] at b" with a delay 3 
(counted from a ). Hence, with a delay 4, that cell of the cycle which 
is in contact with b" will go into the state U and cease to emit. 
To this extent the stop has been effected. However, there remains 
the question of the further fate of this cell. Indeed, this cell is now a 
U; hence the next stimulus [0] traveling through the cycle will con- 
vert it into an S e (unless a stimulus [1] does this, by reaching the 
cell earlier via b"). After this, stimuli ([0] via the cycle of [1] via 
6"), as well as their absence, will transform it further, through the 
sensitized states {the S2 with 2 from list (S.l) of Sec. 2.8.1}, to an 
(ordinary or special) transmission state or to the confluent state 
{T, T' or C, i.e., an S 2 with 2 from list (S.2) of Sec. 2.8.1}. Which 
of these terminal states will actually be reached depends on the 
stimuli circulating in the cycle (plus stimuli possibly delivered via 
b ), i.e., on the sequence i l • • • i n , as well as on the phase of this 
occurrence. 

In order to have a scheme that will work for all i 1 • • • i n and — 
what is more important — for all phases in which the stop may be 
ordered, it is best to see to it that U gets an adequate and uninter- 
rupted sequence of stimuli in any case. Hence it is best to deliver 
these from b" . Consequently they will be stimuli [1]. The first one 
will convert U into S# , and three more will transform this into 
Sin = Coo . These stimuli [1] follow the original stimulus [1], which 
produced the U, immediately. Hence a total of five consecutive 

1 [As explained in Sec. 2.8.1 above, "[0]" represents an ordinary stimulus, 
and "[1]" represents a special stimulus.] 



THEORY OF SELF-REPRODUCING AUTOMATA 



stimuli [1] at 6" is called for, i.e., five consecutive stimuli [0] at a" . 
However, before we discuss these any further, let us consider the 
sequence of transformations that have been induced : 

, Cycle cell (one of the C, X, 1, lo, I) -> 

(4) 

— > U — > Se — > Si — > Sn — > Sin — Coo . 

In the end it will be necessary to restore the cycle cell that has 
been so transformed to its original condition. It is simplest to use the 
sequence (4') itself for this purpose. This sequence ends with a C; 
hence it should have begun with a C, too. That is, the cycle cell in 
question should be the C in the cycle. This means that the b" of 
Figure 18g must be the &i of Figure 18f. Consequently the (ultimate) 
output b must be 6 2 . 

Let us now return to the five stimuli [0] at a" . These could be 
furnished by a pulser P (11111), whose output b* will then feed a", 
while its input a* is fed by a_ . However, one more point needs to 
be considered. 

The above 5 stimuli paralyze the output of the cycle cell of (4') at 
the time when it goes into the state U, also at the 4 successive times 
when it goes into the states , Si , Sn , S m = Coo ; and finally, 
since C responds with a delay 2, there will also be no output from 
this cell at the next time. That is, this cell is silent for 6 consecutive 
times. If the length n of the period is ^6, then this is sufficient to 
silence the cycle for good. However, if n > 6, then this sequence of 5 
stimuli [1] at b" , i.e., of 5 stimuli [0] at a ' , must be repeated. Let 
us repeat it p times. It is clear from the above, that we could insert 
a 0 (no stimulus) between every two groups of 11111 (5 stimuli), 
but it is simpler to omit this insertion. So we have a sequence 1 • • • 1 
of order 5p. This will take the cycle cell p times through the sequence 
(4 ), and thereby silence the cycle (i.e., its output b = 62) for 5p + 1 
consecutive times. Hence it will silence it for good if n ^ 5p + 1, 
i.e., if 

(5 ) p ^ — - — . 

0 

The simplest choice of p is, of course, as the smallest integer that 
satisfies condition (5'). 

Thus the pulser referred to above (with its output b* feeding a' 
and its input a* fed from a_) must be P(l • • • 1) of order 5p. The 
entire arrangement is shown in Figure 18h. Starring all quantities 
that refer to this pulser, we see that n* = k* = 5p and vh* = h(h = 



DESIGN OF SOME BASIC ORGANS 



1, • • • , 5p) obtain; hence u*° = 0 by rule (2') of Section 3.2.1, so 
that we can choose u* = 0. 

The border of LPs around the right and lower sides of the pulser 
in Figure 18h serves again to avoid the possibility of unwanted stimu- 
lations due to border C's. 

Before continuing, we will point out an important special case, 
where the introduction of the pulser of Figure 18h can be avoided. 
This is the case of the simplest possible period, namely 1. This has 
the order n = 1. In this case 1 we proceed as follows. 

The order n = 1 violates the condition (3 ); hence we must 0-fold 
it, so that On fulfills (3'), i.e., with an 0 ^ 5. (This would suggest 
0 = 5, but we postpone the choice.) The new n is 0 (the previous 
0n); the cycle in Figure 18f has the period n = 0. Now assume that a 
stimulus [1] at bi of Figure 18f (i.e., at b of Fig. 18g; cf. above), 
converts the right upper C in this cycle (Fig. 18f) into a U. This 
starts the sequence (4 ). To complete this sequence 4 more stimuli 
are needed. They are rigidly timed. In our discussion after sequence 
(4') we made them [l]'s from b" but [0]\s from the cycle would do 
just as well. After the cycle cell has undergone its first transformation 
in (4') {i.e., (C — * U)}, the cycle will still deliver n — 2 stimuli 
[0] to it. If ft — 2 = 4, i.e., n = 6, then this is precisely what is needed 
to complete the sequence (4'). At the same time the cycle will have 
been silenced for good, as desired (cf. the discussion leading to condi- 
tion (5'), or condition (5') itself with n = 6, p = 1). Hence we 
choose ft = 0 = 6. (Note that this fulfills our earlier condition 0 ^ 
5, but it is not the minimum choice 0 = 5 referred to there!) 

With this choice, then, a single stimulus [1] at b" (Fig. 18g) suf- 
fices. In this case, therefore, the addition of Figure 18h to Figure 18g 
is not needed. However, it is more convenient in this case to have the 
input a on the left, rather than on the upper side of the C. Also, 
since the pulser P(l • • • 1) of Figure 18h is now omitted, it is con- 
venient, for the unification of the notations, to identify a and a*. 
This is shown in Figure 18i. 

We can now perform the main synthesis by joining the network of 
Figure 18f with the network of Figure 18h {the general case: 
PP^ 1 • • . i n )) or the network of Figure 18i {the special case: PP (1 )} . 
As noted before, the contact must be made at b" = bi , and the 
(ultimate) output is b = 6 2 . The result is shown in Figures 18 j and 
18k, respectively. Note that in the second case n = 6, also h = 6, 
v h = h(h = 1, • • • , 6); hence by formula (2') we have u = 0, so 
that we can choose u = 0. Consequently, 2k = 12 and u + 2 = 2, 



1()8 THEORY OF SELF-REPRODUCING AUTOMATA 

us shown in Figure 18k. Note also that though Figure 18j is drawn 
as if I < u + 2 and lOp < 2k + 1, either or both of I ^ u + 2 and 
lOp ^ 2k + 1 are equally possible. 

Figures 18j and 18k contain certain delays, from the stimulation at 
a + , or rather at a, to the start at b, and from the stimulation at a_ , 
or rather at a*, to the stop at b. We will determine these. 

Consider the first case, i.e., Figure 18 j . 

The path from a to b 2 = b in Figure 18j lies entirely in the lower 
half of the network, i.e., in the part shown in Figure 18f. Using Figure 
18f, we saw at the end of Section 3.2.1 that the stimuli of i 1 • • • % n 
appear at b with an absolute delay of 2k + u + 1 after the stimulus 
at a . To this absolute delay must be added the relative delays 1, • • • , 
n, so that the sequence i 1 • • • i n appears at b f with the delays 2k + 
u + 2, • • • , 2k + u + n + 1. The delay from b' to b is clearly 4; 
hence the first period of i l • • • i n appears at b with the delays 2k + 
u + 6, • • • , 2k + u + n + 5. Thus the start at b (6 = 6 2 ; cf. Fig. 
18 j ) is delayed against the stimulus at a by 2k + u + 6. 

The path from a* to b 2 = b in Figure 18j consists of the part from 
a* to bi = \1\ followed by the part from bi = b" to b 2 = b. The part 
from a* to b" lies entirely in the upper half of the network, i.e., in 
the part shown in Figure 18h. Using Figure 18h, we saw at the end 
of Section 3.2.1 that the stimuli 1 • • • 1 (order 5p) appear at b* 
with an absolute delay of 2k* + u* + 1 after the stimulus at a. Since 
fc* = 5p, u* = 0, this delay is lOp + 1. To this absolute delay must 
be added the relative delays 1, • • • , dp, so that the sequence 1 • • • 1 
appears at b* with the delays lOp + 2, • • • , 15p + 1. The delay 
from b* to b" is clearly 5; hence the sequence 1 ••• 1 appears at 
bi — b" with the delays lOp + 7, • • • , I5p + 6. Now a stimulus [1] 
at bi (Fig. 18f) inhibits the next output from the affected C {cf. the 
discussion of sequence (4')} . Hence the output at 6 2 is stopped with a 
total delay of lOp + 8. Thus the stop at b (b = b 2 , cf. Fig. 18j) is 
delayed against the stimulus at a* by lOp + 8. 

Consider now the second case, i.e., Figure 18k. 

The lower halves of Figures 18 j and 18k have the same structure; 
hence the delay from a to the start at b is given by the formula de- 
rived above for the first case. This is 2k + u + 6. Since now k = 
6, u = 0 (cf. the discussion of Fig. 18k), this delay is 18. 

The delay from a* = a to the stop at b 2 = b in Figure 18k is also 
easy to determine. The path from a to bi = b" is also shown in 
Figure 18i. Its delay is clearly 3. The delay from a stimulus [1] at 



DESIGN OF SOME BASIC ORGANS 



1G9 



bi to a stop at 6 2 (Fig. 18f ) is the same as in the first case, that is, 1. 
Thus the stop at b (b = 62 ; cf. Fig. 18k) is delayed against the stimu- 
lus at a* (= a"; cf. Fig. 18k) by 4. 

We restate: The delay from a stimulus at a to the start at b is 
2k + u + 6 in the first (general) case (Fig. 18j) and 18 in the second 
(special) case (Fig. 18k). The delay from a stimulus at a* to the stop 
at b is lOp + 8 in the first (general) case (Fig. 18j ) and 4 in the second 
(special) case (Fig. 18k). 

In contrast with the situation at the end of Section 3.2.1, corrup- 
tion by interference in this network is a definite possibility. It is easily 
verified that this is controlled by the following rules. 

Throughout (6'.b)-(6'.d) stimulations at a', a* should be 
viewed in a modified chronological ordering, which differs 
from the ordinary one by certain systematic (relative) 
shifts. All statements in (6'.b)-(6'.d) about simultaneity, 
precedence, and ordering of such stimuli must accordingly 
be understood in this modified chronology. This chronology 
is defined as follows. The ordering of stimulations at a 
relatively to each other is unchanged; also the ordering of 
stimulations at a* relatively to each other is unchanged. 
Stimulations at a are displaced relatively to stimulations 
at a* in accordance with the difference between the delays 
from a to the start at b, and from a* to the stop at b. In 
other words, all these stimulations are ordered chrono- 
logically, not by the times of their respective occurrences 
(at a or at a*), but by the times at which they take effect 
(at b, in the form of a start at b for a , and of a stop at b 
for a*). 

Multiple stimulations at a', between which no stimu- 
lations at a* occur, simply superposes their effects. 
One could therefore say, as at the end of Section 3.2.1, 
that no corruption by interference takes place in this case. 
However, it must be noted that this means that the periodic 
emission of the organ (at b) may be changed in this process, 
since each new stimulation at a superposes the period 
with its own phase on the (possibly already composite) 
period produced by the previous stimulations. Finally, this 
change is void in the special case, where the period consists 
of stimuli only (it is l!), so that additional superpositions 
cannot alter it. 



170 



THEORY OF SELF-REPRODUCING AUTOMATA 



A stimulation at a* which follows upon stimulations at a 
{for these cf. rule (6'.b)} stops the period generated by 
these. Multiple stimulations at a* have no effect beyond 
that one of the first one among them; i.e., the stop caused 

< by the first one is maintained by the others, but it would 
be equally maintained without them. 2 It should be noted 
that if stimulations at a and at a* are simultaneous, the 
latter overrides the former; i.e., no emissions at b will 
occur. 

A stimulation at a which follows upon stimulations at 
a takes full effect {in the sense of (6 .b)j only if it nas a 
delay ^ 5p + 1 (this is the general case; in the special 

< case put p = 1 ) against the former. If it comes earlier, 
then those stimuli in the period whose first occurrences 
would take place with a delay < 5p + 1 will be perpetually 
removed from the period. 

To conclude, we will make some adjustments to equalize the delays 
from a to the start at b and from a* to the stop at b, thereby eliminating 
the complications caused by the special ordering of (6'. a) in (6'.b)- 
(6'.d). This will be done by inserting suitable delay paths between 
a + and a on one hand, and between a_ and a* on the other. We will 
distinguish three sub-cases, as follows. 

First, assume the case of Figure 18j and lOp < 2k + 1. Define z x 
by lOp + z\ = 2k + 1, so that Z\ = 1, 2, • • • . Attach the network 
shown in Figure 19a to the left side of Figure 18j. The delay from a + 
to a' is 3; hence the delay from a+ to the start at b is 3 + (2k + 
^ + 6) = 2k + u + 9. The delay from a_ to a* is u + z x ; hence 
the delay from a_ to the stop at 6 is (u + z x ) + (lOp + 8) = (lOp + 
zi) + (u + 8) = (2k + 1) + (u + 8) = 2k + u + 9. 

Thus both delays are 2k + u + 9, and Figures 18j and 19a put 
together assume the aspect of Figure 19c. This figure is drawn as if 
I < u + 2, but I ^ u + 2 is equally possible. 

Second, assume the case of Figure 18j and lOp ^ 2k + 1. Define 
z 2 by (2k + 1) + z 2 = lOp, so that z 2 = 0, 1, 2, • • • . Attach the net- 
work shown in Figure 19b to the left side of Figure 18j. The delay 
from a+ to a is 3 + z 2 ; hence the delay from a+ to the start at b is 
(3 + z 2 ) + (2k + u + 6) = ((2k + 1) + z 2 ) + (u + 8) = lOp + 
u + 8. The delay from a_ to a* is u\ hence the delay from a_ to the 
stop at b is u + (lOp + 8) = 10p + u + 8. 

Thus both delays are lOp + u + 8, and Figures 18j and 19b put 

2 [There is an error here, which we will discuss at the end of the subsection.] 



(6'.c) 



(6'.d) 



DESIGN OF SOME BASIC ORGANS 



171 



together assume the aspect of Figure 19d. This figure is drawn as if 
( < u + 2, but I ^ u + 2 is equally possible. 

Before going on to the third case, note that the common delay in 
the first case (where lOp < 2k + 1) is 2k + u + 9, while in the 
second case (where 10p ^ 2k + 1) it is lOp + u + 8. Both cases 
are covered by the expression M + u + 8, where 

(?') Af = Max (10p, 2fc + 1). 

Also, Figures 19c and 19d have in every way the same outward 
appearance, except that the first has the width 2k + 4, while the 
second has the width lOp + 3. Again both cases are covered by one 
expression, namely by M + 3. Hence Figure 19e represents both 
Figures 19c and 19d. Figure 19e is drawn as if I < u + 2, but I ^ 
u + 2 is equally possible. 

Third, assume the case of Figure 18k. Attach the structure shown 
in Figure 19f to the left side of Figure 18k. Also, identify a with a + . 
The delay from a+ to the start at b is then the same as that one from 
a, that is, 18. The delay from a_ to a* is 14; hence the delay from a_ 
to the stop at b is 14 + 4 = 18. 

Thus both delays are 18, and Figures 18k and 19f put together 
assume the aspect of Figure 19g. 

Figures 19d and 19g are the desired network in the case of a general 
PP^ 1 • • • i n ) and of the special PP(T) {represented with n = 6, i.e., 
as PP (111111)}, respectively. It is worthwhile to introduce a joint 
abbreviated representation for these. In the case of Figure 19d we 
bring it to the uniform height of N + 3, where 

(8') N = Max (u + 2, 

by filling in U's as needed. Figure 19g need not be changed at all. The 
result is shown in Figure 19h. For the general case of PP^' 1 • • • i n ): 

K = M + 3, L = N + 3, Li = 2, L 2 = 4, L z = L - 4. 

For the special case of PP (I) : 

K = 15, L = 4, U = 0, L 2 = 3, L 3 = 1. 

The delays from a+ to the start at b and from a_ to the stop at b 
are the same. This common delay is M + u + 8 in the general case 
and 18 in the special case. We can now reformulate the rules (6.a)- 
(6.d), with a + in place of a and a_ in place of a*, and with the common 
delay for starting and stopping. This eliminates, as observed earlier, 
the need for the special ordering of rule (6'.a). We obtain accordingly 
this. 



172 



THEORY OF SELF-REPRODUCING AUTOMATA 



The rules that govern possible corruption by interference are now 
modified from their form in (6'.a)-(6'.d) in this way: 

The special chronology for stimulations at a and a*, as de- 
fined in (6'. a), is now replaced by the ordinary chronology 
of the occurrence of stimulations at a+ and a_ . That is, all 
(9') \ comparisons must now be made according to the times at 
which stimulations occur at a+ and a_ . With this modi- 
fication (and replacing a by a+ and a* by a J) the rules of 
v (6 / .b)-(() / .d) remain valid. 

[As noted above, there is an error in von Neumann's rule (6'.c) 
concerning the timing of a periodic pulser. There is also an important 
case which von Neumann didn't discuss, namely, the case in which a 
"stop" stimulus precedes a "start" stimulus. 

Let us consider first the timing of von Neumann's periodic pulsers, 
using Figure 17 as an example. For the periodic pulser PP (10010001) 
we have the following phasing. A "start" stimulus into input a+ at 
time 2 will cause the sequence 10010001 to enter cell H4 from cell G4 
at times 2 + 31 through 2 + 38. A "stop" stimulus into input a_ at 
time 2 will cause the sequence 1111111111 to enter cell H4 from cell 
H3 at times t + 32 through t + 41. In this case nothing is emitted 
from the output b, since the killing signal from H3 dominates the 
transmission signal from G4- If the "stop" stimulus comes one moment 
later (at time 2 + 1), the sequence 1111111111 enters cell H4 from 
cell H3 at times 2 + 33 through 2 + 42. In this case cell H4 is in 
state Coo at time 2 + 31, in state Coi at time 2 + 32, in state Cio at 
time 2 + 33 (and a pulse is emitted from b at time 2 + 33), and in 
state U at time 2 + 34. 

Consider next the phasing of von Neumann's special periodic 
pulser PP(T). A "start" stimulus into input a+ at time 2 produces 
the sequence 111111 into cell E3 from cell D3 at times 2 + 16 through 
t + 21. A "stop" stimulus into input a_ at time t enters cell E3 from 
cell E2 at time 2+17. Consequently, nothing is emitted from b if 
the "start" and "stop" stimuli are simultaneous. If the "stop" 
stimulus comes one moment later, then one pulse is e mitted from 6, 
as was the case with the periodic pulser PP (10010001). 

It should be noted that we have measured the delay from the input 
a+ and a_ up to (but not including) the cell containing the confluent 
state C which drives the output b. Von Neumann measured these 
delays through this cell to the output b. For a signal from the "stop" 
input a_ the delay through this final cell is only 1 unit, since killing 
takes only 1 unit of time. Hence the delay from both a+ and a_ to 



DESIGN OF SOME BASIC ORGANS 



173 



the stop at b is 33 for PP (10010001) and 18 for von Neumann's 
PP(1). See Table I. 

Thus the phasing for all periodic pulsers PP(^ • • • i n ) constructed 
by von Neumann's algorithm is as follows. Assume the periodic 
pulser is as designed and is quiescent. If the "start" stimulus into 
input a+ and the "stop" stimulus into input a_ are simultaneous, 
nothing is emitted from output b, and the periodic pulser is left in 
its initial state. If the "stop" stimulus follows the "start" stimulus 
by T(T > 0) units of time, the sequence i l • • • i n is emitted v times 
and then an initial sequent of it of length \x is emitted, where T = 
n*> + ju, ju < n, and either v or /x may be zero. This phasing is as von 
Neumann intended and is in accord with his rules. But von Neumann 
did not consider all the cases in which the "stop" stimulus precedes 
the start stimulus (i.e., T < 0). Apparently, he planned to use each 
periodic pulser in such a way that there would be a one to one corre- 
spondence of "start" to "stop" stimuli, with the m'th "stop" stimulus 
coming no earlier than the m'th "start" stimulus, with the m'th 
"stop" stimulus preceding the (m + l)st "start" stimulus by a 
sufficient length of time for the periodic repeater to be cleared pro- 
perly. But he did not state this intention, and in his subsequent use 
of periodic pulsers he did not always conform to it. 

Consider the effect of feeding the sequence 101 into the "stop" 
input a_ of PP (10010001). This sequence will cause the upper pulser 
P (1111111111) to emit 12 (rather than 10) pulses and (depending 
on the phasing of the "start" input a+) may leave cell H4 in an 
undesired state. For some conditions of the phasing of 101 into a_ 



Table I 

External characteristics of periodic pulsers 





General Case 


Special Case 


In terms of 
parameters 


Example of 
PP(10010001) 


Von Neu- 
mann's 

PP(1) 


Alternate 
PP(T) 


Width K 


M + 3 


23 


15 


13 


Height L 


N + 3 


10 


4 


4 


Number of cells below an input or 










output 










Input a + 


2 


2 


0 


1 


Input a 


4 


4 


3 


3 


Output b 


L - 4 


6 


1 


1 


Delay from input a + to b. Also, de- 










lay from input a., to stop at 6.. 


M + u + 8 


33 


18 


19 



174 



THEORY OF SELF-REPRODUCING AUTOMATA 



and 1 into a + , the cell H5 may be killed to U. This difficulty may 
be solved by the following rule o f usage. The "stop" input a_ of a 
"general case" periodic pulser PP^ 1 • • • i n ) is never to be stimulated 
twice in any time span of length n. Actually, all of von Neumann's 
later uses of "general case" periodic pulsers conform to this rule. 

There is a more serious problem in von Neumann's "special case" 
periodic pulser PP(T). A stimulus into the "stop" input a_ changes 
cell E3 into U, and then the four pulses in the pulser P (111111 ) or the 
periodic repeater (or both) operate by the direct process to leave E3 
in the confluent state C. But suppose a "stop" stimulus precedes a 
"start" stimulus. Then cell E3 will not be left in the correct state! 
This fact causes no trouble when von Neumann's PP(1) is used in 
the triple-return counter (Sec. 3.4 below), but it does cause incorrect 
operation of his control organ CO as it is used in the read-write-erase- 
control RWEC (Sec. 4.3.4 below). We could modify some of the 
contextual circuitry of the CO, but it is better to use a PP(1) in the 
CO which is not contaminated if it is turned off before being turned 
on. 

This alternate PP(I) is shown in Figure 20. It uses two modified 
pulsers P(lllll), one above and one below. A "start" stimulus into 
input a+ at time t causes the sequence 11111 to enter cell G3 from 
cell G4 at times t + 17 through t + 21. If cell G3 is in the confluent 
state C this sequence will emerge from the output b at times t + 19 
through t + 23, so the delay from input a+ to b is 19 units of time. A 
"stop" stimulus into input a_ at time t causes the sequence 11111 
to enter cell G3 from cell G2 at times t + 18 through t + 22. Since 
killing takes only 1 unit of time, the delay from input a to the stop 
at b is 19 units, which is the way von Neumann counted it. See Table I. 

Note further that if the start input a+ is stimulated from 1 to 4 
units of time after the stop input a_ is stimulated, from 1 to 4 pulses 
will be left in the repeater .Hence we stipulate the following rule of 
usage for the alternate PP (1) of Figure 20: the stop input a_ is never 
to be stimulated twice within 5 units of time, and the start input 
a+ is never to be stimulated from 1 to 4 units of time after the stop 
input is stimulated. The structure of the read-write-erase-control 
unit RWEC is such that this rule is always satisfied. When operated 
in an environment which conforms to this rule, the alternate PP(T) 
functions as a "flip-flop." It is turned on at a+ , off at a_ , and while 
it is on it emits a continuous sequence of stimuli from b which may 
be used to operate a gate (confluent state). 

We will now summarize the external characteristics of periodic 
pulsers. Von Neumann's special case PP(1) is shown in Figure 17b. 



DESIGN OF SOME BASIC ORGANS 



175 



Our alternate special case PP(T) is shown in Figure 20. The external 
characteristics of both of these PP (1 ) are given in Table I. 

The general case PP (i l • « • i n ), with n ^ 5, is shown in Figure 19h. 
If n < 5 and the special case does not apply, the characteristic is 
iterated until n ^ 5; for example, if PP (101) is a sked for, PP (101101) 
is constructed. The general case PP^ 1 ••• i n ) contains the pulser 
P (i l - • • i n ), for which u and k are defined at the end of Section 3.2.1. 
The following string of definitio ns leads to the parameters K (width) 
and L (height) of PP(i* ... *»). 

I = integer part of (n — l)/2 

N = Maximum of u + 2 and I 

L = N + 3 

p = The smallest integer such that 5p ^ n — 1 

M = Maximum of lOp and 2k + 1 

K = M + 3. 

Further information is given in Table I.] 

3.3 The Decoding Organ: Structure, Dimensions, and Timing 

[Figure 21 shows the decoding organ D (10010001). This decoding 
organ has characteristic 10010001 and order 8. All sequences of length 
8 can be divided into two classes : those which are bitwise implied by 
(contain all the stimuli of) the characteristic 10010001 (e.g., 10010001 
and 11010011), and those which are not (e.g., 10000001 and 
10010010). If a sequence of the former class is fed into input a, a 
single stimulus will come from output b after a suitable delay, while a 
sequence of the latter class produces no output. 

A decoding organ is to be distinguished from a "recognizing" device 
which produces an output for some particular sequence (e.g., 
10010001) and no other. Such devices are discussed in Section 3.5. 

The decoder D (10010001) works in this manner. Suppose the se- 
quence iHHHHHHH 8 , with i 1 = 1, = 1, and i = 1, enters input a at 
times t through t + 7; the three stimuli i 1 , i, and i enter with relative 
delays 0, 3, and 7, respectively. Paths B and D meet at the confluent 
state Dl with delays of 21 and 18, respectively; hence i l and i will 
produce an output from cell Dl at time 23. This output will arrive at 
cell Fl at time 24, coincident with the arrival of i via path F, thereby 
producing an output from b at time 26. Note that since there are 
three ones in the characteristic 10010001, three paths (B, D, and F) 
are needed in the decoder. 

The design of a decoding organ is very similar to that of a pulser 
(Sec. 3.2.1). Indeed, a pulser is in fact a coder. For the decoding 



176 



THEORY OF SELF-REPRODUCING AUTOMATA 



organ, however, confluent states are needed along the top row to 
detect coincidence. When confluent states are needed to introduce 
relative delays between paths of odd amounts (e.g., as in cell B3 
of Fig. 21), then these confluent states must be isolated from those in 
the top row by means of T 0 i € states (as in row 2 of Fig. 21).] 

Our third construction is the decoding organ. This organ has an 
input a and an output b. The ideal form would be one which, upon 
the arrival of a prescribed sequence, say i 1 • • • i n , at a, and only then, 
will emit a single stimulus at b. However, for our specific applications 
of this organ a simpler requirement will do, and this is the one that 
we are going to use. It is as follows. Let a sequence, e.g., i l • • • i n , 
be prescribed. Upon arrival of any sequence j l • • • j n at a, which 
contains all the stimuli of i 1 • • • i n (i.e., such that i = 1 implies 
f = 1), and only then, it will emit a single stimulus at b. 

The relation between the actuating sequence j l • • • j n at a and the 
response at 6 is freely timed; i.e., the delay between these is not pre- 
scribed at this point (cf., however, the remarks at the end of this 
section). 

This decoding organ has the symbol D^' 1 • • • i n ). The sequence 
t i ... {n y which can be prescribed at will, is its characteristic, and n is 
its order. 

The required network is successively developed in Figure 22. This 
construction and the network that results from it are discussed in the 
balance of this section. They have a great deal in common with those 
for the pulser P(i x • • • i n ), as discussed in Section 3.2.1. 

Let vi , • • • , Vk be the v for which i = 1, i.e., the positions of the 
stimuli in the sequence i l • • • i n . (At this occasion, unlike in Sec. 
3.2.1, we are not interested in their monotone ordering.) Write 

/ 10 /x f n — v h = 2nh + r h \ 

UU ; \ where ^ = 0, 1, 2, • • • ; n = 0, 1. 

{Note the difference between equation (10') and equation (l')!} 

The input stimuli at a must be compared, to see whether k stimuli 
with the relative delays v i , • • • , v k are present there. This can be 
done by multiplexing each stimulus that arrives at a to k distinct 
stimuli, numbers h = 1, • • • , k, arriving at some comparison point 
b' with the relative delays n — v h (h = 1, • • • , fc), respectively. Then 
the simultaneous arrival of k stimuli at b' is equivalent to the arrival 
of k distinct stimuli, numbers h = 1, • • • , k, arriving at a with the 
relative delays vi , • • • , vu . 

Hence we need k paths, numbers h = 1, • • • , k, from a to b , which 
have the relative delays n — v h (h = 1, • • • , fc), respectively, with 



DESIGN OF SOME BASIC ORGANS 



177 



respect to each other. However, we cannot sense a /c-fold coincidence 
by a single act, at a single point, if k > 3. (k = 3 is the maximum that 
a single C can handle; cf. Sec. 2.3.2.) It is therefore best to bring 
these k paths together pairwise. First, paths 1 and 2 merge at a 
comparison point b 2 (it is better to begin with b 2 rather than with 
£>/; cf. below), so that joint path 2 continues from b 2 then paths 2' 
and 3 merge at a comparison point 6 3 , so that joint path 3' continues 
from bz\ then paths 3' and 4 merge at a comparison point &/, so 
that joint path 4' continues from &/; • • • ; and finally paths (k — 
1)' and k merge at b k ', so that joint path k! continues from b k ' directly 
to the output b. In this successive way 2-fold coincidences at b 2 \ • • • , 
bk replace the single fc-fold coincidence at b'. A C can, of course, 
handle a 2-fold coincidence without difficulty (cf. Sec. 2.3.2). 

This procedure clearly calls for a network of the type developed in 
Figures 16a and 16b. Consider first Figure 16a. Here the stimulus 
entering at a reaches b over k different paths, and while the delays 
that are desired for each one of these paths are not yet properly 
adjusted, this can be attended to afterwards, with the means of 
Figure 16b, i.e., with the networks N of Figure 16c, as detailed in 
Figures 16d-16f. We discuss first another imperfection that Figure 
16a presents in the present situation. This is the following. 

In Figure 16a the k paths that are produced by the C on the base 
line are directly merged by those o on the top line that lie above the 
C (on the base line) numbered 2, 3, • • • , k. These mergers are effected 
by o cells, i.e., without any coincidence requirements. This is confor- 
mal to the purposes of Figure 16a in Section 3.2.1 but not to our 
present ones. The k — 1 cells o> in question, where these mergers 
occur, are obviously the comparison points b 2 , bz, • • • , referred 
to above. Hence these must be able to sense coincidences; i.e., they 
should be C, not jdi. For reasons of symmetry we also replace the 
o on the top line above the C (on the base line) numbered 1, by C, 
and call it &/. This new arrangement is shown in Figure 22a. 

A stimulus arriving at a has k paths available to reach 6. On path 
h (h = 1, - " , k) it goes horizontally from a to the (base line) C 
numbered h; then it turns vertically up, ascends to the top line (to 
the C at bh) and continues there horizontally to 6. (Up to this point, 
and not further, we are following the pattern of Fig. 16a.) These are 
(2h - 1) + u + (2(fc - h) + 1) = 2k + u steps, but since h + 
(fc — h + 1) = k + 1 of them are C's, the entire delay involved is 
8k + u + 1. That is, the delay is the same on all paths. 

In view of this we must increase the delay on path number h by 
n — vh = 2/x/ + r h '. {Cf. equation (10').} This can be achieved 



178 



THEORY OF SELF-REPRODUCING AUTOMATA 



exactly as was its analog in Section 3.2.1: each vertical branch is 
replaced by a suitable delay network, say branch number h by a 
network Nh. This Nh is like the Nh of Figure 16e; that is, it is made 
up of the parts shown in Figures 16d-16f, except that now , Th are 
replaced by /x/> n . Thus Figure 22b results, which is related to Figure 
22a in the same way that Figure 16b is related to Figure 16a. The 
height u obtains by repetition of the considerations of Section 3.2.1 
that led up to condition (2 ). {Now nh , r h are replaced by /x/, r/, 
i.e., the v h — h of (l') are replaced by the n — v h oi equation (10').} 
We obtain the condition u ^ u° (in analogy to condition (2'), with 
the modifications mentioned above), where 



(11') 



u° = Max {n — Vh) + e'°, 

where e'° = 1 if the Max is even but some n — Vh is odd, 
and e° = 0 otherwise. 

/o 



It is, of course, simplest to put u = u 

[Von Neumann overlooked the need for an extra row of cells 
between row u of Figure 22a and the top row in case row u contains a 
confluent state. Otherwise, the confluent state of row u will be adja- 
cent to a confluent state in the top row, creating an open path, since a 
confluent state cannot directly drive another confluent state. Row 2 
of Figure 21 is this extra row which von Neumann overlooked; if 
row 2 were deleted, the confluent states of cells Bl and B3 would be 
adjacent. 

This oversight may be remedied without changing von Neumann's 
parameters as follows. Let 1 ^ v± < vi < • • • < r k < n. Then there 
are two cases, according to whether an extra confluent state occurs 
in the leftmost path (i.e., the path for vi) or another path. If an extra 
confluent state occurs in the leftmost path, replace it and the con- 
fluent state above it (i.e., in the top row) by a block of four connected 
ordinary transmission states. Thus in Figure 21, put a block of four 
ordinary transmission states in cells Al, Bl, A3 and B3 and delete 
row 2. If an extra confluent state occurs in a path other than the 
leftmost path, place it one cell lower.] 

This completes the construction. As Figure 22b shows, the area of 
this network has the width 2k and the height u + 2. An abbreviated 
representation of this network is given in Figure 22c. 

The delay from a to b through Figure 22a was 3k + u + 1 on each 
path. Through Figure 22b it is therefore 1 + (3k + u + 1) + (n — 
Vh) on path number h. The first term, 1, is due to the insertion of the 
o before the first C in Figure 22a. Hence stimulus number v h (with 



DESIGN OF SOME BASIC ORGANS 



179 



i n = 1 ) in the sequence i 1 • • • i n at a reaches b with a delay (3k + 
u + 2) + n. This is counted from the time immediately before the 
start of the sequence i l • • • i n ; hence the delay counted exactly from 
its start is 3k + u + n + 1. This is the same for all h = 1, • • • , k, 
as it should be. Hence the stimulus indicating this A;-fold coincidence 
(i.e., the presence of a sequence j l • • • j n , containing i 1 • • • i n , at a; 
cf. above) will appear at b with a delay 3k + u + n + 1. Thus, 
in the final arrangement, represented by Figure 22c, there is from the 
arrival of a sequence/ • • • j n (containing i 1 • • • i n ) a delay 3k + u + 
n + 1 to the response at b. 

It is easily seen that in this network no corruption by interference 
occurs. That is, whatever stimuli may arrive at a, whenever there 
occurs a sequence j l • • • j n among them that contains i l • • • i n , there 
will be a response stimulus at b, with a delay 3k + u + n from the 
beginning of that sequence. 

[ We will summarize the external characteristics of the decoding 
organ D^' 1 • - • i n ). See Figure 22c. 

The width of the pulser is 2k, where k is the number of l's in the 
characteristic i l • • • i n . Von Neumann implicitly assumed that k ^ 
2; for k = 0 and k = 1 no organ is needed. 

The height of the decoding organ is u + 2, where u is defined as 
follows. The vi , • • • , v k are the v for which i = 1. Note that n is the 
length of the characteristic and the v x is the superscript of the first 
1 ; hence n — vi is the number of bits in the characteristic which are 
to the right of the first 1. Since all vk (h = 1, • • • , k) are positive, 
Max (n — v h ) = n — v\. Von Neumann's rule for u then becomes 

u = (n — vi) + where 

e'° = 1 if (n — vi) is even but some n — vh (h 2, • • • , k) is odd, 

e'° = 0 otherwise. 

The delay between the input signal i l entering input a and the out- 
put pulse leaving b is 3k + u + n + 1.] 

3.4 The Triple -return Counter 

[ In the preceding sections von Neumann gave algorithms for de- 
sign ing an a rbitrary pulser P^ 1 • • • i n ), an arbitrary periodic pulser 
PP^' 1 • • • i n ), and an arbitrary decoder Q(# • • • i n ). He next 
designed a specific organ, the triple-return counter The completed 
organ is shown in Figure 23, though not to scale. The periodic pulsers 
PP(T) are those of Figure 17, with width 15 and height 4; hence the 
actual width of $ is 24 and its actual height is 26. The long lines with 



180 



THEORY OF SELF-REPRODUCING AUTOMATA 



arrows in Figure 23 symbolize sequences of ordinary transmission 
states being used for transmission only, not for disjunction. 

Von Neumann needed the triple-return counter for a specific pur- 
pose, that of sending a pulse around a connecting loop (Ci or C 2 ) of 
the external tape L three times; see Figure 37. Suppose that the 
secondary output d of <£> is connected to the input t> 2 of connecting 
loop C 2 and that the output w 2 of loop C 2 is connected to the second- 
ary input c of <£. A pulse into the primary input a of $ will go around 
loop C 2 three times and will then be emitted from the primary output 
6 of 

We assume as a rule of usage that once input a of $ is stimulated, it 
will not be stimulated again until a stimulus has been emitted from 
output b of Under this assumption, the triple-return counter <£ 
works like this. A start stimulus into a goes via a coding and decoding 
network to a + *, where it starts the first periodic pulser, and also to 
output d, where it is relayed to the input of C 2 . After a delay, the 
output 6 1 of the first periodic pulser will send stimuli along row 14 to 
impinge on confluent cell F14, which functions as a gate. When the 
output pulse from C 2 enters <£ at c, it travels along row 20 and into 
column D, stimulating the three gates F14, F8, and F2. Only the 
first of these gates (F14) is open; the pulse passes through this gate, 
turns the first periodic pulser off, turns the second periodic pulser 
on, and passes along row 13 and column J to the secondary output d. 

The next pulse entering secondary input c from C 2 turns off the 
second pulser, turns on the third pulser, and goes from secondary 
output d back to C 2 . When this pulse returns from C 2 , it turns off 
the third pulser and passes out the primary output b. This completes 
the operation of <f>. 

Note that the path from primary input a to cell F18 crosses the 
path from secondary input c to cell D14> Actually, no harm results 
from the primary input stimulus going to cell D14 and thence to 
gates F14, F8, and F2, since these gates are initially closed. But if 
the pulses from c to D14 entered cell F18, a malfunction would result. 
This is a special case of the general problem of wire-crossing. The 
general problem is solved by means of the coded channel of Section 
3.6 below. The special problem is solved in <£ by means of the coder 
B17, B18, C17, C18, which produces 101 when stimulated, and the 
decoder D18, D19, E18, E19 9 F18, F19, which emits a 1 to F17 and 
G18 when it is stimulated by 101, but not when it is stimulated by 1. 

The design principle of the triple-return counter can easily be modi- 
fied and generalized to give a counter which will count m pulses; that 
is, will emit every m J th pulse it receives.] 



DESIGN OF SOME BASIC ORGANS 



181 



The three organs that we constructed in Sections 3.2 and 3.3 were 
basic entities of rather general significance. The next ones will be 
much more special; they correspond to specific needs that will arise 
in the course of the first major, composite construction that we must 
undertake. Their names, which refer to very special functions, also 
express this fact. 

The first organ in this series is the triple-return counter. This organ 
has two inputs a, c and two outputs 6, d; a, b are the primary input- 
output pair and c, d are the secondary input-output pair. 

In order to describe its functioning, it is necessary to assume that 
its secondary output d and its secondary input c are attached to the 
input c* and the output d*, respectively, of an (arbitrarily given) 
other organ. This other organ is the responding organ, and we will, 
for the time being, give it the symbol 12. Having been thus attached 
to 12, the triple-return organ now has only its primary input-output 
pair a, b free. Its desired functioning is as follows. 

Upon stimulation at a it responds at d and thereby stimulates at 
c*. Assume that 12 responds, after a suitable delay, at rf*, stimulating 
c. This causes a second response at d, and hence a stimulation of 12 
at c*. Assume that 12 responds for the second time, after a suitable de- 
lay, at d*, stimulating at c. This causes a third response at rf, and 
hence a stimulation of 12 at c*. Assume that 12 responds at d* for the 
third time, after a suitable delay, stimulating at c. This causes a 
response at b and terminates the process. 

The relation between the original actuation at a and the ultimate 
response at b is freely timed; i.e., the delay between these is not 
prescribed at this point. Note that the total process from a to b has 
three phases, namely the three passages from d through c* and d* 
(i.e., through 12) to c, whose delays depend in any event on the re- 
sponding organ 12, and not on the organ to be constructed now. How- 
ever, the other phases of the total process from a to b (i.e., those from 
a to d, twice from c to d, and finally from c to b) depend solely on the 
organ to be constructed now. It is to these that the above observation 
about free timing, i.e., the absence of prescriptions of specific delay 
lengths, applies (cf., however, the remarks at the end of this sec- 
tion). 

The triple-return counter has the symbol <i>. 

The required network is successively developed in Figure 24. This 
construction and the network that results from it are discussed in the 
balance of this section. 

We require that the stimulation should pass precisely three times 
from c* to d* (i.e., through 12). Or, which is more to the point, we 



182 



THEORY OF SELF-REPRODUCING AUTOMATA 



require that when a stimulus arrives at c (from d* of 12) the first 
two times, the consequences (in <i>) should be different from those 
that take place the third time. Hence <£ needs a memory that can 
distinguish the first two occasions from the third one, i.e., keep a 
count up to three. 

With ourpresent means this is best achieved by three periodic 
pulsers PP(T), each of which is turned on (started) at the beginning 
of the count-period that it represents, and turned off (stopped) at 
its end. During that count-period, the corresponding PP(T) effec- 
tively indicates the presence of the count-period by the (continuous) 
availability of its output stimulus. This stimulus must then be used 
to achieve the characteristic operations of the count-period in ques- 
tion. These are the following. 

First count-period: A stimulus arriving at c is routed to d, and 
after this the PP(1) of the first period is turned off and that of the 
second period is turned on. 

Second count-period: A stimulus arriving at c is routed to d, and 
after this the PP(1) of the second period is turned off and that of 
the third period is turned on. 

Third count-period: A stimulus arriving at c is routed to 6, and 
after this the PP (T) of the third period is turned off. 

The initial stimulus at a must, of course, be routed to c, and it 
must also turn on the PP(1) of the first period. 

The obvious way in which the continuous emissions of a PP(T) 
can be used to induce the desired consequences that are characteristic 
of its count-period (cf. above), with the help of the stimulus at d 
that actuates the period in question, is this. Use a confluent state C 
as a coincidence organ by giving it the route from the output of the 
PP(T) referred to above and the route from c as inputs (whose 
coincident stimulations it is to sense). Then use this C to initiate 
the characteristic responses enumerated above. [See cells F14, F8, 
and F2 of Figure 23. Von Neumann thinks of the stimulus at d (i.e., 
c*) as actuating a given count-period. This stimulus will pass through 
Q, and enter <I> at c (i.e., d*).] 

All these arrangements are shown in Figure 24a. Note that in this 
figure the inputs and the output of the PP are designated by d+ , d- and 
b (instead of a + , a- and 6, as in Figs. 19g and 19h). The upper and the 
lower edges of the PP are protected by U's, to prevent unwanted 
stimulations by C's across these boundaries. It is convenient not to 
extend this protection to the left lower corner of each PP (to make 
room for a transmission channel there); that this is permissible is 



DESIGN OF SOME BASIC ORGANS 



183 



not indicated by Figure 19h, but it is by Figure 19g. [See also Figures 
16b and 18k.] 

In Figures 24a and 24b the following simplifications are used: A 
straight line of states 0. is designated by a single arrow, and similarly 
for the three other directions. A connected area of states U is cross- 
hatched. The PP (T) are not shown in their true size (which is 4 X 15; 
cf. Fig. 19g); the area of the network of Figure 24a is therefore 
correspondingly contracted horizontally and vertically. 

Figure 24a does not show how the transmission channels of the 
secondary input-output pair c, d and of the primary input a reach 
their proper endings at c = d*, d = c* and a, respectively. (In the 
figure c = d*, d = c* are shown; a is not shown but should be thought 
of as lying towards the lower left corner.) In the figure the transmis- 
sion channels in question end at Ci or c 2 , at d', and at a/ or a 2 ', 
respectively. The primary output b is shown reaching its proper 
ending. There is no difficulty in connecting d with d, but there is a 
serious topological difficulty in connecting Ci or c 2 with c and at the 
same time a/ or a* with a (without disturbing the connections of b 
and of d in the process) : It is clear that these paths must cross. Note 
that it is not a rigid requirement that a, b, c, d be at the positions 
indicated in the figure (regarding a, cf. the above remark); i.e., 
a, b, c, d could be moved around somewhat. However, it is necessary 
that a, b be on the left edge of the network and that c, d be on its 
right edge (cf. later [Fig. 39]), and this implies the topological diffi- 
culty referred to above, i.e., the necessity of crossing paths. We may 
therefore just as well leave a, 6, c, d in the positions indicated above. 
We must solve the problem of how to cross the channel from c% or 
c 2 f to c and that one from a/ or a 2 ' to a without corruption of informa- 
tion by interference, i.e., without the information intended for one 
channel getting into the other one, and thereby causing malfunctions 
in the network. 

The problem of crossing lines (which is a peculiarity of the 2- 
dimensional case; cf. Chapter 1 and also later) will be solved in a 
general form by the construction of a specific organ for this purpose 
later on [Sec. 3.6]. However, its present occurrence is particularly 
simple, and it does therefore not seem worthwhile to appeal to the 
general procedure referred to, but it is preferable to solve it ad hoc. 

The main simplification present is that, while it is necessary to 
keep signals in the channel from c to Ci or c 2 ' from getting into the 
channel from a to a/ or a 2 ', the reverse is not necessary. Indeed, the 
first type of cross-talk between these channels would cause a pulse 
to cycle indefinitely between the responding organ ft and the triple- 



184 



THEORY OF SELF-REPRODUCING AUTOMATA 



return counter <i>, thereby completely vitiating the operation. The 
second type of cross-talk, on the other hand, merely injects the 
original stimulus (from a) into the Ci -c 2 channel, thereby furnishing 
a stimulus to each one of the three coincidence-sensing C's (these 
are 2 squares to the right of the d channel at the left edge of the 
figure). An examination of the delays (cf. below) will show that at 
this time none of the PP(l) is turned on yet, and therefore none of 
the C's in question receives the other stimulus required for a coinci- 
dence. Hence, the misdirected a-stimuli of this class are harmless. 

Thus, we must only prevent c-stimuli from getting into the a/-a 2 ' 
channel. This can be achieved by the same coding-and-decoding 
trick that we will use subsequently for the general purpose referred 
to above [see Sec. 3.6]. That is, we will use a pulser to replace an 
a-stimulus by a (coded) sequence of stimuli, and protect ai'-a 2 ' by 
placing a decoder in front of it which will respond only to this sequence 
of stimuli and not to a single stimulus. In this way an a-stimulus, in 
its expanded (coded) form, will be able to get through to ai'-a 2 '; 
while a c-stimulus, which remains single, will not be able to reach 
ai'-a/. In addition, it is not worthwhile to appeal at this point to our 
previous constructions of pulsers and decoders, but it is simplest to 
make up the necessary primitive organs specifically for the present 
purpose. 

The coding of an a-stimulus need only be its replacement by two 
stimuli, and it is somewhat more convenient to choose these as not 
immediately consecutive. That is, we are using 101 rather than IT. 
This makes the coding, as well as the decoding operation quite simple. 
The arrangements that achieve this are shown in Figure 24b. (Note 
that Fig. 24b extends the lower end of Fig. 24a. Inspection of the 
two figures shows clearly what their common elements are, i.e., in 
what manner they must be superposed.) The position of a can be at 
ai or a 2 . This stimulates the leftmost C which sends stimuli over 
two paths with a differential delay of 2 to its right-hand neighbor jo . 
Thus a sequence 101 arrives here, if the stimulation came from ai~a 2 . 
Note that this o is also fed by a to immediately under it, which 
represents the channel coming from c. Through this channel only a 
single 1 will come in. Thus, the coding of an ai-a 2 stimulus (and the 
non-coding of a c-stimulus) has already taken place at this o (imme- 
diately to the right of the leftmost C). This stimulates the second C. 
From here this stimulation travels, as it should, unimpeded to c 2 ' 
(whether coded or not). On the other hand, two paths, with the 
differential delay 2, go from this (the second from the left) C to the 
next (the third from the left) C, which now acts as a coincidence 



DESIGN OF SOME BASIC ORGANS 



185 



organ. That is, this last C will respond only if the previous C had 
accepted two stimuli with the compensating delay 2, i.e., a sequence 
101. In other words, only a stimulus coming from ai-a 2 (and not 
one coming from c) can effect (by these indirect means) a stimulation 
of this last C. This C coincides with that one in the lower left corner 
of Figure 24a, i.e., with the one adjacent to the a\-(U entries. Thus, 
the transmission from ai-a 2 together with the exclusion from c, to 
a/-a 2 ', has been achieved, as desired. 

Note that a/ and a/ have both been utilized (their C needed two 
accesses; cf. above), a may be at a\ or a 2 . We use ai , since we want a 
from the left. Of c/ and c 2 ' we have utilized c/, and Ci need not be 
considered further. Figures 24a and 24b having been fitted together, 
it is indicated to make a further addition to the left edge. Indeed, it 
is desirable to straighten it, and to protect its C's (which could cause 
unwanted stimulations in adjacent transmission states) by a border 
of IPs. We add accordingly a border of IPs which fits the left edge 
of Figures 24a and 24b and contains the necessary channels from the 
new position of a to the old one, and from the new position of b to 
the old one. This is shown in Figure 24c, the position relatively to 
Figure 24b being indicated by the dashed line there. Finally, we fill 
the strip around the right upper corner of Figure 24a, marked with a 
dashed line, with IPs, to complete the rectangular shape. 

This completes the construction. The areas PP(T) in Figure 24a 
have the width 15 and the height 4 according to Figure 19g. Hence 
the area of the network of Figure 24a spans a rectangle of width 21 
and of height 26. This has the consequence that the (rectangular) 
area of the entire network (resulting from Figs. 24a-c) has the width 
24 and the height 26. An abbreviated representation of this network 
is given in Figure 24d. Figures 24c and 24d are further contracted in 
comparison with Figures 24a and 24b. 

The delay from a to ai is clearly 1 (cf. Fig. 24c); the delay from 
ai (through three C's, and paying attention to the fact that because 
of the coding-decoding procedure used, the shorter path between the 
two first C's must be combined with the longer one between the two 
last C's, or vice versa, the same delay resulting in both ways) to a + 
is 11 (cf. Fig. 24b for this and for several estimates that follow). 
This is also the delay to the output of the io immediately to the right 
of the last C, i.e., to the entrance of the straight channel leading to 
d. The delay through that channel to d (taking the true length of 
PP(T) into consideration, which is 15 according to Fig. 19g) is 18. 
The delay from a+ through PP (1 ) to b is 18 (cf . the end of Sec. 3.2.1 ) . 
The delay from b to the first coincidence-C (2 squares to the right of 



186 



THEORY OF SELF-REPRODUCING AUTOMATA 



C at C1-C2 in_Fig. 24a; it must again be remembered that the true 
length of PP(T) is 15 and that the true distance of b from the top of 
PP(T) is 2; cf. Fig. 19g) is 22. Thus, the total delay from a to the 
first stimulation of £2 at d = c* is 1 + 11 + 18 = 30. Furthermore, 
the total delay from a to the moment when the first coincidence-C 
becomes passable for stimuli coming from c/-c 2 ' (because it is being 
stimulated from the first PP(1); cf. Fig. 24a) is 1 + 11 + 18 + 
22 = 52. 

Note that it takes a pulse from ai that gets from the second C 
from the left into the Ci -c 2 channel, a delay of 5 or of 7 (depending 
on which of the two available paths between the two first C's is 
followed; cf. Fig. 24b) to get there (i.e., to emerge on the upper side 
of the second C). The delay from here to c 2 ' is 5 (since the true 
height of PP(T) is 4; cf. Fig. 19g), and from there to the first coinci- 
dence-C (cf. Fig. 24a) is 3. Hence, the total delay from a to the first 
coincidence-C through this illegitimate channel, is 1+ 5 + 5 + 3 = 
14, or 1 + 7 + 5 + 3 = 16. This delay increases by 9 for the second 
coincidence-C, and by 8 more for the third one. Hence it is at most 
16 + 9 = 25 and 25 + 8 = 33 for these C, i.e., at most 33 overall. 
Now we saw that even the first coincidence-C becomes passable for 
stimulation from the Ci-c 2 channel only with a delay of 52 after the 
stimulation at a. (For the other coincidence-C this delay is, of course, 
still longer.) Hence this illegitimate stimulation comes too soon to 
cause trouble, confirming our earlier assertion to this effect. 

[Von Neumann next considered the i'th response of £2 at c = d*, 
for i = 1, 2, 3. He calculated the delays through $ for each response 
and made sure that the internal timing was correct. The crucial point 
about timing is best explained in connection with Figure 23. Consider 
the pulse which constitutes the first response of 12; let t be the time 
it arrives at cell D14. At time t gate F14 is open and gates F8 and 
F2 are closed. This pulse will pass through gate F14 and, among 
other things, enter input a + 2 (at time t + 9) to start the second 
periodic pulser, which will in turn open gate F8. Will gate F8 be 
opened in time to let the original pulse through to start the third 
periodic pulser, thereby creating a malfunction? A calculation of the 
delays along the two paths to cell F8 shows that this will not happen: 
the original pulse reaches F8 from the left at time t + 12, while the 
gating pulse reaches F8 from the right at time t + 49. A similar 
calculation shows that there is no trouble at gate F2.\ 

This completes the discussion of the delays within $ and the proof 
of the consistency of its functioning. We restate those of <£'s delay 
characteristics that are externally relevant: from the stimulation at 



DESIGN OF SOME BASIC ORGANS 



187 



a to the first stimulation of £2: 30; from the first response of 12 to the 
second stimulation of 12: 66; from the second response of 12 to the 
third stimulation of 12: 83; from the third response of 12 to the output 
at b (Fig. 24d):59. 

Let the delay from z'th stimulation of 12 to its i'th response be 
Wi (i = 1, 2, 3). (This is a property of 12, not of <£!) Then the total 
delay from a through <£ (and three times through 12!) to 6 is 30 + 
w x + 66 + w 2 + 83 + w z + 59 = 238 + w x + w 2 + w z . 

3.5 The 1 vs. 10101 Discriminator: Structure, 
Dimensions, and Timing 

The next organ in this series is the I vs. 10101 discriminator. This 
organ has an input a and two outputs b, c. It performs a function 
that we have consciously refrained from postulating for the decoder 
(cf. the beginning of Sec. 3.3) : it discriminates between two sequences, 
one of which is part of the other. To be specific, the arrival of a single 
stimulus at a causes an emission at 6, provided that it is preceded by 
a no-stimulus sequence of sufficient length. The arrival of a sequence 
10101 at a causes an emission at c (but not at 6; i.e., it paralyzes the 
effect of the single stimulus that it contains; cf. above). . . . 

The relation between the actuating sequences 1 (or rather 0 • • • 01; 
cf. above) and 10101 at a and the response at 6 and c is freely timed; 
i.e., the delay between these is not prescribed at this point. . . . 

The 1 vs. 10101 discriminator has the symbol ^r. 

[ Von Neumann next developed his network for For a reason to be 
explained, we have replaced his design of ^ by a simpler one. 

The discriminator^ is used in reading an arbitrary cell x n of the 
infinite linear array L (Fig. 37). "Zero" is represented in cell x n by 
the unexcitable state U, and "one" is represented by the quiescent 
but excitable state T030 , which is an ordinary transmission state 
directed downward. Cell x n is read by sending the sequence 10101 
to input Vi of connecting loop Ci and noting whether 1 or 10101 
emerges from output Wi of Ci . The sequence 10101 travels down the 
upper half of Ci and enters cell x n . If cell x n is in state U, the sequence 
1010 converts x n into state T030 , and the remaining T passes through 
x n and travels down the lower half of Ci to output Wi . If x n is already 
in state T030 , the complete sequence 10101 travels around Ci and 
comes out at W\ . Hence a I at Wi represents a "zero" at x n , while a 
10101 at wi represents a "one" at x n . The T vs. 10101 discriminator 
^ discriminates between these two cases. 

At the time von Neumann designed his discriminator he did not 
know what other sequences might be fed into it. As the discriminator 



188 



THEORY OF SELF-REPRODUCING AUTOMATA 



is actually used in the read-write-erase unit RWE (Figs. 37 and 39), 
no other case arises. By taking advantage of this knowledge, and 
improving von Neumann's design in another respect, we can greatly 
simplify both his discussion and his design. For this reason we replace 
von Neumann's discriminator by Figure 25. 

Assume as a rule of usage that is normally quiescent, but that on 
occasion either the sequence 10000 or the sequence 10101 enters 
input a, starting at time t, and that once one of these two sequences 
enters input a, no further stimuli enter input a untile is quiescent 
again. 

We will consider the two cases separately. In the first case, 1 enters 
input a at time t, travels along the path indicated by the long arrow, 
and is emitted from b at time t + 40. This 1 also enters the decoder 
D (10101), but it dies there. 

In the second case the sequence 10101 enters a at times t through 
t + 4, producing two immediate effects. First, the sequence 10101 is 
decoded by D (10101), so that a single pulse emerges from output 
b 1 at time t + 21. Second, the sequence travels along the path from 
input a to output b, entering cell J 14 at times t + 38 through t + 42. 
This sequence would later be emitted from output 6, except that it is 
blocked by the killing action from cell J 13 in the following way. The 
pulse from b 1 enters a at time t + 24, so P (11111) sends stimuli into 
cell J 14 at times t + 39 through t + 43. This means that any pulse 
entering cell J 14 from cell 114 during times t + 38 to t + 43 inclusive 
is lost. Since the sequence 10101 enters cell J 14 from the left at times 
t + 38 through t + 42, it is destroyed and there is no output from 6. 
The pulse from b 1 at time t + 21 is emitted from output c at time 
t + 25. 

Combining both cases we have, for the discriminator of Figure 
25 : if 10000 enters input a at times t through t + 4, a stimulus emerges 
from output b at time t + 40, and nothing comes from output c; while 
if 10101 enters input a at times t through t + 4, a stimulus emerges 
from output c at time t + 25, and nothing comes from output b. 
Hence performs the required discrimination. 

We now summarize the characteristics of the discriminator ^ of 
Figure 25 and von Neumann's design, putting the parameters for his 
design in parentheses. The 1 vs. 10101 discriminator^ has width 10 
(22), height 14 (20); the input a is 8 (1) cells above the bottom, the 
output 6 is 0 (6) cells above the bottom, and output c is 13 (18) cells 
above the bottom. The delay from input a to output b is 40 (86), 
and the delay from input a to output c is 25 (49). It is possible to 
make a much smaller 1 vs. 10101 discriminator than Figure 25, but 



DESIGN OF SOME BASIC ORGANS 



189 



Figure 25 is in the spirit of von Neumann's design and is quite satis- 
factory for our purposes. 

Discriminating between the two sequences 1 and 10101 is a special 
case of the general task of discriminating among binary sequences. 
Another instance of this general task occurs in the coded channel of 
the next section. Von Neumann solved the problem there by using 
sets of sequences such that no sequence bitwise implies (is covered by) 
any other, even if the sequences are shifted in time relative to one 
another. The sequences 1011, 1101, 1110 constitute such a set. For 
the sake of completeness we note that the general task of discriminat- 
ing among binary sequences could be accomplished by a unit which 
"recognized" a given sequence and no other. An example of a recog- 
nizer is given in Figure 26. 

The recognizer R (101001) of Figure 26 performs the following 
function. Suppose that at (relative) times 0 through 5 a sequence 
iHHHHH 6 enters input a, and assume that this sequence is both pre- 
ceded and followed by several zeros. Under these circumstances the 
recognizer R (101001) emits a stimulus from output b at time 48 if 
and only if the entering sequence is 101001 (i.e., i 1 , i, i 6 are 1 and 
i 2 , i, i are 0). We will explain how the recognizer R (101001) accom- 
plishes its purpose. The general principle (algorithm) for designing 
an arbitrary recognizer R(iH' 2 • • • i n ) will be evident at the end of 
this explanation. 

The following conditions obtain. 

(I) The decoder D (101001) emits a pulse from output b 1 at time 
23 if and only if i 1 , f, i 6 are all 1. 

(II) The pulser P(1101) emits a pulse from output b 2 at time 23 if 
and only if i 2 or i or i is 1 . 

There are three cases to consider. 

(A) First case: the input sequence is 101001, i.e., i 1 , i 3 , i 6 are 1 and 
i 2 , i, i are 0. Nothing comes from b 2 or 6 3 , and the pulse from 
output b 1 is emitted from output b at time 48. 

(B) Second case: i 1 , i, i 6 are 1 and one or more of i 2 , i, i are 1. 
Pulses are emitted from both b 1 and b 2 at time 23. These pulses 
enter the confluent state 15 at time 29, causing a pulse to enter 
input a at time 31. The pulser P (11111) emits the sequence 
11111, which enters cell Tl at times 47 through 51 inclusive. 
The pulse from b 1 at time 23 zig-zags along rows 1 and 2 y entering 
cell Tl at time 46, and is destroyed by the killing action of 
11111 into cell Tl. Hence nothing is emitted. 

(C) Third case: not all of i\ i, % are 1. No pulse is emitted from 



190 



THEORY OF SELF-REPRODUCING AUTOMATA 



output 6 1 , and hence none from output 6. If a pulse is emitted 
from 6 2 , it is blocked at confluent state 15. 
This concludes our discussion of the recognizer R (101001).] 

3.6 The Coded Channel 

3.6.1 Structure, dimensions, and timing of the coded channel: 
[In 3-dimensional space, wires can cross one another without 
intersecting, that is, without making any contact which transfers 
information from one to the other. In 2-dimensional space it is 
topologically necessary for communication channels to intersect, and 
so there is a problem of sending information down a channel in such 
a way that it does not appear in intersecting channels. This problem 
could be solved by adding a wire-crossing primitive. Such a wire- 
crossing primitive would itself involve extra states, and it would 
necessitate additional sensitized states for the direct (construction) 
process. Von Neumann solved the problem in his 29-state cellular 
system by means of a "coded channel." 

Figure 27 shows an example of a coded channel constructed in 
accordance with von Neumann's algorithm. There are inputs ai , a 2 , 
a 3 and outputs 61,62,63; each input a t - is associated with the corre- 
sponding output (or outputs) hi . Thus a pulse into input a 2 will 
eventually appear at both 6 2 outputs (not simultaneously) and 
nowhere else. The coded channel is made up of seven pulsers and 
seven decoding organs (all shown in reduced size), together with a 
"main channel" running from the output of P (10011) to the input 
of D(llOOi). 

The coding is done with six sequences of length five such that none 
of these sequences bitwise implies (is covered by) any other. The 
sequences 11100, 11010, 11001, 10110, 10101, 10011 are associated 
with ai , a 2 , a,z , 61 , 6 2 , 63 , in that order. The way the sequences 
operate is best explained by means of an example. Suppose input a 2 
is stimulated. This will cause pulser P (11010) to inject its charac- 
teristic 11010 into the main channel. This sequence will travel to the 
end of the main channel, but because it is distinct from every other 
sequence used it will affect only decoder D (11010). D (11010) will 
then send a pulse to pulser P (10101), which will inject its charac- 
teristics 10101 into the main channel. The sequence 10101 will travel 
to the end of the main channel, but because it differs from every other 
sequence used it will affect only the two decoders D (10101), both 
of which will emit pulses from their outputs 6 2 . 

The inputs and outputs of the coded channel may be positioned in 
any order. It is because of this that two sequences are associated 



DESIGN OF SOME BASIC ORGANS 



191 



with each input-output pair, the conversion from one sequence to its 
mate taking place at the top of Figure 27. 

The inputs to the coded channel must be spaced sufficiently far 
apart in time to avoid corruption or cross talk. For suppose a x and 
a 2 were stimulated so their outputs 11100 and 11010 followed each 
other immediately in the main channel. The combined sequence 
1110011010 contains the sequence 11001 which is assigned to input 
a 3 , and hence which would operate D (11001) and eventually cause 
an output at 63 .] 

Our third construction in this series is the coded channel. Up to 
now it has been our policy to give an exact and exhaustive descrip- 
tion and discussion of each organ that we had in mind. In the present 
case, however, it is preferable to depart from this principle; i.e., it is 
much simpler and quite adequate for our purposes to proceed in a 
more heuristic fashion. We will therefore discuss more broadly what 
the need is that we wish to satisfy at this point, and by what means 
we can do this. We will then deal with the essential, prototype special 
cases, and develop a sufficient discussion, so that in our actual subse- 
quent application (cf. later) the specific organ that is needed will 
emerge with little effort. 

The coded channel performs a function which is necessitated by 
the peculiar narrowness of 2-dimensional space. Indeed, a logical 
network may easily require crossing of lines in a manner which cannot 
be accommodated in 2-dimensional space. For example, if five points 
a, 6, c, d, e are given and every one of them is to be connected to every 
other, this cannot be done in 2 dimensions without intersections, as 
schematically shown in Figure 28a. The fine connecting c to e is 
missing in this figure and cannot be inserted without an intersection. 
An actual instance of interconnections requiring intersections occurred 
during the construction of the triple-return counter (cf. the middle 
part of Section 3.4) in connection with the development of Figure 
24b from Figure 24a. 

It should be noted that this difficulty does not arise in 3 -dimensional 
space. However, since all other phases of our constructions can be 
carried out in 2 dimensions, just as well as in 3, it is worthwhile to 
keep the dimensionality to the former, low value, if this is otherwise 
possible. For this reason we accept the subordinate difficulty of line- 
crossing in 2 dimensions and the necessity of special constructions to 
overcome it. 

The obvious procedure would be to construct a special organ which 
performs the elementary act of line-crossing, as schematically shown 
in Figure 28b. A more detailed representation of this is given in Figure 



192 



THEORY OF SELF-REPRODUCING AUTOMATA 



28d : This organ has two inputs a, b and two outputs c, d. It is desired 
that a stimulation of input a elicit (with a suitable delay) a response 
at output d, and a stimulation at input b (again with a suitable, and 
possibly different delay) a response at output c. Note that the actual 
cyclical arrangement of a, 6, c, d is essential: if they are arranged as 
shown in Figure 28e — or as shown schematically in Figure 28c — there 
would obviously be no difficulty whatever. 

It is preferable to aim immediately at somewhat more than this. 
Indeed, if we only constructed organs as indicated in Figure 28d 
which effect a single line-crossing, we would have to combine large 
numbers of these in our subsequent constructions. This would lead 
to quite awkward geometrical configurations and make our general 
design operations rather inconvenient. It is therefore better to con- 
struct a multiple line-crossing organ, in the sense that we will describe 
more fully below. 

A multiple line-crossing organ that would be universally useful may 
be described as follows. It is a rectangular area A, as shown in Figure 
28f, on whose periphery there are various inputs and outputs. These 
inputs and outputs are shown in Figure 28f, and in Figures 28g-28k 
also, as bars. Every input is designated by a symbol a v , v = 1, • • • , n; 
every output is designated by a symbol b v , v = 1, ■ • • , n. There may 
be several a v with the same v, and there may be several b v with the 
same v. The order in which all of these a v and b v are arranged around 
the periphery of A may be prescribed in any manner whatsoever. It 
is desired that upon stimulation of an a v (i.e., of any one of the — 
possibly several — a/s with the v under consideration) all the 6/s 
(with the same v) should respond, with appropriate (and not neces- 
sarily equal) delays. 

However, for the purposes for which this device will be needed, a 
slightly weaker requirement will do equally well, and this requirement 
will be somewhat easier to implement, as will appear below. This 
weakening of the requirement is as follows. We define a certain 
cyclical sense of the periphery of A, as shown by the arrows in Figure 
28g. We then break this cyclical arrangement; i.e., we change it into 
a linear one, by cutting it at the point designated by p — i.e., it is 
now thought to begin at p, circle the periphery of A in the direction 
of the arrows and end again at p. We now require that the stimulation 
of an a v should cause responses only in those b v (with the same v ) 
which He ahead of the a v in question in the direction of the arrows. 
Finally we cut the periphery of A open at p; i.e., we replace it by an 
open line. This line may be straight or broken; i.e., it may consist 
of one or of more straight pieces, as shown in Figure 28h-28k. At 



DESIGN OF SOME BASIC ORGANS 



193 



any rate, the inputs a v and the outputs b v remain attached to it as 
before, but we may also change (invert) the side of the line to which 
they are attached. (For the latter, cf. Fig. 28k.) Figures 28h-28k 
also show that we keep indicating the (previously cyclical, now 
linear) sense referred to above by arrows (as in Fig. 28g), and we 
replace the point p by pi (beginning) and p 2 (end). 

The obvious way to achieve the functioning described above is by 
coding and decoding. To do this we correlate to every v = 1, • • • , n 
a suitable stimulus-no-stimulus sequence %} • • • i v m . (The length m 
of the sequence %} • • • i v m could have been made dependent on v, 
but this is not necessary.) Now we attach to every input a v a cod- 
ing organ Y*(i v l • • • i v m ), and to every output b v a decoding organ 
D(zV • • • i v m ). We connect all these inputs and outputs by a con- 
nected line of (ordinary) transmission states — o or to or o or io, 
with the arrow each time pointing in the proper direction — following 
the path indicated by the arrows in the illustrative figure (any one 
of the Figs. 28h-28k that one wishes to use for picturing this proce- 
dure). To be more precise, the connecting links (for the entire system 
of a„'s and b v 's) must consist of such transmission states. This is also 
true for the places where the inputs (the a v 's) are tied in. However, 
at the places where the outputs (the 6/s) are tied in, the stimulus 
must be able to progress in two directions (namely along the subse- 
quent transmission states and to the output that is attached at the 
point in question). Hence at each one of these places a confluent 
state C is required. This chain of (ordinary) transmission states and 
of confluent states will be called the main channel of the organ. 

Figure 29 (which may be viewed as an elaboration of Figure 28i) 
shows this in more detail. This figure shows a particular, but typical, 
distribution of a/s and 6/s (v = 1, • • • , n) with n = 2. Note that in 
Figure 29 the input of P(i v l • • • i v m ) is at the distance of one square 
from the true input a v , and is designated by a/, while the output of 
P(z7 • • • i v m ) is designated by &/. Also, the output of D(i v l • • . i v m ) 
is at the distance of one square from the true output b v , and is desig- 
nated by 6,", while the input to Dfc 1 • • • i v m ) is designated by a/'. 
Note, furthermore, that each P(z7 • • • i v m ) or D(z7 • • • i v m ) may be 
rotated by any one of the four angles 0°, 90°, 180°, 270°, and, if neces- 
sary, also reflected about the horizontal or the vertical, against the 
standard arrangements in Figures 16g and 22c. Clearly, this calls 
merely for trivial transformations of the constructions of Figures 16 
and 22, respectively. 

We must now select the sequences i v l • • • i™, v = 1, • • • , n. Each 
b v , i.e., each D^ 1 ••• i v m ), must respond to the i™ with 



194 



THEORY OF SELF-REPRODUCING AUTOMATA 



n = v, and to no other. In view of the properties of the D^' 1 . • . i m ) 
{cf. the beginning of Sec. 3.3}, this means that no . . . i™> with 
fx ^ v must contain i v l • • • i v m . In order to avoid "misunderstandings" 
due to mis -positioning in time, this requirement must also be extended 
to the case when %^ • • • i™ and i v l • • • i™ are shifted relatively to 
each other (each one of these two sequences being both preceded and 
followed by a sufficiently long sequence of O's). 

This is certainly the case if the i v l • • • i v m , v = 1, • • • , n, are pair- 
wise different, and if all of them begin with a stimulus and contain 
the same total number k of stimuli. We assume, therefore, that these 
conditions are fulfilled. 

Hence our problem is to find n pairwise different sequences i v 2 • • • i v m 
(we need not consider i v l , since it is 1 ), corresponding to v = 1, • • • , n, 
each of which has the length m — 1 and contains precisely k — 1 
ones. The number of (different) sequences of this kind is obviously 

(m - l\ 
\k - \)' 

Consequently the choice in the above sense is possible if and only if 

(12-) »s(r:!)- 

Therefore the only task remaining on this count is that of choosing 
m, k so that they fulfill condition (12'). 

It would be unreasonable to choose k so that k — 1, too, fulfills 
condition (12') {with the same m}. Hence 

(ra — l\ /m — l\ 
k -\) > \k -2)> 

which means k — 1 < m — k + 1,2k < m + 2, and so 2k ^ m + 1, 
i.e., 

(13') 

Actually the choices k = (m + l)/2 and k = m/2 are usually the 
practical ones. 

These discussions make it clear how the construction of the coded 
channel must be effected in each specific special case that may arise. 
We will use for its abbreviated representation the schemata of Figures 
28h-28k, with the arrows but without necessarily including the 
letters p± , pi , and with each cell entry or exit marked with its a v or 
b v , to the extent to which this is desirable. 



DESIGN OF SOME BASIC ORGANS 



195 



There still remain some questions of detail that need to be con- 
sidered. 

All the organs P^',, 1 • • • i v m ) and &(i v l • • • i v m ) of Figure 29 have 
the same k (cf. Sees. 3.2.1 and 3.3 above), and therefore the same 
length (cf. Figs. 1 6g and 22c). As Figure 29 shows, the P^V • . . i*) 
and D(^V • • • i v m ) are so arranged (oriented), that it is always this 
length that defines the width of the parallel strip along the main 
channel that they occupy. Hence this width is uniformly 2k. Adding 
to it 1 for the main channel and 2 for two protecting strips of U's 
along either side, we obtain the entire width of the parallel strip 
along the main channel, which forms the organ (i.e., the coded 
channel): 2k + 3. 

The heights of the various organs P(i v l ■ • • i v m ) and D(zV • • • i v m ), 
u + 2 with their various u, might vary a priori (cf. Figs. 16g and 
22c). Inspection of the definition of the u of a P(zV • • • i v m ) from 
its u° (cf. (2') and the remark before it in Sec. 3.2.1) shows that the 
u of the P(tV • • • i v m ) can be made all equal (as the maximum of all 
u°). Let u be the common u of all P(iV ••• i v m ). Inspection of 
the definition of the u of a D(zV • • • i y m ) from its u° {cf. equation 
(ll') and the remark before it in Sec. 3.3} shows that the u of the 
D(tV • • • i v m ) can be made all equal (as the maximum of all u°)\ 
in addition, it is not hard to verify that all u° are automatically 
equal. At any rate, let u be the common u of all Dfe 1 • • • i v m ). 

As Figure 29 shows, the P(i p l • • • i v m ) and D(i v l • • - i™) are so 
arranged (oriented), that it is always these heights that define the 
distances of the organs P(zV • • • i v m ) and D(zV • • • i v m ) along the 
main channel. However, the distances obtained in this way are in 
any event only lower limits: whenever it is desired to increase the 
distance between two neighboring organs {from the P(zV • • • i v m ), 
D^', 1 • ■ ■ i v m ) class}, it suffices to replace the single separating line 
of U's (as shown in Fig. 29) by a suitable, larger number of such lines. 

The next question relates to delays. The delay from the stimula- 
tion at a particular a v to the response at a particular b v is easily deter- 
mined by inspection of Figure 29. The [total] delay from aj through 
P(tV i v m ) to bj is d! = 2k + u + 2 (cf. the end of Sec. 3.2.1); 
the delay from a" through D (i v l - . . i v m ) to b" is d" = 3k + u + m 
(cf. the end of Sec. 3.3). Let A be the distance along the main channel 
from the 1 (or to or or io) where the a v is tied in to the C where 
the b v is tied in (taking every C twice, and including the terminal 
C, but not the initial t — or to or 1 or io — into the count of squares, 
i.e., the distance). Then the total delay from the stimulation at a v to 



196 



THEORY OF SELF-REPRODUCING AUTOMATA 



the response at b v is 1 + d! + 2 + A + 1 + d" + 1 = A + d' + 
d" + 5. 

We must also consider the question of corruption by interference, 
which assumes here the following shape. Clearly the stimulation at 
an a v will cause a response at every b v (assumed to lie ahead in the 
direction of the arrows along the main channel). Also, the stimulation 
at an a v will not cause a response at any 6 M with /x ^ *>, provided that 
no other stimulations (at one or more ax's, with X's among which 
X = v as well as X ^ v may occur) have taken place. The question is 
therefore : Can stimulations at several a/s (with repeated or different 
v's) cause a response at any 6 M , where neither of the stimulations 
referred to above could have caused a response by itself? Or, to be 
more specific: What rules must we establish for the stimulations at 
the a v , to prevent such occurrences? 

These rules will be given in the form of prescribing certain minimum 
delays between a stimulation at an a v and a simultaneous or subse- 
quent stimulation at any other a\ . (This includes X = v as well as 
X v\ also for X = v and a subsequent stimulation the two a v may 
or may not be the same.) 

Consider, accordingly, a 6 M . A response will take place there if its 
Diin 1 ••• in m ) receives, at its a/', a stimulus-no-stimulus sequence 
containing • • - i M m . Each a v can produce a sequence i v l ■ • . i™. 
What we must prevent is that a superposition of several shifted speci- 
mens of these should contain a • • • i™ which did not actually 
occur (in the correctly shifted position) as one of them. 

This could only occur if two shifted i v l • • • i v m (of different origin) 
contributed together (but neither of them separately!) ^ k stimuli 
to the same sequence j 1 • • • j m . Let this sequence j l • • • j m have subse- 
quences of lengths m \ m" , respectively, in common with these two 
sequences i v l • • • i v m . Then m', m ^ 1, m + m ^ k. The distance 
of the beginnings of the two shifted sequences i v l • • • i v m must then 
be either ^ | m — m" \ (if they are both at the same end of the 
sequence j l • • • j m ) or ^ 2m — m — m (if they are at opposite 
ends of the sequence j 1 • • • j m ). The former is ^ k — 1, the latter 
is ^ 2m — k, and (since (12') requires k ^ m) k — 1^2m — k. 
Hence the distance in question is at any rate ^ 2m — k. Hence the 
occurrence in question is excluded if the distance (of the beginnings) 
of the two shifted sequences i v l • • • i v m is > 2m — k, i.e., ^ 2m + 
1 - k. 

Consider therefore two inputs a v , a\ , such that the output 6 M lies 
ahead of them in the direction of the arrows along the main channel. 



DESIGN OF SOME BASIC OKGANS 



197 



(For the relationships between v, X, and between a v , a\ , cf. above.) 
By interchanging a v , a\ , if necessary, let a x lie ahead of a v in the 
direction of the arrows along the main channel. Let the distances 
from the tie-in points of a v , a\ to that one of 6 M be A 7 , A 77 , respectively, 
and between the tie-in points of a v and of a\ (taking every C twice, 
and including one, but not the other endpoint, into the count of 
squares, i.e., the distance) A* = A 7 — A 77 . 

Now let a v , a\ be stimulated at the times t\ t 11 , respectively. Then 
the sequences i v l • • • i™, ix 1 • • • i\ m , that are thus created, appear at 
a/' at the times t 1 + 1 + d! + 2 + A 7 + 1, t 11 + 1 + d! + 2 + 
A 77 + 1, respectively. The difference of these is (t 1 + A 7 ) — (t 11 + 
A 77 ) = (t 1 — t 11 ) + A*. Hence our above condition becomes 
| (t 1 - t 11 ) + A* | ^ 2m + 1 - k. This means, that either (t 1 - 
t 11 ) + A* ^ 2m + 1 - fc, i.e., 

(14') t 1 ^ t 11 + (2m + 1 - k - A*), 

or (t 1 - t 11 ) + A* ^ - (2m + 1 - k), i.e., 

(15') t 11 ^ t 1 + (2m + 1 - k + A*). 

At this point it is best to distinguish certain special cases, as 
follows. 

First, t 1 ^ 2 77 , i.e., a stimulation (at a v , time t 1 ) is followed by a 
stimulation ahead of it in the direction of the arrows along the main 
channel (at a\ , time £ 77 ). First subcase: A* < 2m + 1 — k. In this 
case condition (14') is unfulfillable and condition (15') requires that 
the delay (from t 1 to £ 77 ) be ^ (2m + 1 — k) + A*. Second subcase: 
A* ^ 2m + 1 — k. In this case condition (14') requires that the 
delay from (t 1 to t 11 ) be ^ A* — (2m + 1 — k) and condition (15') 
requires that it be ^ (2m + 1 — A: ) + A *. 

Second, t 1 ^ £ 77 , i.e., a stimulation (at ax , time t 11 ) is followed by 
a stimulation behind it in the direction of the arrows along the main 
channel (at a v , time t 1 ). Third subcase: A* < 2m + 1 — k. In 
this case condition (14') requires that the delay (from t 11 to t 1 ) 
be ^ (2m + 1 — k) — A*, and condition (15') is unfulfillable. 
Fourth subcase: A* ^ 2m + 1 — k. In this case condition (14') is 
automatically fulfilled and condition (15') is unfulfillable. 

These four subcases can now be summarized, forming a rule, whose 
observance excludes corruption by interference. This rule is given in 
what follows. 



198 



THEORY OF SELF-REPRODUCING AUTOMATA 



If there has been a stimulation at an input a p , then a stimula- 
tion at an input a ff is only permissible with a delay d (^0), 
subject to the following conditions. 

Let the distance between the tie-in points of a p and of a„ 
(taking every C twice, and including one, but not the other 
endpoint, into the count of squares, i.e., the distance) be 

(16') A *^°>- 

First case: a a lies ahead of a p in the direction of the arrows 

along the main channel. Then either d ^ (2m + 1 — k) + 

A*, or, if A* ^ 2m + 1 — fc, alternatively d ^ A* — (2m + 

1 - k). 

Second case: a c lies behind a p in the direction of the arrows 
along the main channel. Then there is a limitation only if 
A* < 2m + 1 — k, in which case d ^ (2m + 1 — k) — A*. 

8.6.2 Cyclicity in the coded channel. To conclude the discussion of 
the coded channel, we come back to the question of cyclicity, which 
was briefly referred to early in Section 3.6.1. 

The reason for interrupting the main channel at p in Figure 28g, 
that is, for not continuing it from p 2 to pi in Figures 28h-28k and 
29, is clear. If such a connection existed, i.e., if the main channel were 
a closed loop, then a sequence i v l • • • i v m (injected by an a v into this 
channel) would keep circulating in it forever, i.e., keep stimulating 
each b v periodically. This is not wanted — we want a stimulus at an 
a v to stimulate each b v precisely once. However, there are various 
ways to circumvent this difficulty. We are giving in what follows one 
way that seems particularly simple. 

Consider Figure 30a, which is the equivalent of Figure 28g, but 
without the interruption at p. We have nevertheless indicated a direc- 
tion along the main channel by the arrows, but this is now to be 
viewed as cyclical. We have also put two points p\ , p 2 on the main 
channel, but these are not meant as interruptions or terminations. 
The unmarked bars on the main channel are the a v and b v , as de- 
scribed in connection with Figures 28f and 28g-28k. We now proceed 
to transform the structure indicated by Figure 30a in the following 
way. 

In addition to the n index values v = 1, • • • , n, we introduce n 
additional index values v = l', • • • , n. We may identify these with 
n + 1, • • • , 2n, i.e., put v = v + n, but this is not relevant. However, 
it is relevant that n has been replaced by 2n {e.g., in condition (12')} . 

We now cut the periphery open between pi and p 2 , and attach the 
continuations q x , pi and p 2 , q 2 to the main channel at these two 
points, as shown in Figure 30b. Along the original main channel, i.e., 



DESIGN OF SOME BASIC ORGANS 



199 



P1P2 , we leave the a v unchanged, but we replace each b v by the corre- 
sponding b v > . This is indicated in Figure 30b by affixing an asterisk 
to each bar along the original main channel, i.e., on the portion p x p 2 . 

Next we place outputs bi , • • • , b n (in this order) on the inner side 
of P2 , #2 (the one turned towards Pi , qi), and the inputs av , • • • , a n >, 
(in this order) on the inner side of pi , qi (the one looking toward 
P2 , #2). We then connect each b v (v = 1, • • • , n) directly to its corre- 
sponding a v > . All this refers to the area in Figure 30b that is enclosed 
by dashed line and cross hatched. These arrangements are shown in 
detail on Figure 30c, which gives an enlargement of that area. Note 
that the endings pi , qi and p 2 , qi of the main channel are moved 
apart in Figure 30c by as much as needed to accommodate the outputs 
61 , • • • , b n and the inputs av , • • • , a n >, that we introduced above 
for this region. More specifically, each a v > means that a P(zV • • • i v m ) 
is there, and each b v means that a D(z7 • • • i v n ) is there. It is for 
these that space (i.e., the necessary distance between pi , q x and 
P2 , #2) must be provided. The connection from b v to a v > , shown in 
Figure 30c, therefore follows the pattern shown in Figure 29: the 
output b" of D(zV • • • i v m ) is followed (by one square) by b v ; the 
input a v > of P(z'J> • • • i™,) is preceded (by one square) by a v > . The 
connection from b v to a v > is shown in Figure 30c (by an arrow) as a 
vertical channel (plausibly of {ordinary} transmission states io) ? but 
it suffices to make it a direct contact of b y and a v > . In fact, the single 
U borders of D(tV • • • i v m ) between b" and b v and of P(iJ/ • • • i™, ) 
between a v > and a v > (cf. the analogs to this in Fig. 29) can be identified 
so that b v ,f merges with a v > , and b v with a v > . 

The functioning of this organ, according to Figures 30b-30c is now 
easily analyzed. A stimulation at an a v (which must be one of the 
bars marked with an asterisk on the portion pip 2 of Fig. 30b) can only 
stimulate a b v . There exists precisely one b v , and this lies ahead of 
the a v in the direction of the arrows along the main channel, namely 
on the portion p 2 q 2 , as detailed in Figure 30c. Hence the b v will be 
stimulated, and from it the stimulus goes directly (along the arrow) 
to a v > , on the portion q x pi , as detailed in Figure 30c. Thus the a v > is 
stimulated, which can only stimulate All &„>'s lie ahead of the 
a v > , in the direction of the arrows along the main channel, namely, 
on the portion p x p 2 of Figure 30b, i.e., among the bars marked with 
asterisks. To sum up, a stimulus at an a v (necessarily at a bar marked 
with an asterisk in Fig. 30b) will stimulate all the b v > (all among the 
bars marked with asterisks in Fig. 30b) — irrespective of their positions 
relative to each other on the portion pip 2 of Figure 30b. Also, after 
this has taken place the stimulus will die, i.e., no periodic repetition 



200 



THEORY OF SELF-REPRODUCING AUTOMATA 



will take place, since the main channel of Figure 30b is open (from 
qi to #2 , i.e., is not cyclical). Thus, from the point of view of Figure 
30a, we have precisely the events that were intended. 

[ This concludes von Neumann's discussion of the coded channel. It 
will be recalled that he concluded his discussion of the non-cyclic 
coded channel (Sec. 3.6.1) with rule (10 ), which concerns corruption 
resulting from two stimuli entering the coded channel too close to- 
gether. It is natural to ask why he didn't state a corresponding rule 
for the cyclical coded channel of Figure 30 (Fig. 27 is an example). 
Von Neumann gave no reason, but it may be because he later con- 
structed the control organ for his universal self -reproducing autom- 
aton with a non-cyclic coded channel; see Figure 37.] 



Chapter 4 



DESIGN OF A TAPE AND ITS CONTROL 



4.1 Introduction 

[4.1.1 Abstract. In the present chapter von Neumann shows how 
to embed an indefinitely extendible tape and its control in his infinite 
cellular structure. 

The following units are involved in the tape and its control: 

(1) A linear array L for storing information: "zero" is represented 
in cell x n by state U and "one" is represented by state io. 

(2) A connecting loop Q for reading an arbitrary cell x n . 

(3) A timing loop C2 , used in modifying the length of the connecting 
loop Ci . 

(4) A memory control MC, used to control the operations of L. Ci . 
and C 2 . 

(5) The constructing unit CU, which controls MC. 

Except for CU, all these units are shown in Figure 37, though not in 
correct proportion. See also Figure 50. 

Von Neumann describes these units in a general way in the re- 
mainder of the present section. He develops the detailed operations 
for lengthening and shortening loops Ci and C 2 and for writing in cell 
x n in Section 4.2. He designs most of the memory control MC in 
Section 4.3; the design will be completed in Section 5.1. 

Von Neumann developed his design in several stages, which he 
thought out as he proceeded. The final design operates as follows. 

The constructing unit CU sends a pulse to the memory control 
MC signifying that cell x n is to be read. This pulse causes the sequence 
10101 to enter the connecting loop Ci . The sequence 10101 then 
enters cell x n with the following effect : if x n is in state U the sequence 
1010 changes it into state 10 and T returns to MC, while if x n is in 
state 10 the whole sequence 10101 returns to MC. A T vs. 10101 
discriminator ^ detects the output and informs the constructing unit 
CU whether x n stored a "zero" or a "one." In either case cell x n is 
left in state io. 

The constructing unit CU then tells the memory control MC 



201 



202 



THEORY OF SELF-REPRODUCING AUTOMATA 



whether the loop Ci is to be lengthened so as to pass through cell 
x n +i or shortened so as to pass through cell x n -i , and whether cell x n 
is to be left in state U ("zero") or 10 ("one"). Loop Ci is used to 
time the lengthening (or shortening) of loop C 2 . Then loop C 2 is 
used to time the lengthening (or shortening) of loop C x . The new 
bit of information is written in cell x n while loop Ci is being lengthened 
(or shortened). At the end of the whole process the memory control 
MC sends a finish signal to the constructing unit CU. 

We have indicated here only those functions of the constructing 
unit CU which concern the memory control MC. The primary purpose 
of CU is to carry out the construction of a secondary automaton 
whose description is stored in L. Thus the universal constructing 
automaton has two parts : the constructing unit CU, and an arbitrarily 
large memory and its control (MC, L, Ci , and C 2 ). See Section 1.6.1.2 
and Chapter 5.] 

4.1.® The linear array L. We have reached the point where the 
subsidiary constructions are completed and our first major synthesis 
can be undertaken. This highly composite organ that we will now 
construct has a single purpose, but it will actually account for approxi- 
mately half of the entire self -reproducing organism. 

It is best, therefore, to stop here for a moment and make some 
general observations regarding the overall organization that is to be 
developed. 

We will need an automaton that can carry out general logical 
functions. The specific way in which this is integrated into the effect- 
ing of self -reproduction will be worked out in detail later, but the 
qualitative need for a general logical automaton should be clear a 
priori. These matters were broadly discussed in Chapter 1. They were 
first brought up in the formulation of the questions (A)-(E) in 
Section 1.1.2.1; their elaborations ran through Sections 1.2 to 1.7, 
and they were particularly in the foreground in Sections 1.4 to 1.6. 
We will therefore, for the time being, take the need for a general 
logical automaton for granted and discuss the ways and means of 
constructing one. 

We noted in Sections 1.2.1 and 1.4.2.3 that a general logical auto- 
maton is necessarily organized around two main parts. The first part 
is a network that can carry out the elementary logical functions 
(+, • , and — ; cf. the main discussion in Sec. 1.2.1 ), and by combining 
these can evaluate all propositional functions in logics (cf . the end of 
Sec. 1.2.1). The second part is an arbitrarily large (finite, but freely 
adjustable) external memory, and the network that is needed to 
control and to exploit that memory. We know that the first part, 



DESIGN OF A TAPE AND ITS CONTROL 203 

the machine for propositional functions, can be constructed with 
relatively simple means; indeed, the principles that are involved in 
this construction were in continuous use in our past constructions. 
We will take this part up in detail later. We turn our attention first 
to the second part, i.e., to the arbitrarily large external memory and 
its ancillary network. 

The arbitrarily large external memory's physical embodiment is 
the linear array L that was discussed in Sections 1.4.2.1-1.4.2.4. We 
saw there that it was desirable to form it from cells, each of which 
could be in any one of k preassigned states. It became clear in Section 
1.4.2.1 that these states, in order to be convenient for their "nota- 
tional" role, must be quasi-quiescent states, i.e., states like U or the 
unexcited forms of transmission or confluent states. For a variety of 
reasons it is most practical to use U's and ordinary transmission 
states. As indicated in Section 1.4.2.4 we will use a binary notation; 
i.e., we put k = 2. Accordingly, we will use the state U and a suitable 
ordinary transmission state. The orientation of the latter has to bear 
a certain relation to the orientation of the linear array L. Specifically, 
in view of the way in which we will use L, it is desirable that the 
orientation of this transmission state be transversal to the direction 
of L. The latter will be horizontal. Accordingly, it will be found con- 
venient to orient the transmission state vertically down, that is, to 
use the state io. (io represents in this case the unexcited form of that 
state, i.e., T030 ; cf. Sec. 2.8.2.) In order to have a firm relationship 
to the binary notation, we stipulate that U correspond to the digit 0, 
and 10 to the digit 1. 

The linear array L should therefore be thought of as a sequence of 
cells x n , n = 0, 1, 2, • • • , which form a continuous horizontal line, 
and for each of which the two states U, 10 are available. With each 
x n we associate a numerical variable £„ , which designates the binary 
digit represented by x n . Thus, £ n = 0 indicates that x n is U, £ n = 1 
indicates that x n is io. Strictly, the length of L should be finite; i.e., 
the range of n should terminate, say immediately before N: n = 0, 
1, • • • , N — 1 . In any case, it is best to assume that for sufficiently 
large n, £ n = 0. In view of this, the end of L merges into the field of 
U's that we expect to find outside the specifically constructed organs. 
It is therefore not important in what way L is actually terminated. 

In order to make this external memory L useful, the ancillary 
networks referred to above must be provided; that is, it is necessary to 
construct the means for "exploring" L. This "exploration" includes 
reading L at any prescribed place, and also altering it in any desired 
way at any prescribed place. 



204 



THEORY OF SELF-REPRODUCING AUTOMATA 



We are therefore passing to the discussion of these manipulations 
on L: reading and altering at assigned locations. 

4.1.3 The constructing unit CU and the memory control MC. 1 Read- 
ing L at an assigned location, say n, means to observe the value of 
£ n . Altering it there means to change it from its present value, (f n , to 
its next value, £ n '. 

It is convenient to use an index 8 (s = 0, 1, 2, • • • ) to enumerate 
the successive applications of this step. Hence the present value of a 
£ n will be designated by £ n & , and its next value by £ n 8+1 . Thus f n * +1 
replaces the £ n ' that was used above. 

The n involved in both operations (reading and altering), too, 
depends on s, and this dependence must be shown explicitly. Let 
therefore n s be the n used in the above sense at step number s. 

The number n may be specified absolutely or relatively. The latter 
will prove to be the preferable alternative. By this we mean the 
following. It is not convenient — and if logical generality is absolutely 
insisted on, it is not even possible — to limit the size of n (or of N\ 
cf. above). Hence, the number of binary digits of n is not limited 
either. Consequently, n cannot be (or cannot be conveniently) held 
"inside" the logical automaton, i.e., in its first main part according 
to the partition defined in Section 4.1.2. On the other hand, it would 
be very inconvenient to hold n in the second main part, i.e., in the 
unlimited "outside" memory L. In other words, it is undesirable at 
this stage of our automaton development (although we will do it 
later on, when the integration of our automaton will have progressed 
further) to use L itself for "addressing" within L. These two un- 
desirable alternatives exclude between them the possibility of direct 
addressing, i.e., of "absolute" specification of n. The plausible proce- 
dure is therefore to use "relative" specification of n. That is, each 
time when it becomes necessary to specify n, we will not do this 
directly, but rather by stating how the n to be used is related to the 
n used immediately before. It suffices to allow for changing n at each 
such unit time step by a single unit. 2 In other words, the n to be 
considered, n, will always be related to the one considered imme- 
diately before it, n 5-1 , by 

(17') n s = n 8 ' 1 + e s (e s = ±1). 

1 [Von Neumann did not have titles for these two units but merely called 
them "A" and "B", though the units are not the same as his A and B of Sec. 
1.6. His title for the present subsection was "The detailed functioning of L. 
The networks A and B. The function of A and its relationship to B."] 

2 Turing, "On Computable Numbers, With an Application to the Ent- 
scheidungsproblem." 



DESIGN OP A TAPE AND ITS CONTROL 



205 



The "relative" specification then consists of stating which of its 
two possible values t assumes, i.e., whether e = 1 or e = —1. 

Let us now reconsider the two main parts into which the general 
logical automaton was divided in Section 4.1.2. We can now make this 
division more precise while reformulating it with some changes. The 
first part is the network that carries out the elementary logical func- 
tions and combines them to form general logical propositional func- 
tions. Let this part be designated by CU. The second part is the ex- 
ternal memory L, and the network that is needed to control and to 
exploit L. Let the latter be designated by MC. Thus the second part, 
in the sense of Section 4.1.2, is L plus MC. We can now define the 
respective functions of CU and MC more precisely than heretofore. 

[Fig. 37 shows the relation of the memory control MC to the linear 
array L, the connecting loop Ci , and the timing loop C 2 .] 

We know that the function of the memory control MC is to read 
and to alter L at assigned locations. We saw above what this involves 
specifically. Given e 8 , MC must replace n~ l by n according to equa- 
tion (17'); it must sense the % s n » ; and given f£t\ it must replace 
£n» by i^ 1 . The only portions of this definition that require further 
explanation are those relating to the "given e s " and the "given 
£nt* ". That is, we must specify by what processes e and ijnt 1 are to 
be obtained. 

It is best to attribute this function (the obtaining of e and 
to the constructing unit CU. The unit CU must form them as a 
function of its own previous state, and of the information obtained 
in the process. The latter is, of course, the reading of % 8 n « that is per- 
formed by the memory control. 

The reading of £n« by MC occurs after the formation of e s , i.e., of 
n {according to equation (17')}, but before the formation of It 
must therefore affect the latter, but not the former. However, it is 
preferable to transpose these operations, i.e., to describe the forma- 
tion of n s+1 from n\ rather than that of n from n~ l . This means 
replacing s by s + 1 in equation (17'), i.e., replacing (17') by 

(18') n s+1 = n s + e s+1 (e sfl - ±1). 

Now we can define the functioning of units CU and MC as follows : 
CU first gives a "start" pulse to MC, thereby starting MC on step 
number s (if this is the step that has been reached at this point ) . The 
memory control MC then reads and communicates the result of 
this reading to the constructing unit CU. This new information 
changes the state of CU. The unit CU thereupon forms i^t 1 and e s+1 
and communicates these to MC. The unit MC now changes % s n » into 



20G 



THEORY OF SELF-REPRODUCING AUTOMATA 



i^t 1 ; then MC forms n +] according to equation (18') and establishes 
the same contact with n s+1 (i.e., with x n *+i in L) that it had previously 
with n (i.e., with x n s in L). This concludes step number s. The mem- 
ory control MC communicates this fact to CU. If it is so prescribed 
for the state in which CU is at that point, CU then gives the next 
"start" pulse to MC, thereby starting MC on step number s+l, 
etc., etc. 

For the time being, we wish to give only a schematic description 
of the constructing unit CU. It suffices therefore to say that CU, 
being a finite network, has only a finite number of states, say a. Let 
these states be enumerated by an index a = 1, • • • , a. Let the state 
of CU at the beginning of step number s be a. 

Then the relevant facts involving CU and its relationship to MC 
are contained in these specifications: 



(19'.a) < 



(19\b) 



(19'.c) 



(19'.d) 



If CU is in the state a\ and MC communicates to 
it a value , then CU goes over into the state 



s+l • v 

a , given by 



a = A (a, £n«)« 



If CU is in the state a s+1 , then it forms (and 
communicates to MC) the i^t 1 given by 

& 1 = X(a s+1 ). 

If CU is in the state a s+ \ then it forms (and 
communicates to MC) the e s+1 given by 

S+l 77T/ 

e = lii \a ). 

CU delivers the "start" pulse for step number s to 

MC if and only if its state a lies in the subset $ of the a. 



Thus the three functions 
(20'.a) 



A(a, {) (a = 1, 



(20'.b) 
(20'.c) 
and the set 
(20'.d) S 



X(a) 
E(a) 



(a = 1, 
(a = 1, 



• • , a; % = 0, 1 ; values of A = 

a; values of X = 0, 1 ) , 
a; values of E = d=l), 



(subset of the set of all a = a), 



contain the operational description of CU and of its relationship to 
MC, insofar as these are relevant for our immediate purposes. Later 



DESIGN OF A TAPE AND ITS CONTROL 



207 



on, we will describe CU in detail, 3 but for the present the above 
description suffices. Our present task is to perform complete and 
detailed construction of MC, including its connections with CU. 

4.1 4 Restatement of the postulates concerning the constructing unit 
CU and the memory control MC. Let us restate the postulated func- 
tioning of CU and MC: 

(1) CU and MC are assumed to have reached the beginning of step 
number s. CU is in the state a\ MC is connected to the cell 
x n * in L. 

(2) If a is not in S, then CU will not communicate further with MC. 
If a is in S, then CU gives a "start" pulse to MC. The "start" 
pulse initiates the events of step number s. 

(3) MC reads the state of x n * , i.e., the value of ££• , and communi- 
cates this to CU. 

(4) CU then forms a s+1 according to (19 .a), and goes into the state 

«+l 

a 

(5) CU then forms fjt 1 and € s+1 according to (19'.b) and (19'.c) and 
communicates these to MC. 

(6) MC changes x n » , as required to change £n« into £nt\ 

(7) MC then forms n s+1 = n + e s+1 {cf. equation (18')} and estab- 
lishes its connection with x n »+i , relinquishing its connection 
with x n * . That is, it moves its connection within L one square 
forward if e s+1 = 1, and one square back if e s+1 = —1. 

(8) Finally, MC gives a "completion" signal to CU. 

(9) Now step number s is completed and step number s + 1 begins. 
The cycle therefore resumes with postulate (1) above, etc., etc. 

Since we wish to give only a schematic description of CU at this 
stage (cf. Sec. 4.1.2), the purely formal statements of postulates 
(1), (2), (4), (5), concerning CU will suffice for the time being. The 
statements of postulates (1), (2), (3), (6), (7), (8), on the other 
hand, concern MC, and our present task is to carry out an effective 
and detailed construction of MC. Hence these last mentioned state- 
ments must be implemented by actual constructions. We will do this 
in Sections 4.1.5-4.1.7, which follow. 

4.1.5 The modus operandi of the memory control MC on the linear 
array L. We must first define the nature of the "connection" between 
MC and a specific cell x n in L, as used in postulate (1) {for n = n] 
and postulate (7) {for n = n s+l ] above. This connection must be 
such that it makes the "reading" of x n by MC possible {cf. postulate 
(3), with n = n\, as well as the changing of x n by MC {cf. postulate 

3 [Von Neumann never reached the part of the manuscript where he planned 
to design the constructing unit CU. See Chapter 5 below.] 



208 



THEORY OF SELF-REPRODUCING AUTOMATA 



(6) , with n = n). It must also be possible to move this connection, 
operating from MC, one square forward or backward {cf. postulate 

(7) 1. 

The linear array L is a linear sequence of cells x n , n = 0, 1, 2, • • • . 
We assume it to be horizontal, extending, and numbered, from left 
to right, as shown in Figure 31a. We assume furthermore that MC 
lies to the left of L, as shown in the same figure. 

The obvious way for reading cell x n {with n = n; cf. postulate (3) 
in Sec. 4.1.4} consists of having a line of ordinary transmission states 
leading to it from MC, and then back from it to MC. We place the 
line from MC to x n on the upper side of L and the line from x n back 
to MC on the lower side of L. Together with x n these form a loop 
from MC to MC, to be called the connecting loop and to be desig- 
nated Ci , as shown in Figure 31b. Now cell x n can be "read" by 
sending a stimulus into the Ci loop at its entry Vi , and observing 
what emerges at its exit W\ . If x n is U (i.e., £ n s = 0) then the stimulus 
will not be able to pass through x n (i.e., no response will appear at 
Wi). If x n is io (i.e., £ n s = 1) then the stimulus will pass through 
x n (i.e., a response will appear at W\). 

Note that the stimulus coming from v x through Ci will nevertheless 
affect x n if it is U (i.e., if £ n s = 0). It will transfer x n into the sensi- 
tized state S0 (cf. Sec. 2.8 for this and for all subsequent discussions 
of transitions between states). If nothing further happened, x n would 
then go from the state Se spontaneously through certain successive 
states, the complete sequence being 

(21 ) U — > S0 — > So — ► Soo — ► Sooo — > Soooo = Tooo = ^. 

Now, it is preferable to have x n terminate in the same state in which 
it would have been otherwise (i.e., if £ n s = 1 had held instead of 
%n = 0), namely, the state 1<>. The reason for this is that our sub- 
sequent operations with x n can be organized more simply if they are 
known to start with a fixed state of x n under all conditions. The 
state is T030 , i.e., Soio . Hence the original stimulus (which trans- 
ferred x n from U into S ) should be followed by the stimulus-no- 
stimulus sequence 010. Hence a total sequence 1010 (at ^i) is called 
for. 

Injecting 1010 at v x thus transforms x n , in the case £„* = 0, succes- 
sively according to 

(22 ) U — > S0 — > So — ► Soi — ► Soio = To3o = 1°. 



Since the io appears at the end, none of these stimuli is able to pass 



DESIGN OF A TAPE AND ITS CONTROL 



209 



through x n , so no response appears at W\ . In the case £ n a = 1, on 
the other hand, x n is 1<>; hence it is not modified, and all stimuli 
pass through it, so that the response at w x is 1010. To sum up, in- 
jecting 1010 at Vi leaves x n at the end in any case in the state, !<> and 
it produces at W\ no response or the response 1010, for % n s = 0, 1, 
respectively. 

This procedure still has one shortcoming: the criterion of "no 
response at Wi ," which appears here as the characteristic of £ n s = 0, 
is meaningless if it is not known at what time this "no response' ' at 
Wi is expected, since there is equally no response at wi at other times, 
when vi has not been "questioned " (stimulated). This means that 
the delay from v\ to Wi must be established by other means. This will 
be needed for other purposes, too, and it will be achieved with the 
help of the so-called timing loop (cf. later). However, for the present 
purpose the desired result can be secured more simply by a direct 
procedure, and we are therefore choosing this procedure. 

Indeed, it suffices to add one more stimulus to the input sequence 
1010 at Vi , i.e., to use the sequence 10101 there. The cell x n will be 
in any event in the state lo before the last stimulus; hence this 
stimulus will pass, appear at W\ , and still leave x n in the state I©. 
Thus injecting 10101 at v\ leaves x n at the end in any case in the state 
lo, and it produces at w\ the response 1 or 10101 for £ n * = 0, 1, re- 
spectively. 

If the input 10101 at vi is followed by 00, then for £ n s = 0 the I 
at Wi is also followed by 00, and since this 1 is certainly preceded by 
0000 (cf. above), we have for £ n s = 0 the sequence 0000100 at w x . 
This sequence cannot overlap with a sequence 10101 (which corre- 
sponds to & = 1 ) . Hence the response at w x times itself and discrimi- 
nates unambiguously between £„ s = 0 and £„ s = 1. The organ which 
effects this discrimination is our 1 vs. 10101 discriminator (organ 
¥of Fig. 25). 

Thus the problem of reading x n (with n = n) is solved, and the 
outline of the construction of a network to implement this procedure 
is clear. (For the actual construction, cf . later. ) 

[ The operation of inserting the sequence 10101 is the first of a 
complicated sequence of operations involving the linear array L, 
the connecting loop Ci , and the timing loop C 2 . The remaining 
operations are developed in the balance of the present section (4.1) 
and the next section (4.2), and are summarized in Figure 38 of Sec- 
tion 4.3.2. See also Figure 37. 

When von Neumann's design is finished, the following sequence of 
operations will be used to read from and write in cell x n of L, and to 



210 



THEORY OF SELF-REPRODUCING AUTOMATA 



lengthen (or shorten) the connecting loop Ci and the timing loop C 2 
preparatory to reading cell x n+i (or cell x n -i): 

(1) Read cell x n with the sequence 10101, leaving it in state lo. 

(2) Lengthen (or shorten) the timing loop C 2 , using the connecting 
loop Ci to time this operation. 

(3) Lengthen (or shorten) the bottom of the connecting loop Ci , 
using the timing loop C 2 to time this operation. 

(4) Change cell x n to U if necessary. 

(5) Lengthen (or shorten) the top of the connecting loop Ci , using 
the timing loop C 2 to time this operation.] 

4 J S The connecting loop Ci . The connecting loop Ci , as defined 
above, must also handle the problem of altering x n (with n = n; cf. 
postulate (6) in Sec. 4.1.4), that is, of transferring x n from that 
state U, lo, which corresponds to £ n s = 0, 1, respectively, into that 
one which corresponds to £ n s+1 = 0, 1, respectively. 

It is natural, as well as simplest, to perform this transformation of 
x n immediately after reading it according to Section 4.1.5. However, 
we will see later that this conflicts with some other desiderata, and 
we will therefore depart from this arrangement. Nevertheless, it is 
instructive to carry out a preliminary discussion of the altering of 
x n based on this assumption, to be corrected later on. 

Consider, accordingly, the process of altering Xn , as if it took 
place immediately after the "reading" of x n . At that moment x n is 
certainly in the state lo (cf. above), which simplifies the task some- 
what. Indeed, it is now only necessary to distinguish between two 
cases: £ n s =1, in which case Xn is already in the desired state and 
nothing need be done; £ n s+1 = 0, in which case x n must be transferred 
from lo to U. (If x n had not been altered from its original form, there 
would be three distinct cases, as follows. First, & = £» f+1 { = 0, 1; 
it is irrelevant which}, and nothing need be done. Second, £ n * = 1, 
£ n s+1 = 0, and x n must be transferred from 1© to U. Third, £ n s = 0, 
£ n s+1 = 1, and x n must be transferred from U to lo.) 

Thus the only operational problem that has to be considered here, 
is that of transferring x n from lo to U. Since lo is an ordinary trans- 
mission state, this cannot be done with ordinary stimuli. Hence we 
have the problem of bringing special stimuli into the proper contact 
with x n . 

This can be done in the following manner. Let a special transmis- 
sion state be in direct contact with the first °* in Ci , i.e., the one next 
to Vi and above xo . This can be done by placing a state 1 1 immedi- 
ately above the °> in question, i.e., at the asterisk in Figure 31b. Now 
a special stimulus from the asterisk will transfer the °* in question 



DESIGN OF A TAPE AND ITS CONTROL 



211 



(i.e., the one above x 0 ) into U. It is desirable to transfer this further 
into the state i, which is Tioo (i.e., Son). Hence the original, special 
stimulus should be followed by the stimulus-no-stimulus sequence 
1011. These could be ordinary or special stimuli, since both kinds 
have the same effect on the sensitized states. It is simplest to keep 
the entire sequence of one kind, i.e., of special stimuli only. This 
means, then, injecting a total sequence of special stimuli 11011 from 
the asterisk. Hence the square above xq will be successively trans- 
formed according to 

(23 ) i — > U — >S# — > So — > Soi — > Son = Tioo — 1 >- 

Now the square above x Q is a i. Hence a second sequence of special 
stimuli 11011 injected from the asterisk will pass through the i above 
Xq as special stimuli and hit the next °, i.e., the one above Xi . It will 
therefore put this square through the successive transformations of 
expression (23'), and in the end leave it as a i. Now the squares 
above x 0 and Xi are both in state L Hence a third sequence of special 
stimuli 11011 injected from the asterisk will pass through the two 
1 above x 0 and Xi as special stimuli and hit the next °, i.e., the one 
above x 2 . It will therefore put this square through the successive 
transformations of expression (23'), and in the end leave it in state 
i. Now the squares above xq , X\ , and x 2 are all in state i. Clearly 
this process can be continued at will. After the injection of n sequences 
of special stimuli 11011 from the asterisk, the n squares above xo , 
Xi , • • • , x n -i are all in state i. 

At this point the procedure should be changed, since the square 
above x n must be transformed so that it will direct a special stimulus 
to x n , and not to the next ° (the one above x n +i). That is, it should 
become a ii, which is Ti 3 o (i.e., Sno). Therefore a sequence of special 
stimuli 11110 should now be injected from the asterisk, transforming 
the square above x n according to 

(24') i -* U -> So -» Si -> S n -> S 110 = T 130 = U. 

Now we have a continuous chain of special transmission states from 
the asterisk to x n ; hence we can transform x n . This is a transforma- 
tion from lo to U, and a single special stimulus delivered from the 
asterisk will induce this transformation. 

Thus we need the following system of sequences of special stimuli 
injected from the asterisk: n sequences 11011, followed by the se- 
quence 111101 (this is, of course, 11110 and then 1). The subsequent 
condition of C x and L is shown in Figure 31c. 

It is not desirable to leave the part of Ci that has been so modified 



212 



THEORY OF SELF-REPRODUCING AUTOMATA 



(i.e., its upper line) in this condition. However, it is easy to return 
it from its altered condition (in Fig. 31c) to its original condition (in 
Fig. 31b), leaving x n in its altered condition (which is U). Indeed, 
since the upper line of Ci now consists of special transmission states, 
ordinary stimuli will do this. These can be injected at Vi . The n first 
squares (above xo , Xi , • • • , x n -i) should be transformed from i into 
ft, which is Tooo (i.e., Soooo). Hence n sequences of ordinary stimuli 
110000 must be injected at v x . The first one will transform the square 
above xq from i into ft, according to 

(25 ) 1 1 — > U — > S# — »So — ► Soo — ► Sooo — ► Soooo — Tooo = 1 0 . 

The second one will do the same to the square above x x , etc., etc., 
and the n-th will do the same to the square above z w -i . The square 
above x n can now be transformed from ii into |o. This is T 0 3o (i.e., 
S 0 io). Hence a sequence 11010 of ordinary stimuli must be injected 
at vi . This will transform the square above x n according to 

(26') H -> U -> S 9 -> So -> Soi -> Soio = T 03 o = !<>. 

Thus we need the following system of sequences of ordinary stimuli 
injected at Vi : n sequences 110000, followed by the sequence 11010. 
The subsequent condition of Ci and L is again that shown in Figure 
31b, but with x n being U. 

The above discussions show that ordinary as well as special stimuli 
must be injected into Ci . It is desirable to control both kinds by 
ordinary stimuli from memory control MC. This calls for no extra 
arrangements for the ordinary stimuli to Ci ; they can be injected 
directly at v\ . However, further arrangements are needed for the 
special stimuli to Q , which have to be injected from the asterisk 
of Figures 31b and 31c. Thus there is need for an organ that converts 
ordinary stimuli into special stimuli. We constructed such an organ 
earlier for the purposes of the periodic pulser. It is shown in Figures 
18g and 18i. The arrangement of Figure 18i is more convenient. Its 
attachment to Ci and L is shown in Figure 3 Id. The input to this 
subsidiary organ is designated by U\. 

In this way we have a system where the original stimulations are 
uniformly made by ordinary pulses either at u x or at Vi . The former 
cause the injection of special pulses into Ci ; the latter inject directly 
ordinary pulses into Ci — in both cases into the square above cell x 0 . 

We can now give the final form of the description of the treatment 
that transfers x n from U into i o, and leaves everything else in Ci and 
L in its previous condition: 
(27 .a) Inject n sequences 11011 at U\ . 



DESIGN OF A TAPE AND ITS CONTROL 



213 



(27'. b) Inject a sequence 111101 at u x . 

(27'. c) Inject n sequences 110000 at v x . 

(27'.d) Inject a sequence 11010 at Vi . 

Jf.1.7 The timing loop C 2 . The scheme of the previous subsection, 
as summarized in rules (27 .a-27 .d) is still incomplete in one respect. 
Rules (27'. a) and (27'. c) call for the n-fold repetition of certain 
operations, injecting 11011 and 110000 at Ui and Vi , respectively. 
This can be done with the periodic pulsers PP(llOll) and 
PP (110000), but these must then be turned off with the delays 5n 
and 6n, respectively, after they have been turned on. How are these 
delays to be ascertained ? 

Prima facie there would seem to be the same difficulty, connected 
with the size of n, and its consequent unsuitability for storage "in- 
side" MC, which we observed at the beginning of Section 4.1.3. 
This difficulty can be overcome by essentially the same device: rela- 
tive instead of absolute definition of n. However, it is impractical 
to sense n, with our present arrangements, on L itself. It is therefore 
necessary to introduce a second "outside" organ to store n — or 
rather to express it in so explicit a manner that its sensing for the 
above purposes becomes immediately feasible. The best way to do 
this seems to be the introduction of another loop from MC to MC, 
whose length each way is n, or n plus a fixed amount. That is, we 
assume that there is, at a suitable distance below L and parallel to 
it, a sequence of n ordinary transmission states ° leading out from 
MC, this sequence being terminated by an ordinary transmission 
state 1 o, and immediately under this another line of n + 1 ordinary 
transmission states 1 leading back to MC. We position these below 
the squares x 0 , Xi , • • • , x n of L. This will be called the timing loop 
and designated C2 , as shown in Figure 31e. Clearly a stimulus injected 
into the timing loop at its input v 2 will reappear at its output w 2 with 
a delay of 2n + 2. It will appear later on that it is desirable to pro- 
vide C 2 with the same additional facilities for the injection of special 
stimuli, as were provided for Ci . Hence we place again the equivalent 
of Figure 18i above the t under Xq , and designate its input by u 2 . 
This arrangement defines the distance from Ci to C 2 and the position 
of both relatively to MC, as shown in Figure 31f. 

Since the delay from v 2 to w 2 is 2n + 2, the delay three times around 
this loop is 6n + 6. This differs from the delay 6n, required in con- 
nection with operation (27'. c) of Section 4.1.6 by 6, which is a fixed 
amount (i.e., independent of n). Hence it can be used to define the 
delay called for in connection with operation (27'.c), subject only 



214 



THEORY OF SELF-REPRODUCING AUTOMATA 



to the insertion of suitable fixed delays, which is feasible "inside" 
MC. 

The delay required in connection with operation (27'. a) of Section 
4.1.6 is 5n, and no integer multiple of 2n + 2 differs from this by a 
fixed amount (i.e., by one that is independent of n). This difficulty 
can be circumvented by adding a 0 (i.e., a "no stimulus") to the 
sequence 11011 in operation (27 .a). This gives the sequence 110110; 
i.e., it replaces operation (27 .a) as follows: 

(27'. a') Inject n sequences 110110 at u\ . 

Now operation (27'.a'), too, requires a delay 6n; i.e., it can be treated 
precisely like operation (27 .c). 

The sensing of three trips around the timing loop can be effected 
with the triple-return counter. This organ <£ is shown in Figure 23. 
Its output d must then be attached to v 2 and its input c to w 2 . Let 
the delay from d to v 2 be b 2 and the delay from w 2 to c be b 2 . In the 
terminology of Section 3.4, then, 12 is the line from d to v 2 , plus C 2 
from v 2 to w 2 , plus the line from w 2 to c. Hence the delay from c 
through 12 to d is b 2 + (2n + 2) + b 2 . Consequently, according to 
the end of Section 3.4 and Figure 24d, the delay from a through <i> 
(and 12) to b is 238 + 3(6/ + (2n + 2) + b 2 ) = 6n + (3(S 2 ' + 
b 2 ) + 244). This exceeds the delay 6n, called for in connection with 
rules (27\a) and (27'.c), by 3(S 2 ' + b 2 ) + 244, which is a fixed 
amount (i.e., independent of n). Hence this can be adjusted for by 
suitable fixed delays that can be provided "inside" the memory 
control MC. 

4.2 Lengthening and Shortening Loops Ci and C 2 , 
and Writing in the Linear Array L 

4.2.1 Moving the connection on L. There remains one more problem 
for our preliminary consideration: that of moving the connection of 
MC with L one square forward or backward (cf. postulate (7) in 
Sec. 4.1.4). 

This means lengthening or shortening both lines of the connecting 
loop Ci by one square. Now we saw in Section 4.1.6 that the timing 
loop C 2 is expected to have the same length as the connecting loop 
Ci . Hence both lines of C 2 , too, must be lengthened or shortened 
by one square. 

It is desirable to perform these operations at a time when Ci forms 
an uninterrupted loop, from Vi to wi . (The loop C 2 is normally un- 
interrupted from v 2 to w 2 .) This is the case when x n is in state I». 
We saw at the end of Section 4.1.5 and the beginning of Section 4.1.6 



DESIGN OF A TAPE AND ITS CONTROL 



215 



that this can be guaranteed during the period between the reading 
and the altering of x n • (The reading necessarily precedes the alter- 
ing. ) We will therefore perform the operations in question (lengthen- 
ing or shortening each line of Ci and of C 2 by one square) during this 
period. 

Beyond this point the two alternatives (e s+1 = 1, lengthening, and 
e s+1 = —1, shortening) are better considered separately. 

[ The general process to be carried out is that of altering the loops 
Ci and C 2 at their ends, and perhaps modifying the contents of the 
storage cell x n . The procedure for doing this was illustrated in Figure 
14 of Section 2.8.3. The construction path is changed back and forth 
between a path of ordinary transmission states and a path of special 
transmission states. A path of ordinary transmission states is con- 
verted into a path of special transmission states by means of special 
stimuli, and a path of special transmission states is converted into a 
path of ordinary transmission states by means of ordinary stimuli. 

The following inputs are used : u x and v x for Q , and and v 2 for 
C 2 . See Figures 31-37 and 39. A sequence of ordinary stimuli into 
Ui changes the upper part of Ci into a path of special transmission 
states, and a sequence of ordinary stimuli into V\ changes the upper 
part of Ci back into a path of ordinary transmission states. Similarly 
for u 2 , v 2 , and C 2 . 

The conversion of a cell from an ordinary transmission state to a 
special transmission state (or vice- versa) takes six stimuli in the 
worst case (see Fig. 10). Hence to change the path from vi to x n 
from ordinary to special states (or vice-versa) requires approximately 
6n pulses. Now n is an arbitrary finite number, and hence cannot 
be stored in the finite automaton MC (or the finite automaton CU). 
Von Neumann ingeniously solved this problem by introducing the 
timing loop C 2 . 

The timing loop C 2 is attached to a triple-return counter <£ 2 ; see 
Figure 39. In the terminology of Section 3.4, the loop C 2 is the re- 
sponding organ £2 of the triple-return counter <J> 2 . Three times the 
delay around loop C 2 is approximately the variable part of the needed 
delay 6n. The desired sequence of pulses into u x or V\ is obtained from 
periodic pulsers PP(iWi 4 i 5 i 6 ) turned on for approximately this 
period 6n. Similarly, the triple-return counter <£i of Figure 39 and 
the loop Ci are used to time the sequence of pulses needed to modify 
loop C 2 . Finite sequences of pulses are needed for changing the ends 
of Ci and C 2 and writing in cell x n ; these are easily obtained from 
pulsers. 

Before the lengthening (or shortening) process begins, cell x n has 



216 



THEORY OF SELF-REPRODUCING AUTOMATA 



been read by means of a sequence 10101 and has been left in the state 
io (see Sec. 4.1.5). When von Neumann has finished his design the 
following sequence of operations will be used to lengthen (or shorten) 
the loops Ci and C 2 and write in cell x n : 

(1) Lengthen (or shorten) the timing loop C 2 by means of periodic 
pulsers and pulsers feeding into inputs u 2 and v 2 . The length of 
time each periodic pulser is on is controlled by the triple-return 
counter <f>i to which loop Ci is attached as its responding organ 12. 

(2) Make the following modifications in loop Q and cell x n : 

(a) Lengthen (or shorten) the bottom of loop Ci, gaining access 
to it through cell x n so as not to disturb cells x n -i or x n +i . 

(b) Leave cell x n in state U if a "zero" is to be stored; leave cell 
x n in state i« if a "one" is to be stored. 

(c) Lengthen (or shorten) the top of loop d . 

The stimuli required for this process are fed into inputs Ui and 
Vi from periodic pulsers and plain pulsers. The length of time 
each periodic pulser is left on is controlled by the triple-return 
counter $ 2 to which the loop C 2 is now attached as its responding 
organ 12. 

At the end of these two steps cell x n is left in the desired state, the 
connecting loop Ci passes through cell x n+i on the lengthening option 
and through cell x n -i on the shortening option, and the timing loop 
C 2 is of the same length as loop Ci . 

In Sections 4.2.2-4.2.4 von Neumann derived 31 formulas de- 
scribing the sequences to be fed into loops Ci and C 2 to accomplish 
the purpose just described. These formulas are all summarized in 
Table II on p. 235 below. They are numbered 28'. a through 3l'.h in 
Sections 4.2.2-4.2.4 and renumbered 0.1 through 0.31 in Table II. 

Table III on p. 236 below summarizes the periodic pulsers needed 
to produce sequences repeated approximately n times and the excess 
delays involved in timing these periodic pulsers by means of the 
triple -return counters $i and <£ 2 and the loops Ci and C 2 . 

Table IV on p. 237 below summarizes the pulsers needed to produce 
the fixed sequences called for by the formulas. 

The reader may find it helpful to refer to Tables II-IV while read- 
ing the rest of the present section.] 

4.2.2 Lengthening on L. Consider first the case e s+1 = 1 (lengthen- 
ing). In performing this operation, n (orn + 1) ordinary transmis- 
sion states (i on the upper lines of Ci and C 2 , J? on the lower lines 
of Ci and C 2 ) must be transformed repeatedly into the corresponding 
special transmission states (_i, J:, respectively), and conversely. This 
requires arrangements like (27'. a') and (27 .c) (for the — * class), 



DESIGN OF A TAPE AND ITS CONTROL 



217 



and others similar to these (for the <— class); hence there exists again 
the necessity of measuring delays 6ft (or 6ft plus a fixed amount) 
between the turning on and off of the periodic pulsers that are used. 
While loop Ci is being worked on, these delay measurements can be 
effected with the help of loop C 2 (and the triple-return counter, 
together with suitable fixed delays; cf. Sec. 4.1.7). While loop C 2 
is being worked on, it is plausible to use loop Ci for the same purpose 
(with similar auxiliary equipment). This is a reason why Ci should 
at this time form an uninterrupted loop (from MC to MC), i.e., 
why cell x n should be in state 1© (cf. above). We know that this 
condition is satisfied when our present procedure begins; hence the 
operations on loop C 2 should be the first ones that are undertaken 
(before those on loop Q ) . 

Thus our first task is to lengthen both lines of loop C 2 by one square. 
This is best begun by transforming the entire upper line of C 2 (the 
squares of C 2 under Xq , X\ , • • • , x n — their number is n + 1 ) into 
i, i.e., from its original condition in Figure 32a into that of Figure 
32b. Since these squares are originally °, i.e., ordinary states, special 
stimuli are required. We saw in Section 4.1.6 {cf. the discussion of 
operation (23')} that each transformation requires the stimulus-no- 
stimulus sequence 11011, or, according to Section 4.1.7 {cf. the passage 
from operation (27'. a) to operation (27'. a')}, the stimulus-no-stimu- 
lus sequence 110110. Hence we have this requirement: 

(28'. a) Inject n + 1 sequences 110110 at u 2 . 

Note that this calls for a PP (110110), with a delay 6ft + 6 between 
turning on and off. It is appropriate to attach a triple-return counter 
<£ (cf. Fig. 24) to loop Ci , since the latter is now being used for tim- 
ing. Hence we attach the output d of <£ to V\ and the input c of 3> to 
Wi . Let the delay from d to Vi be 5/, and the delay from w\ to c be 
Si". The delay from Vi through loop Ci to Wi is 2n + 3. In the termi- 
nology of Section 3.4, Q, is the path from d to v x , plus Ci from vi to 
Wi , plus the path from w\ to c. Hence the delay from d through 12 
to c is S/ + (2n + 3) + 5i". Consequently, according to the end of 
Section 3.4 and Figure 24 the delay from a through $ (and 12) to b is 
238 + 3(5/ + (2n + 3) + 5/') = 6ft + (3(5/ + 5/') + 247). This 
exceeds the delay 6ft + 6, called for in connection with rule (28 .a), 
by 3(5/ + 5/ ) + 241, which is a fixed amount (i.e., independent of 
ft). Hence this can be adjusted by suitable fixed delays, that can be 
provided "inside" MC. 

Next we transform the square to the right of the last i (which is 
U) into li, and the square under this (which is also U) into Z-] i.e., 



218 



THEORY OF SELF-REPRODUCING AUTOMATA 



we go from Figure 32b to Figure 32c. The state li is Ti 30 (i.e., Sn 0 ) 
and the state 1 is T020 (i.e., S001). Hence the sequences 1110 and 1001 
of special stimuli are needed, i.e. : 

(28'.b) Inject the sequence 11101001 at u 2 . 

Now the lower line of the timing loop C 2 is in its desired condition, 
and there remains the task of dealing with the upper line. This begins 
by transforming all of it, except its last square (i.e., the squares under 
£0 , Xi , • • • , x n — their number is n + 1 ) into state t. This loop C 2 is 
transformed from its condition in Figure 32c into that of Figure 32d. 
Since the cells to be changed are in state ordinary stimuli are re- 
quired. We saw in Section 4.1.6 {cf. the discussion of operation (25')} 
that each transformation requires the stimulus-no-stimulus sequence 
110000. Hence we have this requirement: 

(28'.c) Inject n + 1 sequences 110000 at v 2 . 

This calls for a PP (110000), with a delay 6n + 6 between turning 
on and off. The same triple-return counter <i>, attached to Ci , that we 
used in connection with operation (28'.a), can take care of this delay 
too. The details are the same as in the discussion after operation 
(28'. a); i.e., the delay from a to b {cf. that discussion and Fig. 24} 
exceeds the desired delay by 3(5/ + 6/') + 241, which, being fixed, 
calls for adjustments that can be provided "inside" MC. 

Finally, we transform the square to the right of the last t, (which 
is li) into i<>; i.e., we go from Figure 32d to Figure 32e. The state 1<> 
is T030 (i.e., S010). Hence the sequence 11010 of ordinary stimuli is 
needed : 

(28'.d) Inject the sequence 11010 at v 2 . 

This completes the lengthening of the timing loop C 2 . We must now 
perform the corresponding operation on the connecting loop Ci . 

The state of loop Ci is shown in Figure 33a. It is best to begin the 
lengthening operation on the lower line of Q , the access to it being 
obtained through the upper line of Ci and the state 1 0 at the place of 
x n . We transform accordingly the entire upper line, except its last 
square (the squares above Xo , X\ , • • • , x n -r — their number is n) 
into L. That is, we transform the upper line of Ci from its condition 
in Figure 33a into that of Figure 33b. Since these squares are i (i.e., 
ordinary states), special stimuli are required. Just as for operation 
(27'. a') {or operation (28'.a)}, each transformation requires the 
sequence 110110. Hence: 

(29'. a) Inject n sequences 110110 at u x . 



DESIGN OF A TAPE AND ITS CONTROL 



219 



This calls for a PP(llOllO), with a delay 6n between turning on 
and turning off. The timing can be achieved with a triple-return 
counter tied to Vz , w 2 (with its d, c, respectively), precisely as in the 
discussion after operation (27'.a'). However, the length of C 2 is 
now 1 unit greater than it was there; hence the delay through C 2 
is increased by 2, and the delay 3 times through C 2 is increased by 6. 
Hence the fixed excess delay, which must be compensated by adjust- 
ments "inside" the memory control MC, is increased by 6; i.e., it 
is now 3 (5/ + 5 2 ") + 250. 

Next we transform the square to the right of the last i. (which is 
lo) into li, the square under this (which is also lo) also into H, 
the square under this (which is 1) into i (these three are the square 
above, at, and under the place of x n ), and the square to the right of 
this (which is U) into l_; i.e., we go from Figure 33b to Figure 33c. 
The state li is Ti 30 (i.e., Su o), I i s Ti 00 ( i. e., Son ), 1 is T 020 (i.e., 
Sooi). Hence the sequences 11110, 11110, 11011, 1001 of special 
stimuli are needed, i.e.: 



Now we may restore the squares at and below the place of x n to 
their final condition. We must again get in through the upper line. 
Hence we transform all of it, except its last square (the squares above 
Xq , Xi , • • • , x n -i — their number is n) into °; i.e., we go from Figure 
33c to Figure 33d. Since the affected part of the upper line now con- 
sists of special transmission states, ordinary stimuli will do this. Just 
as for operation (27'.c), each transformation requires the sequence 



This calls for a PP (110000), with a delay 6n between turning on 
and turning off. It can be handled by the same timing arrangements 
used for operation (29'. a). 

Next we transform the square to the right of the last i (which is 
1 1 ) into 1 o, the square under this (which is also I i ) into i o, and the 
square under this (which is i) into J_ (these three are the squares 
above, at, and under the place of x n ); i.e., we go from Figure 33d to 
Figure 33e. Th e state |o is T 03 o (i.e., Soio), 1 is T 020 (i.e., Sooi). Hence 
the sequences 11010, 11010, 11001 of ordinary stimuli are needed, 
i.e.: 





110000, i.e.: 
(29'.c) 



Inject n sequences 110000 at Vi . 



(29'.d) Inject the sequence 110101101011001 at v x . 



220 



THEORY OF SELF-REPRODUCING AUTOMATA 



Now we can carry out the lengthening of the upper line of Ci . We 
begin by transforming the entire upper line (the squares above xo , 
x\ , • • • , x n — their number is n + 1) into i; i.e., we go from Figure 
33e to Figure 33f. Since the upper line now consists of ordinary trans- 
mission states, special stimuli will do this. Just as for operation 
(29'. a), each transformation requires the sequence 110110, i.e.: 

(29\ e) Inject n + 1 sequences 110110 at u x . 

This calls for a PP(llOllO), with a delay 6n + 6 between turning 
on and turning off. It can be handled by the same timing arrange- 
ments used for operation (29 .a), except that the delay to be timed 
is now longer by 6. Hence the fixed excess delay, which must be 
compensated by adjustments "inside" the memory control MC, is 
decreased by 6; i.e., it is now 3 (82 + 82 ) + 244. 

Next we transform the square to the right of the last ! (which is 
U) into 10; i.e., we go from Figure 33f to Figure 33g. The state l« is 
T030 (i.e., S010). Hence the sequence 1010 of special stimuli is needed, 
i.e.: 

(29'. f) Inject the sequence 1010 at u x . 

Finally we transform the balance of the upper line (the squares 
above xq , Xi , • • • , x n — their number is n + 1) into °>; i.e., we go 
from Figure 33g to Figure 33h. Since the affected part of the upper 
line now consists of special transmission states, ordinary stimuli will 
do this. Just as for operation (27'.c), each transformation requires 
the sequence 110000, i.e.: 

(29'.g) Inject n + 1 sequences 110000 at v x . 

This calls for a PP (110000), with a delay 6n + 6 between turning 
on and turning off. It can be handled by the same timing arrange- 
ments used for operation (29 .e). 

This completes the lengthening of the connecting loop Q , i.e., 
the entire first case, e s+1 = 1. 

4.2.3 Shortening on L. Consider now the case e s+1 = — 1 (shorten- 
ing). The treatment runs, in its main outline, parallel to that of the 
first case. Hence the operations on C 2 should again be undertaken 
first (before those on Ci). 

Thus our first task is to shorten both lines of C 2 by one square. 
This is best begun by transforming the entire upper line of C2 , except 
its last square (i.e., the squares under xo , Xi , • • • , x n -i — their num- 
ber is n) into i.e., from its original condition in Figure 32a into 
that of Figure 34a. Since these squares are originally t (i.e., ordinary 



DESIGN OF A TAPE AND ITS CONTROL 



221 



states), special stimuli will do this. Just as for transformation (29'.a), 
each transformation requires the sequence 110110, i.e.: 

(30'.a) Inject n sequences 110110 at u 2 . 

This calls for a PP(llOllO), with a delay 6n between the turning 
on and turning off. It can be handled by the same timing arrange- 
ments that take care of operation (28 .a), except that the delay to 
be timed is now shorter by 6. Hence the fixed excess delay, which 
must be compensated by adjustments "inside" the memory control 
MC, is increased by 6; i.e., it is now 3(5/ + O + 247. 

Next we transform the square to the right of the last i (which is 
lo) into |i, and the square under this (which is 1) into U; i.e., we go 
from Figure 34a to Figure 34b. The state U is Ti 30 (i.e., Sn 0 ). Hence 
the sequences 11110 and 1 of special stimuli are needed, i.e.: 

(30'.b) Inject the sequence 111101 at u 2 . 

Now we restore the upper line of C2 , except its last square (i.e., 
the squares under x 0 , X\ , • • • , x n -i — their number is n) into °; i.e., 
we go from Figure 34b to Figure 34c. Since these squares are now 
special transmission states, ordinary stimuli will do this just as for 
operation (27'.c). Hence each transformation requires the sequence 
110000, i.e.: 

(30'.c) Inject n sequences 110000 at v 2 . 

This calls for a PP (110000), with a delay 6n between turning on 
and turning off. It can be handled with the same timing arrange- 
ments that take care of operation (30 .a). 

Next we transform the square to the right of the last t (which is 
i 1) into U; i.e., we go from Figure 34c to Figure 34d. For this a single 
ordinary stimulus is needed, i.e. : 

(30'.d) Inject a single stimulus at v 2 . 

At this point the shortening of C 2 is effected, since the two right- 
most squares have been transformed into U's, but the rightmost 
square of what is left of the upper line (the one under av-i) is not in 
the desired state (it is ° instead of lo). We go on to correct this. 

We transform the upper line, except its last square (the squares 
under — their number is n — 1) into i; i.e., we go 
from Figure 34d to Figure 34e. Since these squares are now ordinary 
transmission states, special stimuli will do this. Just as for operation 
(29'. a), each transformation requires the sequence 110110, i.e.: 

(30'.e) Inject n — 1 sequences 110110 at u 2 . 



222 



THEORY OF SELF-REPRODUCING AUTOMATA 



This calls for a PP(IlOllO), with a delay On — (> between turning 
on and turning off. It can be handled by the same timing arrange- 
ments that take care of operation (30'.a), except that the delay to 
be timed is now shorter by 6. Hence the fixed excess delay, which 
must be compensated by adjustments "inside" MC, is increased by 
6; i.e., it is now 3(5/ + Si") + 253. 

Next we transform the square to the right of the last L (which is 
°) into 1»; i.e., we go from Figure 34e to Figure 34f. The state lo is 
Toso (i.e., Soio). Hence the sequence 11010 of special stimuli is needed, 
i.e.: 

(30'.f) Inject the sequence 11010 at u 2 . 

Finally we transform the balance of the upper line (the squares 
under Xq , X\ , • • • , x n -2 — their number is n — 1) into °>; i.e., we go 
from Figure 34f to Figure 34g. Since the affected part of the upper 
line consists now of special transmission states, ordinary stimuli will 
do this. Just as for operation (27'. c), each transformation requires 
the sequence 110000, i.e.: 

(30'.g) Inject n — 1 sequences 110000 at v 2 . 

This calls for a PP (110000), with a delay 6n — 6 between turning 
on and turning off. It can be handled by the same timing arrange- 
ments that take care of operation (30 .e). 

This completes the shortening of C 2 . We must now perform the 
corresponding operation on Ci . 

The state of Q is shown in Figure 33a. It is best to begin the short- 
ening operation (just as we did the lengthening operation) on the 
lower line of Ci , the access to it being obtained through the upper 
line of Q and the 1<> at the place of x n . We transform accordingly 
first the entire upper line, except its last square (the squares above 
Xq , Xi , • • • , x n -i — their number is n) into i; i.e., from its condition 
in Figure 33a into that in Figure 35a. Since these squares are ° (i.e., 
ordinary states), special stimuli are required. Just as for operation 
(29'. a), each transformation requires the sequence 110110, i.e.: 

(3l'.a) Inject n sequences 110110 at %h . 

This calls for a PP (110110), with a delay 6n between turning on 
and turning off. The timing can be achieved with the same means that 
were used after operation (29'.a). However, the length of C 2 is now 2 
less than it was there; hence the delay through C 2 is decreased by 4 
and the delay three times through C 2 is decreased by 12. Therefore, 
the fixed excess delay, which must be compensated by adjustments 



DESIGN OF A TAPE AND ITS CONTROL 



223 



"inside" the memory control MC, is decreased by 12; i.e., it is 3 (S 2 ' + 
5 2 ") + 238. 

Next we transform the square to the right of the last i (which is 

10) into li, the square under this (which is also io) into U, and the 
square under this (which is 1) into U (these three are the squares 
above, at, and under the place of x n ); i.e., we go from Figure 35a to 
Figure 35b. The state li is Ti 3 o (i.e., S no ). Hence the sequences 
11110, 11110, 1 of special stimuli are needed, i.e.: 

(3l'.b) Inject the sequence 11110111101 at u\ . 

Now we may restore the square at the place of x n to its final con- 
dition. We must again get in through the upper line. Hence we trans- 
form first all of it, except its last square (the squares above xo , 
Xi , • • • , x n -i — their number is n) into ?t; i.e., we go from Figure 35b 
to Figure 35c. Since the affected part of the upper line now consists 
of special transmission states, ordinary stimuli will do this. Just as 
for operation (27'.c), each transformation requires the sequence 
110000, i.e.: 

(3l'.c) Inject n sequences 110000 at Vi . 

This calls for a PP (110000), with a delay 6n between turning on 
and turning off. It can be handled by the same timing arrangements 
that take care of operation (31 .a). 

Next we transform the square to the right of the last t (which is 

1 1 ) into i o, and the square under this (which is also i i ) also into i o 
(these two are the squares above and at the place of x n ); i.e., we go 
from Figure 35c to Figure 35d. The state io is T030 (i.e., S010). Hence 
the sequences 11010, 11010 of ordinary stimuli are needed; i.e.: 

(3l'.d) Inject the sequence 1101011010 at v x . 

Now we can carry out the shortening of the upper line of Ci . We 
begin by transforming the entire upper line, except its last square 
(the squares above xq , Xi , • • • , x n -i — their number is n) into i; i.e., 
we go from Figure 35d to Figure 35e. Since the affected part of the 
upper line consists now of ordinary states, special stimuli will do this. 
Just as for operation (29'.a), each transformation requires the se- 
quence 110110, i.e.: 

(3l'.e) Inject n sequences 110110 at Ui . 

This calls for a PP (110110), with a delay 6n between turning on 
and turning off. It can be handled by the same timing arrangements 
that take care of operation (31 '.a). 



224 



THEORY OF SELF-REPRODUCING AUTOMATA 



Next we transform the square to the right of the last i (which is 
lo) into U; i.e., we go from Figure 35e to Figure 35f. For this a single 
special stimulus is needed, i.e.: 

(31 i) Inject a single stimulus at u t . 

At this point the shortening of G is effected, since the two right- 
most squares have been transformed into IPs, but the upper line is 
not in its desired state. We go on to correct this. 

We transform the upper line, except its last square (the squares 
above xq , X\ , • • • , # n _2 — their number is n — 1) into i; i.e., we go 
from Figure 35f to Figure 35g. Since these squares are now special 
transmission states, ordinary stimuli will do this. Just as for operation 
(27'. c), each transformation requires the sequence 110000, i.e.: 

(3l'.g) Inject n — 1 sequences 110000 at vi . 

This calls for a PP (110000), with a delay 6n — 6 between turning 
on and turning off. It can be handled by the same timing arrange- 
ments that take care of operation (31 .a), except that the delay to be 
timed is now shorter by 6. Hence the fixed excess delay, which must 
be compensated by adjustments "inside" the memory control MC, is 
increased by G; i.e., it is now 3(«2 + «2 ) + 244. 

Finally, we transform the square to the right of the last ^ (which 
is A) into 10; i.e., we go from Figure 35g to Figure 35h. The state io 
is T030 (i.e., Soio). Hence the sequence 11010 of ordinary stimuli is 
needed, i.e.: 

(3l'.h) Inject the sequence 11010 at Vi . 

This completes the shortening of the connecting loop Ci , i.e., the 
entire second case, e s+1 = — 1. 

4.2.4 Altering x n in L. We must now reconsider the procedure of 
altering x n , as discussed in Sections 4.1.6 and 4.1.7. 

The assumption underlying the procedure of Sections 4.1.6 and 
4.1.7 was that the altering of x n follows immediately upon the "read- 
ing" of x n . However, in Sections 4.2.1-4.2.3 we carried out the length- 
ening or the shortening of Ci and C 2 as if it occurred between these 
two, i.e., as if this lengthening or shortening followed immediately 
upon the reading of x n . These two assumptions are in conflict, and 
we now stipulate that the last mentioned assumption (i.e., the 
assumption of Sees. 4.2.1 and 4.2.3) is valid. Hence Sections 4.1.6- 
4.1.7 must be reconsidered. 

The most natural procedure would be to review the changes that 
the lengthening or shortening has produced in the relevant structures 



DESIGN OF A TAPE AND ITS CONTROL 



225 



— i.e., to Ci , on which Sections 4.1.6-4.1.7 operate, and to C 2 , 
which is used to time these operations — and to correct Sections 4.1.6- 
4.1.7 to account for these changes. However, it is simpler to analyze 
the procedures for the lengthening and the shortening themselves, 
and to see in each case where the operations of Sections 4.1.6-4.1.7, 
or equivalent ones, fit in best. It turns out that these insertions can 
be effected with the help of considerably simplified variants of Sec- 
tions 4.1.6-4.1.7. 

The conclusion reached at the beginning of Section 4.1.6 still holds: 
in the case = 1 nothing need be done, while in the case i^ -1 = 
0, x n must be transferred from !» to U. Hence we need to consider 
the latter case only. However, we must discuss the two main alterna- 
tives (e s+1 = 1, lengthening, and e s+1 = — 1, shortening) separately. 

Consider first the case e s+1 = 1 (lengthening). We must decide 
where to insert the process of altering x n from I o into U into the evolu- 
tion of Ci according to Figures 33a-33h. It is easily seen that it fits 
best after Figure 33e, involving a change of the evolution through 
Figures 33f-33h, i.e., of the steps (29 , .e)-(29 , .g). 

We modify the passage from Figure 33e to Figure 33f by transform- 
ing one square less, i.e., into the passage from Figure 33e to Figure 
36a. This reduces the number of iterations in operation (29'. e) by 
one: 

(29'.e / ) Inject n sequences 110110 at Ui . 

This calls for a PP (110110) with a delay 6n between turning on 
and turning off. It can be handled by the same timing arrangements 
that take care of operation (29'. a). 

Next we transform the square to the right of the last -i (which is 
lo) into 1 1, and the square under this (which is also 1«) into U (these 
two are the squares above and at the place of x n )\ i.e., we go from 
Figure 36a to Figure 36b. The state ii is Ti 3 o (i.e., Sno). Hence the 
sequences 11110, 1 of special stimuli are needed, i.e.: 

(29'.f') Inject the sequence 111101 at U\ . 

Now we transform the entire upper line (the squares above xo , 
Xi , • • • , x n — their number is n + 1) into -S; i.e., we go from Figure 
36b to Figure 36c. Since the upper line consists of special transmission 
states, ordinary stimuli will do this. Just as for operation (27'.c), 
each transformation requires the sequence 110000, i.e.: 

(29'.g') Inject n + 1 sequences 110000 at v x . 



This calls for a PP (110000) with a delay 6n + 6 between turning 



226 



THEORY OF SELF-REPRODUCING AUTOMATA 



on and turning off. It can be handled by the same timing arrangements 
that take care of operation (29'.e). 

Finally we transform the square to the right of the last -°* (which 
is U) into 10; i.e., we go from Figure 36c to Figure 36d. The state lo 
is T030 (i.e., Soio). Hence the sequence 1010 of ordinary stimuli is 
needed, i.e.: 

(29'.h') Inject the sequence 1010 at Vi . 

This completes the discussion of the first case, e 8+1 = 1. 

Consider now the case e s+1 = — 1 (shortening). We must decide 
where to insert the process of altering io into U into the procedure of 
Section 4.3.2, i.e., into the evolution of Ci according to Figures 33a 
and 35a-35h. It is easily seen that it fits best after Figure 35c, but it 
affects only the step from there to Figure 35d, i.e., the step (3l'.d). 

Indeed, io at the place of x n was made from U; hence it suffices 
to make U instead. That is, the second sequence mentioned before 
step (3l'.d) {it is 11010} must be replaced by a single stimulus. Hence 
operation (3l'.d) is replaced by this: 

(3l'.d') Inject the sequence 110101 at vi . 

Now the io at the place of x n is replaced by U in Figure 35d. After 
this the evolution through Figures 35e-35h, i.e., the steps (3l'.e)- 
(?>\ r .Yi) can go on unchanged. In all Figures 35e-35h the only altera- 
tion will be the same replacement of the 1 o at the place of x n by U, 
and this has no effect on the operations mentioned above. 

This completes the discussion of the second case, e s+1 = —1. 

4.3 The Memory Control MC 

[ The organization and operation of MC. Figure 37 is a 
schematic diagram of the memory control MC and the organs it 
controls: the linear array L, the connecting loop Ci , and the timing 
loop C 2 . This figure is not drawn to scale; MC is actually 547 cells 
high and 87 cells wide. We will first describe the organization of MC, 
and then we will explain how MC operates. 

The most important parts of MC are the read-write-erase unit 
RWE and its control RWEC. The unit RWE is shown in Figure 39 
and developed in Section 4.3.3. The organs marked "0.6," "0.10," 
etc., in Figure 39 are the pulsers of Tables III and IV. The pulser 
marked "0.0" is a P (10101 ) which sends the sequence 10101 into 
input Vi of loop Ci for the purpose of reading x n ; the result which 
emerges from output Wi goes to the 1 vs. 10101 discriminator ^. 
The triple-return counter <f>i has loop Ci as its secondary organ £2, 



DESIGN OF A TAPE AND ITS CONTROL 



227 



and the triple-return counter 3> 2 has loop C 2 as its secondary organ. 
The stimuli into RWE from the left come from decoders in CCi , 
and the stimuli going from RWE to the left go into pulsers in CCi . 
The unit CQ of RWE is part of the coded channel of MC. 

The coded channel of MC consists of CCi , CC 2 , CC 3 , X, Z, and 
the main channel. The main channel goes from the solid circle to the 
solid square. The unit CC 2 contains decoders whose inputs come from 
the main channel and whose outputs go into RWEC. The unit CC 3 
contains pulsers whose inputs come from RWEC and whose outputs 
go to the main channel. The units X, Z, and CCi contain both de- 
coders receiving their inputs from the main channel and pulsers 
feeding their outputs into the main channel. 

The read-write-erase control RWEC is shown in Figure 41 and de- 
veloped in Section 4.3.5. It contains 16 control units CO; each CO 
contains a PP(D which is active while that CO is in control (see 
Fig. 40 and Sec. 4.3.4). The RWEC also contains four PP(T) which 
are used to store the bit which is to be written in cell x n . 

The inputs and outputs of RWEC are labeled to correspond to the 
inputs and outputs of RWE. For example, a stimulus from vi-1 goes 
into a pulser of CC 3 which sends a coded sequence into the main 
channel; this sequence is recognized by a decoder of CCi which sends 
a pulse into input vi-1 of RWE, thereby stimulating the pulser 
labeled "0.0" to send the sequence 10101 into V\ for reading cell x n . 

The area Y of MC is used only to transfer stimuli from outputs 
02 , 03 , and o 5 of the constructing unit CU to pulsers in X, and to 
transfer a stimulus from a decoder in X to input % of CU. 

The area W has the following function. Each periodic pulser of 
RWE is to be on for about 6n units of time, where x n is the cell of 
L which is being scanned. The triple-return counter $i (with d as 
its secondary organ) is to supply this delay when a periodic pulser 
is feeding loop C 2 , and the triple-return counter <I> 2 (with C 2 as its 
secondary organ) is to supply this delay when a periodic pulser is 
feeding loop Ci . But time is lost between each of these triple-return 
counters and its secondary organs. Moreover, the output of a triple- 
return counter (<fv6 or <f> 2 -b) cannot be used to stop a periodic 
pulser until it passes through CCi , the main channel, CC 2 , RWEC, 
CC 3 , the main channel again, and CCi again. Altogether about 2000 
units of time are lost in these two ways. Consequently, a periodic 
pulser of RWE must be turned on about 2000 units of time later than 
the triple-return counter associated with it. The exact amount of 
delay needed varies with each of the 16 control organs CO of RWEC. 
Von Neumann solved the problem by dividing the delay area W into 



228 



THEORY OF SELF-REPRODUCING AUTOMATA 



four parts and associating one part with each periodic pulser of RWE. 
Most of the delay needed is obtained by sending a pulse through this 
delay area. The variable part of the delay is obtained in the delay 
area D of each CO (Fig. 40). 

We will next explain briefly how the memory control MC functions 
under the general direction of the constructing unit CU (Figs. 37 and 
50). The basic operation of reading, writing, and lengthening (or 
shortening) the loops Q and C 2 is carried out in two stages. First, 
CU sends a signal to MC telling it to read; MC then reads cell x n 
and sends the result to CU. Second, CU sends signals to MC telling 
it what to write in cell x n and whether to lengthen or shorten the loops 
Ci and C2 ; MC then executes these instructions and sends a comple- 
tion signal to CU. 

The first stage of the basic memory operation begins when a stimu- 
lus emerges from output o x of CU and enters the top of RWEC. It 
enters CC 3 at v x • 1 , is coded by a pulser of CC 3 , enters the main chan- 
nel, enters CCi , is decoded by a decoder of CCi , enters RWE at 
Vi'l, and stimulates the pulser P (10101) of RWE; this pulser is 
labeled "0.0" in Figure 39a. The sequence 10101 enters Q at Vi and 
goes down the upper part of Ci . If cell x n is in state U (representing 
a "zero") the sequence 10101 changes x n to state io and a 1 emerges 
from exit Wi of Ci ; if cell x n is in state Jo (representing a "one"), 
the complete sequence 10101 emerges from exit Wi of Ci . In both 
cases the output goes to the 1 vs. 10101 discriminator ^ of RWE. 
If x n stored "zero," a stimulus emerges from output b of and goes 
via ty-b, CCi , the main channel, CC 2 , Sf'-fr, and RWEC (near the 
top) to the input i\ of CU. If x n stored "one," a stimulus emerges 
from output c of and goes via ■ c, CCi , the main channel, CC 2 , 
^•c, and RWEC (near the top) to the input i 2 of CU. The unit CU 
now knows the contents of cell x n . This completes the first stage of 
the basic memory operation of MC. 

To start the second stage of the basic memory operation of MC 
the constructing unit CU sends the following signals to the top of 
MC: 

(1) A stimulus from output o 2 if "zero" is to be written in x n ; a 
stimulus from output o 3 if "one" is to be written in x n . 

(2) A stimulus from output o 4 if loops Ci and C 2 are to be lengthened; 
a stimulus from output o 5 if loops Ci and C 2 are to be shortened. 

We will trace the effects of these stimuli separately. 

A stimulus from o 2 passes through area Y and stimulates a pulser 
in area X. This pulser emits a coded sequence which travels along 
the main channel and enters two decoders in CC 2 . These decoders 



DESIGN OF A TAPE AND ITS CONTROL 



229 



then turn on two PP(1) of RWEC. One of these PP(1) is shown at 
the top of Figure 41b; it is used to control the writing operation if 
loops Ci and C 2 are lengthened. The other PP(T) is the upper PP(T) 
of Figure 41 e; it is used to control the writing operation if loops 
Ci and C 2 are shortened. Similarly, a stimulus from_o 3 causes two 
PP(T) of RWEC to be activated. One of these PP(1) is shown in 
Figure 41c; it is used to control the writing operation if loops Ci and 
C 2 are lengthened. The other PP(1) turned on by the signal from 
o z is the lower PP(T) of Figure 41e; it is used to control the writing 
operation if loops Ci and C 2 are shortened. Both PP(T) turned on by 
signals from o 2 (or o 3 ) are turned off at the end of the basic memory 
operation by the completion signal that goes to input i 3 of CU. 

The signals from output o 4 of CU (signifying lengthening) and 
output o 6 of CU (signifying shortening) enter MC in different ways. 
A stimulus from o 4 enters the top of RWEC and turns on the first 
control organ, namely COi . After control organ COi directs RWE 
to carry out certain operations, it is turned off and the control organ 
C0 2 is turned on. Then C0 3 and C0 4 are used in a similar way. At 
this point there is a branch. If a "zero" is to be written in x n , control 
organs C0 5 and CO e are used; while if a "one" is to be written in 
x n , control organs COy and CO s are used. When either CO e or CO s 
is finished, a pulse is emitted from the output i% below it. This pulse 
travels via CC3 and the main channel to do two things: first, it enters 
area X, passes through area Y, and enters input U of CU, signifying 
that the basic operation of MC is finished; second, it enters CC 2 at 
four places to turn off the two PP(T) of RWEC that were storing 
the bit to be written in x n . 

The signal from output o b of CU (signifying shortening) passes 
through Y, X, the main channel, and CC 2 , and enters the control 
organ C0 9 . Control organs CO9 , CO10 , COn , CO J2 , CO^ , and 
CO14 are used in that order. The output P from COh is gated by the 
PP(1) of Figure 41e. If a "zero" is to be written in x n , the signal 
Vi'4 is used; while if a "one" is to be written in x n , the signal Vi-5 
is used. In either case the control organs CO15 and COi 6 are then used. 
When COie is finished, a pulse is emitted from the output i z below it. 
As in the case of lengthening, this pulse does two things: it turns 
off the PP(T) of RWEC that were storing the bit to be written in 
x n , and it enters input i% of CU to signify that the basic memory 
operation of MC is finished. At the end of a basic operation the 
memory control MC is left in its original state. 

Table V summarizes the effect of the control organs of RWEC on 
the pulsers and periodic pulsers of RWE. We will illustrate how these 



230 



THEORY OF SELF-REPRODUCING AUTOMATA 



control organs work by tracing the action of COi . The design of a 
CO is given in Figure 40; all CO are the same except for the length of 
the delay path (B. The particular control organ COi is located at the 
top of RWEC (Fig. 41a). The state of the connecting loop Ci and 
the linear array L at the time COi takes control is shown in Figure 
33a; loop Ci passes through cell x n and consists entirely of ordinary 
transmission states. The state of the timing loop C 2 at the time COi 
takes control is shown in Figure 32a; the effect of COi on C 2 is shown 
in Figures 32b and 32c. 

The action of COi is as follows. The stimulus from output o 4 of 
CU (signifying lengthening) enters input a of COi and accomplishes 
three tasks. First, it goes via the top and left side of COi to enter 
input a+ of the alternate PP(T) of COi , thereby activating this 
periodic pulser, which will be on while COi is in control. Second, the 
input a of COi goes from output c of COi into input $i • a of CC 3 ; 
from there via CC 3 , the main channel, and Cd to output <3va of 
CCi ; and from there into input a of the triple-return counter <f>i of 
RWE (Fig. 39a), thereby starting this triple-return counter. Third, 
the input a of COi goes via the delay path (B of COi to output b of 
COi ; from there to input u 2 -a + of CC 3 ; thence via CC 3 , the main 
channel, and Z into the delay area W; through a portion of W and 
back through Z to the main channel; through CCi to output u 2 -a+ 
of CCi ; and from there to input a+ of the periodic pulser PP (110110) 
in the lower part of RWE (Fig. 39b), thereby starting this periodic 
pulser. The sequence 110110 repeatedly enters input u 2 of loop C 2 . 
Each 110110 converts a cell of the upper part of C 2 from ^ to in 
the following way: T kills _o to U, 1011 changes U to according to the 
direct process of Figure 10, and the final 0 has no effect. Hence n 
occurrences of the sequence 110110 into u 2 will convert the upper part 
of loop C 2 from the path shown in Figure 32a to the path shown in 
Figure 32b. 

The production of n sequences 110110 is controlled by the triple- 
return counter <f>i of RWE, which uses the loop Ci as its secondary 
organ U. The output from b of <$>i is used to turn off the PP (110110) 
of Figure 39b in the following way. The output from b of <$i enters 
CCi at goes through CCi to the main channel and to CC 3 , and 
enters input d of each of the control organs COi , C0 2 , C0 9 , COio , 
COn , and CO i2 . All these CO are inactive except for COi , so in 
every case except this one the effect of the input to d to a CO is only 
to attempt to turn off an alternate PP(1) which is already off. This 
does no harm, as we saw at the end of Section 3.2.2. In the case of 
COi , the input to d turns its PP(1) off and also passes through the 



DESIGN OF A TAPE AND ITS CONTROL 



231 



confluent cell at the lower right of COi and exits at e and /. The 
"off" signal into a_ of PP(T) causes a stop at 6 too late to block the 
emission from e and /. 

The output from e of COi enters CC 3 at w 2 -a_ ; goes through 
CC 3 , the main channel, and CCi ; l eaves C Ci at u 2 • a_ and enters 
PP(llOllO) at a_ . In this way PP (110110) is shut off after it has 
emitted exactly n sequences 110110. 

The output from / of COi goes to two different places. First, it enters 
CC 3 at u 2 - 1 ; it passes through CC 3 , the main channel, and CCi ; and 
finally it leaves CCi 2Xu 2 -l and enters the pulser P (II 101001 ), which 
is marked "0.2" in Figure 39b. The sequence 11101001 then enters 
input u 2 of C 2 . The first half (1110) of this sequence changes a U to 
ii, while the last half (1001) of this sequence changes the next U to 

Hence loop C 2 is left in the state shown in Figure 32c. 

Second, the output from / of COi enters input a of C0 2 , thereby 
starting the operation controlled by C0 2 . The periodic pulser of 
RWE which is controlled by C0 2 produces the result given in Figure 
32d, and the pulser of RWE which is controlled by C0 2 produces the 
result shown in Figure 32e. Thus the control organs CO x and C0 2 
together bring about the lengthening of the timing loop C 2 . The 
control organ C0 2 then passes control to C0 3 .] 

4.3.2 Detailed discussion of the functioning of MC. We enumerated 
in Section 4.1.4 the specific functions of the memory control MC, 
namely: a purely descriptive static statement in (1), the start in (2), 
the substantive operations in (3) {reading x n ), (6) {altering x n }, and 
(7) {moving x n , i.e., n = n\ ; and the completion in (8). {All these 
numbers refer to the listing in Sec. 4.1.4.} Postulate (1) calls for no 
action. The implementation of (2) {responding to a starting signal 
from CU to MC} and of (8) {delivering a completion signal from 
MC to CU} , is easy; we will attach these where they obviously belong, 
at the beginning of (3) and at the end of (8), respectively. Carrying 
out the other substantive operations (3), (6), and (7) is, of course, 
much more complicated; Sections 4.1.5-4.2.4 gave an outline of this. 
Now that this outline is completed, we must perform the necessary 
constructions in detail. In other words, we must fill in specific com- 
ponentry to execute the operations enumerated and discussed in 
Sections 4.1.5-4.2.4. We proceed to do this in what follows. 

We discussed operation (3) first in Section 4.1.5. We then took up 
operation (6) in Sections 4.1.6-4.1.7, but this was only a preliminary 
discussion; the final form was developed in Section 4.2.4. This form 
was actually meant to be meshed with the processing of operation 
(7). The latter was discussed in Sections 4.2.1-4.2.2. Thus the com- 



232 



THEORY OF SELF-REPRODUCING AUTOMATA 



plete discussion must begin with Section 4.1.5 and then continue with 
Sections 4.2.1-4.2.3, the latter being combined with Section 4.2.4. 

Section 4.1.5 requires us to inject the sequence 10101 (with a 
guaranteed 00 thereafter) at vi and to feed the output of w\ into the 
input a of the T vs. 10101 discriminator ^ of Figure 25. The outputs 
b and c of ^ then indicate T (i.e., x n = U and £ n = 0) and 10101 (i.e., 
x n = io, and = 1), respectively. 

Sections 4.2.1-4.2.3, combined with Section 4.2.4, call for more 
complicated arrangements. Section 4.2.1 does not describe any 
specific operations. Section 4.2.2 postulates a definite sequence of 
operations: (28'.a)-(28'.d), and (29'.a)-(29'.g). Section 4.2.4 re- 
places operations (29'.e)-(29'.g) by operations (29'.e')-(29'.h'). 
Section 4.2.3 postulates these operations : (30 / .a)-(30 / .g) and (3l'.a)- 
(31 / .h). Section 4.2.4 replaces operation (3l'.d) by operation (3l'.d'). 
Note that Sections 4.2.2 and 4.2.3 are alternative, depending on the 
value of e s+1 (whether e s+1 = 1 or —1, respectively), i.e., on the first 
response of CU according to postulate (5) in Section 4.1.4. The 
insertion of Section 4.2.4 into Sections 4.2.2 and 4.2.3 is also condi- 
tional, depending on the value of £ n s+1 (it takes place only if £ n s+1 = 0; 
cf. the beginning of Sec. 4.1.6), i.e., on the second response of CU 
according to postulate (5) in Section 4.1.4. 

In order to describe the communications between CU and MC, and 
in view of the fact that we will not describe here in detail the internal 
functioning of CU, we must define certain specific inputs and outputs 
of CU. 

The inputs of CU correspond to the signals that go from MC to 
CU according to postulates (l)-(9) in Section 4.1.4. These are the 
following : 

(11) A signal according to (3), indicating that £ n s (n = n) has been 
read by MC and found to be 0; i.e., x n = U. 

(1 2 ) A signal according to (3), indicating that £ n s (n = n) has been 
read by MC and found to be 1; i.e., x n = i°. 

(z 3 ) The completion signal of MC according to (8). 

For each one of these signals the symbol in parentheses which 
precedes it is the designation of the input of CU to which it is to go. 
That is, these inputs are . 

The outputs of CU correspond to the signals that go from CU to 
MC according to postulates (l)-(9) in Section 4.1.4. These are the 
following : 

(01) The start signal to MC according to (2). 

(0 2 ) A signal according to (5), indicating that £ n s+1 (n = n) has 
been formed by CU and found to be 0. 



DESIGN OF A TAPE AND ITS CONTROL 233 

(a 3 ) A signal according to (5), indicating that (n = n) has 

been formed by CU and found to be 1. 

(0 4 ) A signal according to (5), indicating that e s+1 has been formed 
by CU and found to be 1. 

(0 5 ) A signal according to (5), indicating that e s+1 has been formed 
by CU and found to be —1. 

For each one of these signals the symbol in parentheses which 
precedes it is a designation of the output of CU from which it is to 
come. That is, these outputs are 01-05 . 

Our discussion above further shows that the signals from 04 and 
05 are immediately effective on MC: they determine whether MC 
enters the action cycle (28'.a)-(28'.d), (29'.a)- (29'.g) {with a 
possible replacement of (29'.e)-(29'.g) by (29'.e')-(29'.h')} or the 
action cycle (30'.a)-(30'.g), (3l'.a)-(3l'.h) {with a possible re- 
placement of (3l'.d) by (3l'.d')}. Obviously, the signal from 01, 
too, is immediately effective on MC: it is a start signal. 

On the other hand, the signals from 0 2 and 03 are not immediately 
effective on MC: they determine whether the replacements referred 
to above will be made in the action cycles generated by 04 and 05 . 
(03 causes the replacement to be omitted; o 2 causes it to occur.) 
Consequently, o 2 and 03 cannot act directly on the modus operandi 
of MC. They must, instead, activate two (alternative) memory 
organs in MC which can then effect the operations of MC when they 
reach those points where this is required. Let the memory organs 
activated by 02 and 03 be designated by a 0 and a x , respectively. 

These considerations are summarized in Figure 38, which shows 
schematically the logical structure of the procedure that MC is to 
follow. 

In this figure the arrows (horizontal and vertical) indicate the 
path of stimulation. The double horizontal lines separate areas within 
which the stimulation passes entirely through the channels in MC 
to be constructed, while these double lines themselves are crossed 
by processes that take place outside these channels. (As a rale these 
processes take place in CU, but in one case, that of the second double 
line from above, they take place within the discriminator ^. ) 

The single horizontal lines separate alternatives; i.e., the two 
processes at the two sides of such a single horizontal line, but between 
a pair of double horizontal lines, represent alternatives that are in 
fact mutually exclusive. 

Each bracketed equation on the left margin indicates a value of 
or £ n s+1 or e s+1 that is characteristically associated with the alterna- 
tive shown next to the equation in question. 



234 



THEORY OF SELF-REPRODUCING AUTOMATA 



The figure contains seven groups of operations, referred to by 
the numbers { (28'.a)-(3l'.h)} by which they were designated in 
Sections 4.2.2-4.2.4. These are restated explicitly in Table II. 

We have given these operations new designations (0.1)- (0.31), 
which are shown in Table II, for the sake of convenience, in juxta- 
position with the old designations (28'.a)-(3l'.h). Among the 31 
operations (0.1)- (0.31) there are 16 with an asterisk and 15 without. 
The former are repetitions (norn+ lorn - 1 repetitions; cf. there), 
hence they have to be executed by suitable periodic pulsers (PP's), 
for whose turning on and turning off arrangements with appropriate 
delays must be made. The latter are single operations; hence they 
call for ordinary pulsers (P's) only. Two of these { (0.19) and (0.29)} 
require even no P, since they call for the injection of single stimuli 
only. 

The turning on and turning off and concomitant delay arrange- 
ments for the PP (for the 16 operations with an asterisk) were dis- 
cussed in Sections 4.2.2-4.2.4. In each case this required using a 
triple-return counter <£ (cf. Fig. 23). The use of this organ for this 
purpose was introduced in Section 4.1.7, after operation (27'.a'), 
and in Section 4.1.9, after operation (28'.a). In the first case <£ had 
to be attached to C 2 , i.e., its output d to v 2 and its input c to w 2 . In 
the second case $ had to be attached to Ci , i.e., its output d to v x and 
its input c to Wi . The subsequent discussions in Sections 4.2.2-4.2.6 
showed that these were the only two forms in which $ was required. 
We therefore provide two triple-return counters, <J> X and <£ 2 , the first 
one being attached to Ci (i.e., its input c = Ci to W\ and its output 
d == di to ^i), and the second one to C 2 (i.e., its input c = c 2 to w 2 
and its output d = d 2 to v 2 ). The notations introduced in Section 
4.2.2 after operation (28'.a), and in Section 4.1.7 after operation 
(29'.a'), apply to 3>i and 3> 2 , respectively. Thus 5/' is the delay from 
Wi to Ci and 6/ is the delay from di to v x (these refer to 3>i), while 
b" is the delay from w 2 to c 2 and 5 2 is the delay from d 2 to v 2 (these 
refer to 3> 2 ). 

Of the 16 operations with asterisks, 6 have to be timed by $i 
{(0.1), (0.3), (0.16), (0.18), (0.20), (0.22)} and 10 have to be timed 
by * 2 {(0.5), (0.7), (0.9), (0.11), (0.13), (0.15), (0.23), (0.25) , 
(0.28), (0.30)}. Of the 6 in the first class, 3 require a PP (110110) 
with i ts outp ut attached to { (0.1), (0.16), (0.20)} and 3 require 
a PP (110000) with its output attached to v 2 { (0.3) , (0.18), (0.22)}. 
Of the 10 in the second class, 5 require a PP(llOllO) with its output 
atta ched to u x {(0.5), (0.9), (0.13), (0.23), (0.28)} and 5 require a 
PP( 110000) with its output attached to v x {(0.7), (0.11), (0.15), 



Table II 

Summary of the pulse sequences sent into loops Ci and C 2 



New 
Designation 



Former 
Designation 



Operation 



(/) Group (28' .a)-(28' .d) and (29' .a)- (29' .d) of Section 4.2., 
Inject n -f- 1 sequences 110110 at u 2 . 
Inject the sequence 11101001 at u 2 . 
Inject n + 1 sequences 110000 at v 2 . 
Inject the sequence 11010 at v 2 . 
Inject n sequences 110110 at ui . 



(0 


1)* 


(28 '.a) 


(0 


2) 


(28'.b) 


(0 


3)* 


(28'.c) 


(0 


4) 


(28 , .d) 


(0 


5)* 


(29\a) 


(0 


6) 


(29'. b) 


(0 


7)* 


(29'. c) 


(0 


8) 


(29'. d) 



Inject the sequence 1111011110110111001 at m 
Inject n sequences 110000 at V\ . 



Inject the sequence 110101101011001 at vi 
(II) Group (29'.e')-(29'.h') of Section 4.24 



(0 


9)* 


(29'. e') 


Inject n sequences 110110 at ui . 


(0 


10) 


(29'.f) 


Inject the sequence 111101 at ui . 


(0 


11)* 


(29' .g') 


Inject n + 1 sequences 110000 at Vi 


(0 


12) 


(29'. h') 


Inject the sequence 1010 at V\ . 



(0 


13)* 


(29' .e) 


(0 


14) 


(29'.f) 


(0 


15)* 


(29'.g) 




(IV) 


Group (3 


(0 


16)* 


(30'. a) 


(0 


17) 


(30'. b) 


(0 


18)* 


(30'.c) 


(0 


19) 


(30'. d) 


(0 


20)* 


(30'. e) 


(0 


21) 


(30'. f) 


(0 


22)* 


(30'. g) 


(0 


23)* 


(31'. a) 


(0 


24) 


(31'.b) 


(0 


25)* 


(31'. c) 



(III) Group (29 f .e)-(29 f .g) of Section 4.2.2 

Inject ?i + l sequences 110110 at ui . 
Inject the sequence 1010 at u\ . 
Inject Ti + l sequences 110000 at V\ . 
a)-(30'.g) and (31' .a)- (31' x) of Section 4-2.3 
Inject n sequences 110110 at u 2 . 
Inject the sequence 111101 at u 2 . 
Inject n sequences 110000 at v 2 . 
Inject a single stimulus at v 2 . 
Inject n — 1 sequences 110110 at u 2 . 
Inject the sequence 11010 at u 2 . 
Inject n — 1 sequences 110000 at v 2 . 
Inject n sequences 110110 at ui . 



Inject the sequence 11110111101 at ui 
Inject n sequences 110000 at V\ . 

(V) Group (31'. d') of Section 4-2 4 
(0.26) | (31'.d') | Inject the sequence 110101 at vi . 

(VI) Group (31'. d) of Section 4.2.8 
Inject the sequence 1101011010 at vi . 

oup (31'.e)-(31'.h) of Section 4.2.3 



(0 


27) 


(31'.d) 






(VII) 


(0 


28)* 


(31'.e) 


(0 


29) 


(31'.f ) 


(0 


30)* 


(31'.g) 


(0 


31) 


(31'.h) 



Inject n sequences 110110 at u\ . 
Inject a single stimulus at ui . 
Inject n — 1 sequences 110000 at vi . 
Inject the sequence 11010 at V\ . 



235 



236 



THEORY OF SELF-REPRODUCING AUTOMATA 



(0.25), (0.30)}. The delay requirements were expressed in Sections 
4.2.2-4.2.4 by specifying the amount by which the delay from the 
turning on to the turning off of each PP is exceeded by the delay from 
the input a to the output b of its <£. These excess amounts are restated 
in Table III, together with the <£ and PP data given above. 

Regarding the 15 operations without an asterisk we need only 
specify which one of the inputs U\ , v x of Ci and u 2 , v 2 of C 2 each 
operation feeds. Actually, 5 feed into wi{ (0.6), (0.10), (0.14), (0.24), 
(0.29)}, 5 feed into t* { (0.8), (0.12), (0.26), (0.27), (0.31)}, 3 feed 
into u 2 { (0.2), (0.17), (0.21)}, and 2 feed into v 2 { (0.4), (0.19)}. They 
are shown in a systematic arrangement in Table IV, together with 
the pertinent P data. We also show in this table a reference to an 
additional sequence that has to be injected at V\ at a certain occasion. 
This is the entry (0.0), which refers to the sequence 10101 that occurs 
in the top line of Figure 38. Finally, we show (for later use) for each 
P its length and height. [ There were several errors in von Neumann's 



Table III 

Summary of excess delay requirements for the periodic pulsers stimulating loops 

Ci and C 2 



Operation 


PP Used 


Output of PP 
Attached to 


3> Used 


Excess Delay 














3(6V + Sf) + a 2| 














where a^ is 


(0.5) 












250 


(0.9) 












250 


(0.13) 




^PP(llOllO) 


Ui 






244 


(0.23) 












238 


(0.28) 












238 


(0.7) 












250 


(0.11) 












244 


(0.15) 




►PP (110000) 


Vi 






244 


(0.25) 












238 


(0.30) 








j 




244 

3(6V + 670 + oi , 
where ai is 


(0.1) 












241 


(0.16) 




>PP (110110) 


U2 






247 


(0.20) 










><*>! 


253 


(0.3) 












241 


(0.18) 




>PP(110000) 








247 


(0.22) 












253 



DESIGN OF A TAPE AND ITS CONTROL 



237 



Table IV 



The pulsers used to stimulate loops Ci and C2 



Operation 


P Used 


Output of P 
Attached to 


Length 

(2k) 


Height 

(«+ 2) 


(0.6) 


P(llllOllllOllOlllOOl) 




28 


7 


(0.10) 


P(llllOl) 




10 


4 


(0.14) 


p(IoTb) 


► Ui 


4 


4 


(0.24) 


P(llllOllllOl) 




18 


5 


(0.29) 


Single stimulus 






1 


(0.0) 


i> /1 ni ni \ 
F(1U1U1 ) 




a 
0 


er 
0 


(0.8) 


P(llOlOllOlOllOOl) 




18 


9 


(0.12) 


p(ioio) 


* Vi 


4 


4 


(0.26) 


P(110101) 




8 


5 


(0.27) 


P(1101011010) 




12 


5 


CO 31) 


P (11010) 




6 


4 


(0.2) 


P(11101001) 


\ 


10 


5 


(0.17) 


P(llllOl) 


( U2 


10 


4 


(0.21) 


P(11010) 




6 


4 


(0.4) 


P(11010) 


| v 2 


6 


4 


(0.19) 


Single stimulus 






1 



Table IV. These have been corrected according to the rule stated 
in the editorial summary at the end of Section 3.2.1.] 

4.3.3 The read-write-erase unit RWE. 4 Figure 38 and the three 
tables in Section 4.3.2, together with the other listings given there, 
provide a definite basis for the construction announced at the be- 
ginning of that subsection. We can therefore approach this construc- 
tion now in more complete and specific terms. 

The operations that we wish to instrument primarily affect Ci 
and C 2 ; i.e., they interact with the inputs u x , V\ and u 2 , v 2 of these 
organs and with their outputs wi and w 2 . It is therefore best to start 
our constructions with the organs that are in direct contact with 
U\ , Vi , Wi and u 2 , v 2 , w 2 . These are specified in Tables III and IV, 
together with the ^ referred to in lines 2-4 of Figure 38. The func- 
tioning of all these organs is controlled in the manner described in 
Figure 38 and in Table II. 

4 [Von Neumann's title for the present subsection was "The effector organs 
of B. Positioning and connecting these organs."] 



238 



THEORY OF SELF-REPRODUCING AUTOMATA 



Hence we begin by providing and positioning the organs of Tables 
III and IV, as well as ^. 

Table IV calls for 16 organs, 14 of which are pulsers (P), while 2 
are merely providers of single stimuli. Each of these is actuated by a 
single input, coming from RWEC (cf. Fig. 37), and has a single out- 
put, feeding into one of u\ , v\ , u 2 , v 2 . 

Table III calls for the foll owing organs. First, 2 periodic pulsers 
{PP(llOllO) and PP (110000)) are required, but since each of these 
is required with two different output connections (u\ and Vi in the 
first case, u 2 and v 2 in the second case), therefore only 4 periodic 
pulsers are actually needed. (Table III also shows that each one of 
these PP is required in several different situations — their numbers 
are 5, 5, 3, 3, respectively — requiring different "excess delays." This 
might incline one to introduce a separate PP for each such case, i.e., 
16 in all, instead of the above 4. However, it is more convenient to 
take care of this by other arrangements, which will appear further 
below, and to introduce only 4 PP's, as indicated above.) Second, 
Table III requires a triple-return counter ($), but since this is re- 
quired with two different c, d-connections (vi , W\ in one case and 
v 2 , w 2 in the other) actually two such organs are called for. Each PP 
is controlled by two inputs a + and a_ , coming from MC, and has a 
single output feeding into one of u x , V\ , u 2 , v 2 . Each <£ has an input- 
output pair a, b connected to MC, and an input-output pair c, d 
connected to Vi , Wi or to v 2 , w 2 . 

Finally, we need the T vs. 10101 discriminator ^. This has a single 
input a which is attached to Wi and two outputs 6, c going to 
RWEC (cf. Fig. 38). 

[ Von Neumann overlooked one point when attaching the input a of 
^ to the output Wi of Ci . When Ci is used to time the lengthening (or 
shortening) of the timing loop C 2 , single pulses will go from w x to the 
triple-return counter $>i in Figure 39. But these pulses will also enter 
^, which will then indicate a T received to the constructing unit CU. 

We will assume that the constructing unit CU will be built so as 
to ignore these spurious signals. Alternatively, the input to ^ could 
be closed by a gate controlled from a PP(1) w 7 hen $i is in use. This 
would, of course, change the design of RWE somewhat.] 

All these organs are shown in Figure 39 in their relative positions 
with respect to Ci , C 2 (i.e., Ui , Vi , Wi , u 2 , v 2 , w 2 ) on one hand and 
with respect to MC on the other. The sub-organs contained in this 
assembly are P's (designated 0.6-0.24, 0.0-0.31, 0.2-0.21, 0.4), PP J s, 
<£'s (designated $i , <£ 2 ), and a ^. All of these are not shown in their 
true sizes. The entire assembly is part of MC. To the left, across the 



DESIGN OF A TAPE AND ITS CONTROL 



239 



vertical dashed line, it is attached to the remainder of MC. To the 
right it is attached to Ci , C 2 , as indicated. The P of Table IV are 
indicated by their 0 symbols, except for the two single-stimulation 
organs which are shown as direct channels. All inputs and outputs of 
the sub-organs that occur (P, PP, <£, ^) are properly indicated. The 
various inputs and outputs that connect this assembly with the 
remainder of MC are marked with symbols that are self-explanatory. 
Inspection shows that there are 30 connections of the latter kind, of 
which 4 are inputs to MC and 26 are outputs from MC. 

Note, furthermore, that while the P, PP, and <£ are in their standard 
orientations, i.e., in the orientations of Figures 16, 18, and 23, the ^ 
is reflected about the vertical in comparison to its standard arrange- 
ment in Figure 25. Clearly, this calls merely for trivial transformations 
of the construction. (Fig. 25 contains a P and a D, and these, too, 
must be reflected about the vertical. This calls for corresponding 
transformations in connection with Figs. 16 and 22. These, too, are 
harmless; cf. the discussion in the middle part of Sec. 3.6.1 in connec- 
tion with Fig. 29.) 

The assembly of Figure 39 must be controlled, as we pointed out 
at the beginning of this section, according to the scheme described in 
Figure 38 and in Table II. The organs which effect this control must 
occupy the deeper interior of MC. We saw in Figure 39 that the num- 
ber of connections between this region, and between the assembly 
that was explicitly described, is quite large, 4 inputs and 26 outputs. 
This makes it a practical certainty that the lines representing these 
30 connections will have to cross each other many times in order to 
find their proper endings on the control organs within MC. This 
means that the need for the organ that circumvents the difficulties 
of line-crossing has arisen; i.e., we need the coded channel of Section 
3.6. 

We are not yet in a position to lay out the entire coded channel 
that will be required, i.e., to select the appropriate arrangement, as 
discussed in the first part of Section 3.6.1 and schematically shown in 
Figures 28g-28k. This will be done later. For the time being, we will 
concern ourselves only with the portion that is adjacent to the 
assembly of Figure 39, i.e., that may be thought of as extending along 
the vertical dotted line on the left edge of that figure. 

We saw above that 30 lines (4 inputs and 26 outputs) connect with 
this portion. Each of these is stimulated (or stimulates) independently 
of the others. That is, in the notations used at the beginning of Sec- 
tion 3.6.1, they correspond to inputs a v or outputs b v (from the point 
of view of the coded channel) with different v. Hence the total range 



240 



THEORY OF SELF-REPRODUCING AUTOMATA 



v = 1, • • • , n must allow for at least 30 different possibilities. That 
is, necessarily n ^ 30. Actually it is advisable to choose n somewhat 
larger, since further requirements of the control MC will call for 
additional v values for the inputs a v and outputs b v of the coded 
channel. (Cf. below.) 

On this basis we will now choose the m and k referred to in the 
middle part of Section 3.6.1, which determine the coded sequences 
i v l • • ■ i v m that correspond to the v (v = 1, • • • , n). In this respect 
equation (12') and the observation following equation (13') are 
relevant: given m, these observations determine k ( = (m + l)/2 or 
ra/2, whichever is an integer) and the former determines 



Max n 

The following m are critical : 

m 
k 

Max n 



(m - l\ 



7 8 9 
4 4 5 
20 35 70 



Since we want n ^ 30, with a reasonable margin to spare, m = 7 is 
inadequate, m = 8 is marginal (it will prove to be inadequate, cf. 
below), while m = 9 is presumably adequate (it will prove to be 
adequate; cf. below). We choose therefore 

(32') m = 9, k = 5 

so that n is limited by 

(33') n ^ 70 

only. 



According to Section 3.6.1, each sequence i, 1 • • • i v m begins with a 
T, has length m = 9, and contains k = 5 I's. That is, it is 1 i v 2 • • • £„ 9 , 
where among the i 2 , • • • , i there are four l's and four 0's. We know 
that there are precisely 



(r:)=C)= ™ 



such sequences. Let us order them lexicographically, and use the 
v = 1, • • • , 70, to enumerate them in this order. This, then, defines 
i, 1 - • • i, m = 1 i v 2 - • • i v 9 for each v = 1, • • • , 70, thus certainly in- 
cluding the v = 1, • • • , n {for any n according to condition (33 )}. 
Let us assign to the 30 a v and b v of Figure 39 (i.e., to the u x • 1, • • • , 



DESIGN OF A TAPE AND ITS CONTROL 



241 



<J> 2 a occurring there) the numbers v = 1, • • • , 30, in a vertically 
descending sequence, according to that figure. 

[ Von Neumann proceeded to calculate the dimensions of the 
assembly of Fig. 39. 

It turns out that the height of CCi is greater than the height of 
RWE. Since von Neumann wanted to place each pulser and decoder 
of CCi nearly opposite the organ of RWE to which it is connected, 
the height of CCi controls the height of the complex CQ-RWE. Now 
CCi has 4 pulsers P and 26 decoders D. Von Neumann said that the 
height of each is 7, and, allowing a protective strip of U's between 
each two consecutive organs, he arrived at a height of 239 for CCi . 

But this is wrong. The sequences von Neumann was using in his 
coded channel are of length nine; each begins with a one and contains 
four additional ones. By the rule of Section 3.3 for computing the 
size of a decoder, the height of a decoder for a sequence of length 
nine with five ones is either 10 or 11, depending on the specific se- 
quence. In fact, the rule of Section 3.3 often gives a height which is 
greater than the actual height of a decoder constructed by the design 
algorithm of Section 3.3. But even when this is taken into account, 
a height of over 300 units is needed by the CCi of von Neumann's 
design. While this is a mere error of calculation, it upsets most of 
von Neumann's later calculations concerning the size of MC and the 
delays through MC. 

This error cannot be corrected simply by increasing the height of 
CCi and thereby the height of MC. The reason it cannot be so cor- 
rected concerns the proper phasing of the periodic pulsers PP and 
triple-return counters 3> of RWE. Each PP should be on for about 
three times the delay around the loop Ci or the loop C2 (see Table 
II of Sec. 4.3.2). But there is a delay within each <£ and between the 
secondary input-output of each <i> and its connecting loop. Moreover, 
the PP of RWE are controlled from RWEC. Hence a primary output 
from a <I> of RWE must go through CCi , the main channel, CC 2 , 
RWEC, the main channel again, and CCi again, all before it can 
turn off the PP of RWE. Altogether, about 2000 time units are lost 
in this way. Thus, for correct phasing, the starting of a PP of RWE 
must be delayed about 2000 time units over the starting of its asso- 
ciated triple-return counter <i>. Some of this delay is obtained in the 
control organs CO of RWEC, but most of it is obtained in area W. 
It will turn out that area W is not large enough to give the delay von 
Neumann needed for his design. 

Von Neumann's error can be corrected by extending area W to 
the right. This destroys the rectangular shape of the memory control 



242 



THEORY OF SELF-REPRODUCING AUTOMATA 



MC, however, and there are a number of ways of correcting the error 
without changing the size of MC. In Chapter 5 we will redesign units 
Z and W so that their heights can be reduced enough to allow CCi 
to extend into the area now occupied by Z. 

Von Neumann calculated the width of RWE and of CCi as follows. 
The widest organ in RWE is the pulser labeled "0.6," which produces 
a sequence with 14 ones (see Table II of Sec. 4.3.2). This calls for a 
width of 28. All other organs of RWE are of width 24 or less. Von 
Neumann added 1 unit for the vertical channel from this pulser to 
input Ui of loop Ci , obtaining a total width of 29 units for RWE. 
The coded sequences entering and leaving CCi contain 5 ones each; 
hence the pulsers and decoders of CCi are each of width 10. Von 
Neumann added a strip on each side in accordance with the design 
of Figure 29. Hence CCi is 12 units wide. The combined width of 
Cd RWE is thus 41 units. 

Actually there is a small error here too. The vertical channel from 
the output of the pulser labeled "0.6" would pass by, and receive 
stimuli from, the confluent state in the lower right-hand corner of this 
pulser, so that the wrong sequence would enter input u x of loop Ci . 
There are a number of ways in which this error can be corrected so 
that the complex CCi-RWE can be accommodated within a width of 
41. The pulser can be turned upside down. The pulser can be re- 
designed so as to be narrower and higher. The best way is to redesign 
the coded channel, and hence CCi , as follows. 

Let us count the number of pairwise distinct coded sequences 
needed for the coded channel of MC. The outputs o 2 , o 3 , o 5 of CU 
and the input of CU pass through area X and require 4 coded 
sequences. It will turn out that 4 coded sequences are needed in 
area Z. An examination of Figures 39 and 41 shows that 31 additional 
coded sequences are needed. Hence 39 pairwise distinct coded se- 
quences are needed for the coded channel of MC. 

Recall that m is the length of the coded sequence and k is the num- 
ber of ones it contains. As von Neumann correctly calculated above, 
his algorithm calls for m = 9 and k = 5 when 39 sequences are needed. 
But a better choice is m = 9 and k = 4; this gives 56 different se- 
quences, which is more than enough. The pulsers and decoders for 
k = 4 are of width 8, a saving of two units over von Neumann's 
pulsers and decoders. 

Von Neumann later used the width of 12 for CC 2 and CC 3 and 
calculated a width of 18 for RWEC. He calculated a height of 545 
for CC 3 ; lesser heights are required for CC 2 and RWEC. Thus he 
obtained a width of 42 and a height of 545 for the complex CC 2 - 



DESIGN OF A TAPE AND ITS CONTROL 



243 



RWEC-CC 3 . But for this height of 545 to obtain, the decoders of 
CC 2 cannot always be placed so that their outputs feed directly into 
the organs of RWEC, and the pulsers of CC 3 cannot always be placed 
so that their inputs come directly from the organs of RWEC. Vertical 
channels are needed for their connections. These vertical channels 
can be provided within the limits of von Neumann's dimensions for 
the complex CC2-RWEC-CC3 by using decoders in CC 2 and pulsers 
in CC 3 which are of width 8, so that CC 2 and CC 3 can each be of 
width 10 and RWEC can be of width 22. 

Hence we modify von Neumann's design of the coded channel by 
choosing m = 9 and k = 4, i.e., coded sequences of length 9 containing 
4 ones. By the rule of Section 3.2.1 for computing the size of a pulser, 
the pulsers of this coded channel will be of width 8 and height 7. 
By the rule of Section 3.3 for computing the size of a decoder, the 
decoders of this coded channel will be of width 8 and height 11. But 
it turns out that every decoder constructed by the algorithm of 
Section 3.3 has a height of 10. This means that CCi can be accommo- 
dated within a height of 320 and a width of 10, including insulating 
strips of unexcitable states. 

We thus end up with the following dimensions for the memory 
control MC and its main parts (cf. Fig. 37): 

MC: 547 cells high, 87 cells wide 
RWE: 320 cells high, 31 cells wide 
RWEC: 545 cells high, 22 cells wide 
CCi : 320 cells high, 10 cells wide 
CC 2 : 545 cells high, 10 cells wide 
CC 3 : 545 cells high, 10 cells wide. 
These figures presuppose that the design can be completed without 
enlarging any of the organs of RWE. This presupposition will be 
confirmed in Chapter 5.] 

4.3.4 The basic control organ CO in MC. We have completed the 
first task formulated at the beginning of Section 4.3.3: providing and 
positioning the organs of Tables III and IV in Section 4.3.2, and 
of ^, i.e., of those organs that are in direct contact with U\ , Vi , Wi , u 2 , 
V2 , W2 (i.e., with Ci and C 2 ). We can therefore now pass to the second 
task formulated there: controlling the organs referred to above, in 
the manner described in Figure 38 and in Table II of Section 4.3.2. 

It is advantageous to deal first with a certain preliminary problem, 
before coming to grips with the above task in its entirety. This 
preliminary problem is that of controlling the functioning of the four 
PP of Figure 39 according to the requirements formulated in Table 
III in Section 4.3.2, and in the discussion immediately preceding it. 



244 



THEORY OF SELF-REPRODUCING AUTOMATA 



According to these, each PP must first be turned on and then turned 
off, so that the delay between these two events exceeds by a specified 
amount the delay from the stimulus at the input a of a certain <£ to 
the response at its output b. Each PP is attached to one of Ui , Vi , 
u 2 , v 2 . Let us designate the one of these under consideration by u. 
Let us also designate its<f> (<l>i or<£ 2 ) by l>. Then we have to stimulate, 
according to the above, first u-a+ and <l-a, and then let the response 
at $-b stimulate u-a^ . The delays of these processes must, in addi- 
tion, be so adjusted that they produce together the desired excess 
delay (of u-a+ to u-a^ over 5>a to !>•&) referred to above. 

This control organ has the symbol CO. 

[ See Figure 40, which differs from von Neumann's design in four 
respects. First, von Neumann placed output 6 above output c; our 
positioning is better, since the stimulus from b needs to be delayed 
relative to the stimulus from c. Second, von Neumann omitted the 
insulating row of U's under the PP (I); this is needed since the bottom 
row of a PP (1 ) contains some confluent states. Third, von Neumann's 
CO had a width of 17, while the CO of Figure 40 has a width of 16. 
Fourth, and most important, von Neumann used his PP(T) of Figure 
17b. This is wrong for the following reason. 

At most one control organ CO of RWEC (Fig. 41) is on at any 
time; when it is on it has control of RWE. Now the output from the 
primary output of a triple-return counter ($i or <£ 2 ) of RWE (Fig. 
39) causes stimuli to enter the stop inputs a_ of several CO of RWEC. 
As mentioned at the end of Section 3.2.2, if von Neumann's PP(1) 
of Figure 17 is stimulated at the stop input a_ when it is inactive, it 
will be damaged. The alternate periodic pulser PP(1) of Figure 20 
is not harmed under these circumstances, and therefore we have used 
it in CO.] 

There are 2 different PP, but they are attached to 4 different u 
(u determined the PP as well as its l>; cf. Table III) ; hence we might 
expect that we will need 4 organs CO. However, these 4 cases are 
further subdivided into 16 sub-cases, according to Tables II and III, 
each sub-case corresponding to a 0 with an asterisk in these tables. 
Indeed, each formula (0.x) with an asterisk requires a different 
follow-up move and sequence of moves according to Table II, and a 
different excess delay according to Table III. (Those lines of Table 
III, which happen to call for the same excess delay, differ in their 
u.) Consequently we need 16 organs CO, one for each (0.x) with 
an asterisk. 

Consider now a specific CO, i.e., a specific (0.x) with an asterisk. 
The input a is the primary input. A stimulus at a should go to u-a+ 



DESIGN OF A TAPE AND ITS CONTROL 24f) 

and to 3>a; these will therefore be connected to two outputs, to b 
and to c, respectively. Note that the u-a+ and the <Ja referred to 
must be thought of as certain suitable inputs of the coded channel, 
while the u-a + and <& a in Figure 39, which are the ones ultimately 
aimed at, are outputs of the coded channel. Hence the functioning 
of this organ, involving u-a + and l>-a, will be affected by the corre- 
sponding delays due to the use of the coded channel. This is equally 
true for the uses involving u-a- and <i>-6, which will appear further 
below. All these coded channel delays will affect, additively or sub- 
tractively, the excess delays prescribed in Table III. Hence precise 
delay adjustments will have to be made in CO to account for these 
things, and it will be possible to make a final determination of these 
only after having effected the precise positioning of this CO with 
respect to the coded channel, and the laying out and positioning of 
the entire coded channel with respect to RWE. We must therefore, 
for the time being, leave these delay adjustments in a schematic and 
adjustable condition. We do this by assigning an area D, of as yet 
unspecified dimensions, to the delay path from a to b which will 
become necessary for this purpose; this delay path is indicated by 
the symbol (B. 

We can now go on to the other functions of CO. These are the 
following. A response stimulus from $-b must stimulate w-a_ , and 
also the follow-up move that Table II (and, more broadly, the general 
scheme of Fig. 38) prescribes for this CO case {i.e., for this 0 with an 
asterisk} . 

It would not do, however, to take a connection directly from 
!>•& to u-a- and to the input of the follow T -up move. Indeed, the same 
l>-6 corresponds to several (6 for 3>i and 10 for $ 2 ; cf. Table III) of 
our CO cases. Accordingly, it corresponds to two possible u's (u 2 , 
V2 for 3>i and u x , V\ for <i> 2 ; cf. Table III) and to several possible 
follow-up moves (6 or 10, respectively; cf. above). Hence our CO 
must be provided with a memory, to cause the b stimulus to activate 
only its (the CO's) own u-a- and follow-up move. 

Actually, this is not necessary for the u-a- ; i.e., !>•& could be 
tied to all of Ui • a_ , v i • a_ , u 2 • a_ , v 2 • a_ . This would merely turn off, 
in addition to the one PP which has been previously turned on (by 
u-a + ) and which it is intended to turn off now (because it corre- 
sponds to the correct u), also the three other PP which had not been 
previously turned on at all. Turning off a PP [of Fig. 39] which has 
not been turned on has no effects whatever [cf. the end of Sec. 3.2.2.] 

For the follow-up move, on the other hand, this special control 
of the <$>•& stimulus is necessary. Indeed, if this stimulus were per- 



246 



THEORY OF SELF-REPRODUCING AUTOMATA 



mitted to start all the follow-up moves corresponding to its l> (6 or 
10, respectively; cf. above) together, this would corrupt the func- 
tioning of the automaton altogether. Since the special control of the 
$-b stimulus is needed for its effects on the follow-up moves, it is 
simplest to apply that control to the effects on u-a- also. We will do 
this in what follows. 

This special control (i.e., the coincidence of the fact that the 
specific CO under consideration is the one that has been activated, 
with the response of l>-6) requires a memory organ, as noted above, 
and a subsequent coincidence organ. The memory organ is, of course, 
a PP(1), turned on from a of CO. The PP(1) then supplies con- 
tinuously one stimulus to the coincidence organ, the C in the lower 
right corner of Figure 40. When the <&-b response arrives (i.e., when 
the input d is stimulated), then the coincidence organ receives its 
other stimulus. The response of the coincidence organ then emits 
stimuli at the outputs e and /. Output e is connected to u • a_ , while 
output / is connected to the follow-up move. 

The input d also clears the memory, i.e., turns the PP(1) off. 
[An examination of Figs. 40 and 20 shows that] the turning off of 
PP(1 ) comes too late to interfere with the coincidence-sensing opera- 
tion. 

The delay area D is now allotted a width of 15, while its height h 
is kept adjustable. We will choose h later, when the detailed laying- 
out and adjusting, referred to above, will take place. The discussion 
and determination of the delays within CO must, of course, also be 
postponed until then. 

4.3.5 The read-write-erase control RWEC. 5 [See Fig. 41.] Let us 
now consider the general problem outlined at the beginning of Section 
4.3.4: controlling the organs that are in direct contact with the con- 
necting loop Ci and the timing loop C 2 (i.e., CCi plus RWE of Fig. 
37), according to Figure 38 and Table II in Section 4.3.2. This will, 
of course, be based on the use of the control organ CO of Section 4.3.4. 

Figure 38 shows that there is still a memory requirement to be 
satisfied: the instructions "starts a 0 " (line 5) and "starts ai" (line 6) 
amount to this. 

Activities depending on one or the other of these two instructions 
(marked "a 0 active" or "on active," in lines 9 and 15) occur twice for 
each. It is simplest to attach a "local" memory with this function at 
each one of the four points referred to. Hence we will have to provide 
four PP(T)'s for this purpose. Thus the activities referred to above, 

6 [Von Neumann's title was "The control-area in B. Overall dimensions of 
B."] 



DESIGN OF A TAPE AND ITS CONTROL 



247 



which depend on a 0 or a x being "active" (lines 9 and 15) will be 
derived from a coincidence organ (a C), which gets one stimulus 
from the relevant PP, and the other stimulus from the logical ante- 
cedent of the activity in question, according to Figure 38. 

Finally, there exist two orders "stops a 0 " and two orders "stops 
ai" (whichever happened to be active; cf. lines 10 and 16); i.e., at 
this point the two PP of a 0 or of a\ , respectively, must be turned off. 
However, here a deviation from Figure 38 is permissible. Indeed, 
these "stop," (i.e., turn-off) operations can be delayed to the end 
of these sequences {i.e., to the points where i z is stimulated (lines 
12 and 19)}. Here all four sequences meet. Therefore, here all these 
turn-offs can be effected by a single stimulus, which stops all four 
PP's. In this way not only the two PP of that one of ao , a\ which 
was turned on will now be turned off, but also those of the other one, 
which was not turned on at all. The latter measure, however, is 
harmless; it has no effects whatever. [As explained at the end of 
Section 3.2.2, this is so for von Neumann's general periodic pulser 
PP, but not for his PP(1) of Figure 17b. Therefore the alternate 
PP(T) of Figure 20 should be used here. That is, the four PP(T) of 
Figures 41b, 41c, and 41e should be the alternate PP (1) of Figure 20.] 

The stimulus to be used for this purpose is obviously % itself. 

Based on all these considerations, we arrive at the assembly shown 
in Figure 41. 

The structure of this figure is very similar to that of Figure 39. 
The vertical dotted lines which border it on both sides have the same 
role as the single vertical dotted line in Figure 39. As discussed in 
Section 4.3.3, the latter indicated the portion of the coded channel 
adjacent to the assembly of Figure 39 (i.e., CCi , which is adjacent 
to RWE in Fig. 37). Similarly now the two first mentioned lines 
indicate the portions of the coded channel adjacent to the two edges 
of our assembly in Figure 41 . We will discuss the relationship of these 
three portions of the coded channel (CCi , CC 2 , and CC 3 ) further 
below; it will appear that they determine together most of the coded 
channel of the memory control MC. 

All inputs and outputs of the sub-organs that occur in Figure 41 
(PP, CO) are properly indicated. The various inputs and outputs 
that connect this assembly with the coded channel (i.e., with CC 2 
and CC 3 ) and with the constructing unit CU are marked with symbols 
that are self-explanatory. 

Everything on the CC 2 side of RWEC is an output of CC 2 ; inspec- 
tion shows that there are 33 of these. Repeatedly, there are several 
specimens of the same output b v in positions that are immediately 



248 



THEORY OF SELF-REPRODUCING AUTOMATA 



o -3 cd 

CM CM CM CM 
CO CO co CO 



CO CO CO CO 



CO CO CO CO 



bfl bC bfi bfi 
pH pt< *Ph Ph 



CO CO co co 
I'll 

CO CO CO CO 

CO CO CO CO 



CO CO CO CO 



CO CO CO co 

CO CO CO CO 

I I I I 

CD c3 o 

CO CO CO CO 

CO CO CO CO 



bfi bfi bfi bC 

ptH pU, £ £ 



pL< pen fi( pt, 



bO ^ 

CO co co 

co co co 

CP cJh b£ 

CO CO CO 

CO CO CO 



Ph 



3 0 



2 + 

3 * 



CD CD 

'5 s '5 s 




3 


eqi 


+3 


CO 


o9 


1— ( 


O 


+ 


O 

rH 




O 




CD 


0 

CD 


In; 


In, 



IB 
S3 



o o 

fa 

o 





1 


1 

e 




1 

53 


e 


1 

e 










^> 










+ 

e cm 


+ 

e cm 


+ 

e co 


+ 

© co 


+ 


? ? a s 


5 5 




5 5 


» 5* 


5 5 





o 



o 



o 



o 



o 



o 



o 



o 













T3 




u 

d 




m of d 


d 

u 


cl 

.s 


d 

c3 

u 


c 
H 

.s 






0 


O 




O 




u 

bC 




bott 


top 


zero' 


top 


one" 


d 


d 


d 


d 




d 






CD 


CD 


CD 




CD 


CD 


'5 






pd 


CD 


-d 


CD 


-t-3 








-1-3 






bO 


bfi 


bC 




bfl 




bJD 


en 


en 


en 




en 




CD 





























DESIGN OF A TAPE AND ITS CONTROL 





fct( (ii tti fin \n fn '(n jzf 



3 



8 
8 



■+3 -4-3 -4-3 -+^> -4-3 -4-^ 

g g g g g g 8 

I S" I £ 'M 'M 



s 

■s 

o 
o 



-4-5 ^> +^ 

g g g g g 



£ 

■s 



s • 

0 3 

1 § 

3 3 



5- » 

I •§> * 2 

I -3 ill 

nit 

£ 5 ^ 3 



£ , i . 

^ I a s ^ I a 



4 4 4 4 
s s § k 

5 3 » » 



ft ^? 

S I a a 



O 



6 

u 



6 



6 



6 5 



3 



6 



6 



1 



si 
1 

.s 

1 

1 



o 



o 



250 



THEORY OF SELF-REPRODUCING AUTOMATA 



adjacent to each other. (In one case two neighboring in one 

case four neighboring and in five cases two neighboring <i> 2 -6.) 
In each one of these cases it would have been possible to replace the 
complex of neighboring b v 's by a single b v , thus shortening (i.e., 
reducing the height of) the CC 2 side. However, the height of the CC 2 
side is even so not controlling (cf. further below); hence there is no 
advantage in reducing it. On the other hand, a merger of &„'s would 
necessitate the introduction of vertical distribution channels in the 
assembly to the right of CC 2 (leading to the various sub-organs CO 
that these b v , i.e., these <i>i-6 and <i> 2 -6, feed), and thereby the width 
of the assembly would be effectively increased. 

Everything on the CC 3 side is an input of CC 3 ; inspection shows 
that there are 68 of these. 

In addition to these there are two inputs, 0\ and o 4 , and two out- 
puts, i\ and i 2 , on the top side of Figure 41. 

The portions CC 2 and CC 3 of the coded channel that we have now 
introduced have as inputs a v and outputs b v (always from the point 
of view of the coded channel, i.e., of CC 2 or CC 3 ) all those that 
occurred on CCi (in Fig. 39), and in addition the following new ones: 
/3, i 3 (these occur as both inputs and outputs), o 2 , o 3 , o 6 (these occur 
as outputs only). Thus the 30 distinct v that were required for CCi 
(cf. Sec. 4.3.3) are augmented by another 5. This ties into our discus- 
sion of formulas (32') and (33'). It means that we have so far n ^ 35. 
(There will be yet another increase of n\ cf . later. ) 

[ In the remainder of this section von Neumann calculated the 
dimensions of CC 2 , RWEC, and CC 3 ; see the end of Section 4.3.3 
for a discussion of his results. 

We add Table V, which collates the information in von Neumann's 
Figures 32-36, 39, and 41 and Tables II-IV. Table V shows how the 
control organs of RWEC control the pulsers and periodic pulsers of 
RWE] 



Chapter 5 



AUTOMATA SELF- REPRODUCTION 



5.1 Completion of the Memory Control MC 

[5.1 J The rest of the manuscript. Von Neumann's manuscript 
continues for six further sections and then abruptly terminates. 
These sections are devoted mainly to detailed calculations of the 
delays within the memory control MC. Most of these delay calcu- 
lations are wrong, owing to the errors von Neumann made in design- 
ing and calculating the sizes of various organs and units in Chapter 3. 
For this reason we will not reproduce the balance of the manuscript 
but will summarize it instead. The omitted portion of the manuscript 
is of about the length of Chapter 3 and contains 7 figures. 

The organization and operation of the memory control MC is 
summarized in Section 4.3.1 and Figure 37. The status of von Neu- 
mann's design of MC at the end of Chapter 4 is as follows. 

Read-write-erase unit RWE: The pulsers, periodic pulsers, triple- 
return counters, and discriminator that constitute RWE have been 
designed and their order in RWE has been decided; see Figure 39. 
The exact positions of these organs in RWE have not been decided 
upon, but each is to be so placed that it is fed nearly directly by a 
decoder of CCi and/or it feeds a pulser of CCi nearly directly. RWE 
is to be 320 cells high and 31 cells wide. There is a wide range of choice 
for the position of exit w 2 of the timing loop C 2 . Von Neumann placed 
it 48 cells above the bottom of RWE. 

Read-write-erase-control unit RWEC: RWEC is mainly composed 
of control organs CO; these have been designed except for specifying 
the size of delay area D and the design of the delay path (B (Fig. 40). 
The 16 CO's and the 4 PP(T)'s are arranged as in Figure 41. The 
exact positions of these organs in RWEC have not been decided 
upon, but each organ is to be placed so as to satisfy as nearly as 
possible the principle that a decoder of CC 2 feeds it directly and it 
feeds a pulser of CC 3 directly. RWEC is to be 545 cells high and 22 
cells wide. 

Coded channel: RWEC and RWE are to be positioned as in Figure 



251 



252 



THEORY OF SELF-REPRODUCING AUTOMATA 



37. The coded channel consists of CC 3 , X, Z, Cd , CC 2 , and the 
main channel. The main channel extends from the solid dot near 
the bottom middle of Figure 37 to the solid square in the upper left- 
hand corner of the same figure — more specifically, the main channel 
ends where it feeds the topmost pulser of CC 2 . The code to be used 
in the coded channel has the following characteristics. Each coded 
sequence will begin with a one, will contain exactly three other ones, 
and will be of length 9 or less. This code allows for 56 different se- 
quences; it will be confirmed later that less than this number of se- 
quences is required. These sequences have not yet been assigned to 
the inputs and outputs of the coded channel. The pulsers of the coded 
channel will be of width 8 and height 7, and the decoders of width 
8 and height 10. Cd is to be of width 10 and height 320. CC 2 and 
CC 3 are each to be of width 10 and height 545. 

The memory control MC will be 547 cells high and 87 cells wide, 
assuming that the undesigned organs and units can be accommo- 
dated in this space. 

We will now summarize what von Neumann accomplished in the 
part of the manuscript we are not publishing. He first combined the 
read-write-erase unit RWE and its control RWEC as in Figure 37. 
An examination of Section 4.3.3 and Figure 39 shows that informa- 
tion flows in both directions through CCi , so that CCi contains both 
decoders and pulsers. An examination of Section 4,3.5 and Figure 41 
shows that RWEC receives information only from CC 2 and trans- 
mits information only to CC 3 . Consequently, CC 2 contains only 
decoders and CC 3 contains only pulsers. Hence, the non-cyclic coded 
channel of Figure 28k is adequate. This confirms that the coded 
channel of Figure 37 is indeed adequate. Otherwise, the cyclic channel 
of Figure 30 would be needed. If this were so, the coded sequences of 
the main channel of Figure 37 would have to be recoded when they 
reached the end of the main channel and then transmitted to the 
beginning of the main channel, as is done in Figure 27. 

The area X was designed next. The outputs o 2 , o 3 , and o b from the 
constructing unit CU pass through direct transmission channels in 
Y, enter pulsers in X, enter the main channel of MC as coded se- 
quences, and pass through CC 2 into RWEC. A completion signal 
destined for input % of CU leaves RWEC at any one of the three 
exits shown in Figure 37, is coded in CC 3 , enters the main channel, is 
decoded in X, and passes through Y to the exit labeled c %". Area X 
thus contains three pulsers and one decoder, and is of height 36 and 
width 10. Von Neumann made area Y the same height as area X, 
and area Z the same width as area X, thus specifying the dimensions 



AUTOMATA SELF-REPRODUCTION 



253 



of areas Y, Z, and W. The actual design of area Y is left until later, 
but it is very simple. Area Y contains four communication lines 
composed of ordinary transmission states; the rest of Y is unused. 
We thus have the following dimensions : 

Area X: 36 cells high, 10 cells wide 
Area Y: 36 cells high, 31 cells wide 
Area Z: 191 cells high, 10 cells wide 
Area W: 191 cells high, 31 cells wide. 

Von Neumann next considered the delays that are needed within 
MC to exclude the possibility of corruption by interference among 
the messages which circulate in this system. These fall into two 
classes: those needed to prevent corruption in the main channel 
and those needed to prevent corruption within RWE and RWEC. 
Von Neumann applied the rule (16' ) of Section 3.6.1 and concluded 
that no corruption would occur. This conclusion is not correct, how- 
ever. 

Consider the pulses from outputs e and / of the control organ CO 
(Figs. 40 and 41). The pulse from e enters CC 3 and thereby causes 
a coded sequence to enter the main channel; this coded sequence is 
decoded in CCi and causes a periodic pulser of RWE to stop. The 
pulse from / usually enters CC 3 and produces a coded sequence which 
eventually starts a pulser of RWE. The pulse from / also usually 
enters the top qf the next control organ CO, exits from c (among 
other things), and enters CC 3 ; this pulse eventually starts a triple- 
return counter of RWE. Hence the pulses coming from e and / of a 
CO normally cause three coded sequences to enter the main channel 
in close succession: the first sequence stops a periodic pulser of RWE, 
the second sequence starts a pulser of RWE, and the third sequence 
starts a triple-return counter of RWE. For some CO these sequences 
would overlap in the main channel if they were not properly delayed 
by means of delay paths between RWEC and CC 3 . There is ample 
room for these delay paths in RWEC. 

A similar problem arises in RWE. Consider the input u x which 
feeds the connecting loop Ci . The input u x is fed by both pulsers and 
periodic pulsers, and when loop Ci is being lengthened or shortened, a 
sequence of pulses from a pulser follows immediately after a sequence 
of pulses from a periodic pulser. These sequences must not overlap. 
Similar remarks apply to input V\ of loop Ci and inputs u 2 and v 2 of 
loop C 2 (see Sec. 4.2 and Table II of Sec. 4.3.2). This undesirable 
overlap of sequences can be prevented by adding delays in RWE or 
by changing the order of the periodic pulsers and pulsers in RWE. 



254 



THEORY OF SELF-REPRODUCING AUTOMATA 



These oversights are minor. A more important case of interference 
arises in the main channel when the periodic pulsers and triple-return 
counters of RWE are being used to lengthen or shorten the connecting 
loops Ci and C 2 . When the upper half of each loop is very roughly 
200 cells long, a periodic pulser PP of RWE is started at about the 
same time as a triple-return counter ($1 or <£ 2 ) emits an output pulse 
from its primary output b (Fig. 23). The coded sequence which starts 
the PP and the coded sequence coming from the decoder fed by <f>i 
or $2 will overlap in the main channel, causing contamination. A 
full explanation of this interference problem, and its solution, will 
be given in Section 5.1.2. 

The proper performance of the activities controlled by each of the 
16 organs CO depends upon the observance of certain precisely speci- 
fied delays, each one of these delays being specific to the particular 
CO involved. Each CO controls a lengthening or shortening operation 
on one of the connecting loops Ci or C 2 , and this is timed by means 
of the other connecting loop and a triple-return counter. We will illus- 
trate this point by discussing the delays associated with control 
organ C0 3 ; see Table V and Figures 39 and 41a. 

The PP (110110) of RWE which has inputs ui-a+ and Ui-a- must 
be turned on for approximately 6n units of time, where x n is the 
square of L under scan at the beginning of the process. The quantity 
6n is unbounded and hence cannot be stored in the finite automaton 
MC. In Section 4.1.7 von Neumann introduced the timing loop C 2 
to store the quantity 2n. A pulse takes approximately 6n time steps 
to travel around loop C 2 three times; the delay is not precisely Qn 
because the loop C 2 is lengthened or shortened before the loop Ci is 
lengthened or shortened. These three circuits around C 2 are counted 
by the triple-return counter <£ 2 . Clearly, the control interactions 
between C0 3 , on the one hand, and PP (110110) and <£ 2 on the other, 
take considerable time because RWEC is remote from RWE. Similar 
remarks apply to the other CO's of RWEC. 

We will enumerate the delays included in this control process, 
measuring them from the output of the confluent state in the upper 
right-hand corner of C0 3 . 

(Ai) Starting from output c of C0 3 , the delay through CC 3 , the 
main channel, and CCi , ending at input 3> 2 -a of RWE. 

(A 2 ) The delay within <i> 2 , the delay between $> 2 and input v 2 of C 2 
(this is 5 2 '); the delay between output w 2 of C 2 and <£ 2 (this 
is b" ) ; and the delay within C 2 . Remember that a pulse goes 
around C 2 three times. The total delay is 3 (5 2 + 5 2 ) + a 2 plus 



AUTOMATA SELF-REPRODUCTION 



255 



the desired delay from the turning on to the turning off of 
PP(llOllO); see Table III of Section 4.3.2. 

(A 3 ) Starting from output <£ 2 -& of RWE, the delay through CQ , the 
main channel, and CC 2 ; the delay from d to e of C0 3 ; and the 
delay through CC 3 , the main channel, and CCi , ending with 
input Ui'd- of RWE. This is approximately a complete loop 
around the memory control MC. 

(A 4 ) The delay through area D (along delay path <B) of C0 3 ; the 
delay from output b of C0 3 through CC 3 and the main channel 
to area Z; the delay through Z, W, and Z again; and the delay 
through the main channel and CCi to input Ui-a+ of RWE. The 
exact amount of the delay along (B and in Z and W has not been 
specified yet. 

The memory control MC is 547 cells high and 87 cells wide, and so 
the delay around its main channel is considerable. It is clear that 
PP (110110) must be started much later than <£ 2 because of the excess 
delay 3 (8 2 ' + 82" ) + o>2 plus the delay from the output of <£ 2 around 
the coded channel of MC, through RWEC, and back down the coded 
channel to the stop input Ui»o_ of PP (110110). Hence the delays in 
area D (path (B) of C0 3 and the delays in areas Z and W are indeed 
necessary. Similar delays are needed for the other control organs 
CO, but the exact amount of delay needed varies with each CO, 
because the positions of the CO's in RWEC and the positions of the 
PPs and *'s in RWE all vary. 

Von Neumann made a lengthy and detailed calculation of the 
amount of delay needed in areas D, Z, and W for each CO. These 
calculations are exceedingly long and complicated because of the 
nature of von Neumann's design procedure. He had not yet specified 
the details of the design of MC: the size of area D of CO had not been 
fixed and the paths (B in the control organs COi through COi 6 had 
not been specified ; the exact code of the coded channel had not been 
chosen; and the exact locations of the organs of CC 2 , RWEC, Z, 
W, and RWE had not been specified. In particular, the height of 
RWE was determined by the height of CCi , and there is in fact 
much vacant space in RWE. 

Von Neumann made his delay calculations with all these design 
parameters unspecified, and then specified these parameters on the 
basis of the results of his calculations. This is a very flexible design 
procedure, but it makes the calculations tedious and involved. 
Moreover, some of the dimensions von Neumann used at this point 
were wrong, notably the height of CCi ; see the end of Section 4.3.3. 
These errors vitiate most of his delay calculations. For these reasons 



25() THEORY OF SELF-REPRODUCING AUTOMATA 

we will not reproduce this part of the manuscript. Instead, we will 
summarize his chief results. These results show that von Neumann's 
design of the memory control MC does work, after certain modifica- 
tions are made in it. 

The delays involved depend on design details which are not yet 
fixed, and on design parameters which must be altered to correct for 
von Neumann's design errors. For these reasons we will give only 
very rough estimates of these delays. The delay 3 (5 2 ' + 5 2 A ') + a 2 ]H 
very roughly 000 units. The delay A 3 from <J> 2 • b of R WE through C0 3 
and back to Ui-a,- of RWE is very roughly 1200 units. Note that 
according to von Neumann's calculation of the height of RWE (239 
units— cf. Sec. 4.3.3), the upper PP (TToilO) is about 200 units 
above <£ 2 . The extra delay needed in areas D, Z, and W turns out to 
be very roughly 2000 units. The variation in delay from one CO to 
another is no more than about 100 units. 

The common part of the extra delay associated with each periodic 
pulser of RWE (i.e., about 2000 units of delay) w T ill be taken care of 
in areas Z and W. The part of the extra delay unique to each CO 
(i.e., about 100 units of delay) will be taken care of in the delay area D 
of that CO. At the end of Section 4.3.4 area D was allotted width 15, 
while its height h was left adjustable. The height of RWEC was 
controlled by the height of CC 3 , which is 545. A height of 545 for 
RWEC permits h = 21, so h is chosen to have this value. This gives 
D of CO an area of 21 X 15, or 315 cells, which is more than enough 
to accommodate that part of the extra delay which is unique to each 
CO. 

Von Neumann now proceeds to design the areas Z and W. He di- 
vides W into four equal areas Wi , W 2 , W 3 , and W 4 , plus a slight 
excess. He divides Z into the corresponding parts Zi , Z 2 , Z 3 , and 
Z 4 . Areas Zi and W x are used to get the extra delay associated with 
the upper PP (110110) of RWE, areas Z 2 and W 2 are used to get 
the extra delay needed for the upper PP (110000) of RWE, and so 
forth for the other periodic pulsers of RWE. 

Take C0 3 as an example again. C0 3 controls the upper PP (110110) 
of RWE, starting it at input u x -a + (Figs. 39 and 41a). In the design 
of Section 4.3.5, the start pulse leaves C0 3 at Ui-a+ , is coded in CC 3 , 
travels through the main channel, is decoded at CCi , and enters 
RWE at input u x • a+ . This design is to be modified now so as to add 
about 2000 units of delay to the path from C0 3 to RWE; this extra 
delay is achieved by running the path through the areas Zi and Wi . 
To make this path pass through areas Zi and Wi , we code the output 
Ui-a+ of C0 3 into a new coded sequence Ui*-a+ , place a decoder for 



AUTOMATA SELF-REPRODUCTION 



257 



Ui*-a+ in Zi , place a pulser for Ui-a + in Zi , and connect the output 
of the decoder to the input of the pulser by a long path which travels 
through Wi . 

The delay path A 4 from C0 3 to input Ui-a+ of RWE is now as 
follows. A pulse leaves the confluent element at the upper right-hand 
corner of C0 3 and travels along path (B, exiting at Ui*-a+ , which is 
labeled Ui-a+ in Figure 41a. A pulser of CC 3 sends a coded sequence 
corresponding to Ui -a + into the mam channel j a decoder in Zi 
detects this sequence and sends a pulse through a long path in area 
Wi . This pulse stimulates a pulser in Zi which sends a coded sequence 
corresponding to Ui-a+ into the main channel. Finally, a decoder 
in CCi detects this sequence and sends a pulse into the start input 
a+ of the upper PP(llOllO) of RWE. 

Since co ntrol or gans C0 5 , C0 7 , COi 3 , and COi 5 also control the 
upper PP(llOllO) of RWE, their outputs Ux-a + should also be 
changed to Ui*-a+ , and the pulsers which they stimulate in CC 3 
should be changed accordingly. 

Similar arrangements should be made for the other CO's of RWEC 
and the other parts of Z and W, so that a pulse from an output b 
of CO will pass through areas Z and W before starting a periodic 
pulser of RWE. For these arrangements coded sequences correspond- 
ing to u 2 *-a+ , Us*-a + , and ih*-a+ are used. 

In short, the following modifications are to be made in the design 
of MC as it was left at the end of Chapter 4. The labels Ut-a+ , 
Vi-a+ , u 2 -a+ , and v 2 -a+ of Figure 41 are to be replaced by the labels 
^i*-a+ , Vi*-a+ , u 2 *-a+ , and v 2 *-a+ , respectively, and the pulsers of 
CC 3 recoded accordingly. Area Zi has a decoder for Ui*-a+ and a 
pulser for Ui*a+, the output from the decoder feeding the pulser 
through a long delay path in area Wi . Area Z 2 has a decoder for 
^i*-a + and a pulser for Vi-a + , the former feeding the latter through 
area W 2 . Similarly, a decoder for u 2 *-a+ in Z 3 feeds a pulser for u 2 -a + 
via W 3 , and a decoder for v 2 *-a + in Z 4 feeds a pulser for v 2 -a+ via 
W 4 . 

It will be recalled that von Neumann made an error in calculating 
the height of CCi ; see the end of Section 4.3.3. Because of this error 
he thought that Wi , W 2 , W 3 , and W 4 could each be of height 68. 
Their widths were to be 29, giving each an area of 1972 cells. He 
planned to attain the needed delay by running a path of ordinary 
transmission states back and forth through each area. He thus ex- 
pected to obtain a delay of 1972 units of time, which is slightly more 
than he needed. Moreover, as he observed, additional delays could be 
obtained in the areas D of the CO's. However, because of his error 



258 



THEORY OF SELF-REPRODUCING AUTOMATA 



in calculating the height of CCi , the areas Wi , W 2 , W 3 , and W 4 can 
be of height 45 at most, which does not give enough delay. 

There are a number of ways this error can be corrected. The area 
W can be extended to the right. The organs of MC can be rearranged 
to take advantage of the unused space in X, Y, Z, RWE, and CC 2 . 
But there are two more elegant ways of making the correction. 

In the first place, each of the areas Wi , W 2 , W 3 , and W 4 can be 
designed to give more than 1 unit of delay per cell. Since confluent 
states do not feed one another, they can be alternated with ordinary 
transmission states to give an average delay of \\ units per cell. An 
even larger delay per cell can be obtained by counting processes; 
we will indicate one way in which this can be done. The repeater of 
Figure 18e (Sec. 3.2.2) will by itself give a periodic sequence of the 
form 100- • -00. Let the pulse to be delayed in area Wi start two of 
these repeaters, one producing a sequence of length 41 and the other 
a sequence of length 47, the numbers 41 and 47 being relatively prime. 
Feed the output from both repeaters to a confluent state. After 41 X 
47 (= 1927) units of time there will be a coincidence at this confluent 
state. The resultant output pulse from the confluent state signals a 
delay of 1927 units, and this pulse can be used to turn off the two 
repeaters. In this way the delay of roughly 2000 units which von 
Neumann needs can be obtained in area Wi , and similarly for areas 
W 2 , W 3 , and W 4 . 

Another way of getting the needed delays in the area W that von 
Neumann actually had available is to replace the four delay paths 
of area W by a single path. To do this, break the connections from 
the main channel to the decoders of Z, and connect each decoder of 
Z directly to the pulser of Z which it formerly drove via a delay path 
in W. For example, the output from the decoder for Ui*-a + will now 
go directly into the pulser for Ui-a+ ; see how D (11001) feeds 
P (10011) at the top of the coded channel of Figure 27. Then make a 
branch of the main channel which passes through the top of Z, 
goes through W to give a delay of about 2000 units in W, and then 
feeds the four decoders of Z. 

The design of areas Z and W adds 4 new coded sequences to the 
coded channel, those symbolized by Ui* -a+ , Vi*-a + , u 2 *-a + , and 
v 2 * ' -a+ . There are 30 coded sequences associated with the decoders 
and pulsers of CCi and 4 coded sequences associated with the de- 
coders and pulsers of X. A coded sequence is associated with the out- 
put 13 of COh ; otherwise CC 2 and CC 3 do not add any sequences to 
the list already given. Thus the coded channel of the memory control 
MC needs 39 different coded sequences. At the end of Section 4.3.3 



AUTOMATA SELF-REPRODUCTION 



259 



we tentatively assumed that a code with sequences of length 9 con- 
taining four ones, which allows 56 different sequences, would be 
adequate. This assumption is now confirmed. 

This completes von Neumann's design of the memory control MC. 
The exact code of the coded channel has not been selected, and the 
exact positions of many of the organs of MC have not been chosen, 
but these are minor matters of detail. 

Von Neumann concluded the manuscript by calculating the dura- 
tion of the operations of the memory control MC. Equation (17') of 
Section 4.1.3 specifies n as the number of the square x n of the linear 
array L which is under scan at a given step s. Von Neumann found 
the total time for lengthening to be approximately 36n s + 13,000, 
and the total time for shortening to be approximately 48n s + 20,000. 

5.1 .2 Solution of the interference problem. The design of the memory 
control MC is now complete and workable except for the interference 
problem left unsolved in Section 5.1.1. In the present subsection we 
will explain this problem and give methods for solving it. 

The problem involves the read-write-erase control RWEC and the 
read -writ e-erase unit RWE. In certain circumstances, a signal from 
RWEC to RWE interferes with a signal from RWE to RWEC in 
the main channel. 

Von Neumann used n to represent the number of the square x n of 
the linear array L which is under scan at a given step s. For certain 
values of n% a coded sequence used to start a periodic pulser of RWE 
will overlap a coded sequence coming from CCi to instruct RWEC to 
stop that same periodic pulser. This overlap will occur in the main 
channel between CCi and the end. In some cases this overlap will 
produce corruption by interference; see formula (16') of Section 3.6.1. 
The two overlapping sequences will form a third sequence which will 
stimulate a decoder of CC 2 that should not be stimulated at this 
point in the operation of MC. 

This interference problem involves all of the periodic pulsers and 
triple-return counters of RWE, but for the sake of concreteness we 
will explain it in terms of the upper periodic pulser PP(llOllO) of 
RWE and the triple-return counter <£ 2 used in timing it. Consider 
first the operation of PP (110110). It is started with a pulse into its 
input U\'d+ at some time r+ and stopped with a pulse into its input 
Ui-a,- at some time r_ , where r + < r_ . Table V shows that r_ — r+ 
equals approximately Qn. Hence the time of the start at U\ • a+ is given 

by 



(l) 



260 



THEORY OF SELF-REPRODUCING AUTOMATA 



Consider next the control of the stop at U\ • a_ . This is controlled 
by the pulse from the primary output <£ 2 -6 of the triple-return counter 
$2 at some time n . This pulse must travel from <£ 2 -6 through CCi , 
the main channel, CC 2 , some control organ of RWEC (e.g., C0 3 ), 
the main channel again, and CCi again, to stimulate U\-a_ 
and thereby stop PP (1101 10). The time required for this transit is 
the delay A 3 of Section 5.1.1, and is very roughly 1200 units. Hence 
the time of the exit from <f> 2 -fr of RWE is very roughly 

(2) r b = r_ - 1200. 

Now compare equations (1) and (2), keeping in mind that n is 
the number of the square of L under scan. For n = 0, r+ is about 1200 
units larger (later) than r b . For large n% r+ is very much smaller 
(earlier) than r b . And for n s equal to about 200, the times r+ and 
r b are equal. Hence when n s equals approximately 200, the coded 
sequence Ui-a+ will be in the main channel near CCi at about the 
same time the coded sequence $> 2 • b enters this part of the main chan- 
nel. The result will be corruption by interference in the coded channel. 

Hence von Neumann's design of MC does not work correctly 
when the loops Ci and C 2 are of certain lengths. We will give two 
different solutions to this interference problem. The first consists in 
avoiding the interference by never using those cells of L where it 
arises. This may be done by programming MC in such a way that no 
value of n less than, say, 250 is used. The cells xo , X\ , • • • , x 24 g of 
L will then never be used. This implies that when MC begins life, 
loop Ci will pass through cell x 2 5o and loop C 2 has a corresponding 
length. And this in turn affects the arrangements for universal con- 
struction, as we shall now see. 

Von Neumann's arrangements for universal construction are as 
follows (Sees. 1.5.2 and 1.7.2.1). Enclose the secondary automaton to 
be constructed in a rectangular area of side lengths a and ($. The de- 
sired state of each cell (i, j) (i = 0, 1, • • • , a. — 1; j = 0, 1, • • • , 
iS — 1) of this rectangular area is represented by X t y . The construc- 
tion process will leave an arbitrary cell (i, j) in any 1 of the 10 unex- 
cited states U, T ua0 {u = 0, 1; a = 0, 1, 2, 3), and C 0 o (see Figs. 9 and 
10), and so is limited to 10 values. The desired construction is 
described to the primary (parent) automaton by giving it the se- 
quence AoO , Ad > * * * > Xo(0-1) j XlO , * * * , Xt; , * * * , X( a _i) (/S_i) . 

The secondary automaton to be constructed will in general have 
an indefinitely extendible linear array L and a memory control MC 
to control it. This being so, von Neumann designed MC so that it 
would be rectangular. Now on this first solution of the interference 



AUTOMATA SELF-REPRODUCTION 



261 



problem, MC must be constructed so that initially loop Ci will pass 
through cell x 25 o and loop C 2 has a corresponding length. To keep the 
rectangular shape of MC, we extend the boundary of MC 250 cells 
to the right, making MC 337 cells wide and 547 cells high (see Fig. 
37 ) . All but about 1 per cent of the cells in the added area will always 
be in the unexcitable state U, which means that this new area is 
being used very inefficiently. 

Hence while this first solution to the interference problem is worka- 
ble, it is most inelegant, and von Neumann would not have been 
satisfied with it. For this reason we will suggest a second solution. This 
solution is also of general interest because it illustrates some funda- 
mental features of von Neumann's cellular structure. 

Von Neumann's cellular structure consists of an infinite iterated 
array of the same 29 -state finite automaton. Any finite number of 
these automata can be active at a given time; therefore the cellular 
structure is capable of an indefinitely large amount of concurrent 
action or parallel data processing. In his design of the self -reproducing 
automaton, von Neumann was not taking advantage of the parallel 
data processing capability of his cellular structure. Rather, he was 
designing the self -reproducing automaton so that for the most part 
it would do only one thing at a time. In this respect, von Neumann's 
logical design of the self -reproducing automaton is similar to his 
logical design of the ED VAC (see pp. 10-11 above). Moreover, when 
he did use parallel activity in the lengthening and shortening of loops 
Ci and C 2 , he ran into timing problems. He needed the delay area 
D of each CO (Fig. 40) and the delay area W of MC (Fig. 37) to 
postpone the start of a periodic pulser of RWE, and in arranging 
for the stop of this periodic pulser he ran into the interference prob- 
lem we are now discussing. 

This interference problem would not arise if two signal paths 
(wires) could be crossed without intersecting; so let us look at the 
possibilities for wire-crossing in von Neumann's cellular structure 
and variations of it. Wires could cross naturally without touching 
in a 3-dimensional cellular structure, but von Neumann wanted to 
construct a self -reproducing automaton in 2 dimensions (Sec. 1.3.3.3). 
Keeping to his 2-dimensional structure, he could have incorporated a 
crossing primitive into each cell. For example, his 29-state automaton 
could be augmented by a new primitive consisting of Too* and Toi«/ 
together, so that information could flow through a cell from left to 
right and from bottom to top simultaneously. 1 Von Neumann did not 

1 Cellular structures which have crossing primitives are considered 
in Church, "Application of Recursive Arithmetic to the Problem of Circuit 



262 



THEORY OF SELF-REPRODUCING AUTOMATA 



say why he didn't include a crossing primitive among the states of 
his basic automaton, but it was probably because he wished to keep 
the number of states small. 

It is actually possible to synthesize a crossing organ in von Neu- 
mann's cellular structure. Such an organ is shown in Figure 42a. 2 
The symbolism of this and subsequent figures is somewhat different 
from von Neumann's symbolism. A single arrow is used to represent 
an ordinary transmission state; later a double arrow will be used to 
represent a special transmission state. A dot besides an arrow in- 
dicates that the cell is initially active, i.e., is excited at time t = 0. 
Figure 42b gives the initial state of each of the five "clocks" of the 
crossing organ in von Neumann's notation. 

Consider first the behavior of the crossing organ when the inputs 
ai and a 2 are 000 • • • . The clocks send alternate zeros and ones into 
both inputs of each of the six confluent states C3, C6, F3, F6, El , and 
H5. The phasing of the ones (i.e., pulses) as they enter these con- 
fluent cells is indicated in Figure 42a by dashed and dotted lines. A 
dashed line signifies a one (pulse) at every even time (t ^ 4) and a 
dotted line signifies a one (pulse) at every odd time (t ^ 3). It is 
clear from Figure 42a that the two sequences arriving at a confluent 
state are out of phase. The function of these clock sequences is to 
gate the sequences coming into a\ and a 4 so that they can cross each 
other. 

The sequence i 0 , ii , i 2 , is , u , H , • * ■ entering a\ is split into two 
sequences by the confluent cell A4- The clocks insert ones into 
every even position of the upper sequence and into every odd position 
of the lower sequence. The odd bits — , % , — , i z , — , % , • • • are 
allowed by the gating pulses to pass along row 3 and out at b\ , while 
the even bits i 0 , — , i 2 , — , u , — , • • ■ are allowed by the gating pulses 
to pass along row T 6 and out at b\ . Similarly, the sequence jo , ji , j 2 , 
jz yji,jh, * * * entering a 2 is split, with the even bits jo , — , j 2 , — , 
j ; 4 , — , • • • traveling up column C and the odd bits — , ji , — , jz , 
— , j 5 , • • • traveling up column F. The phasing of the whole system 
is such that the sequence jo , — , j 2 , — , j* , — , • • • is interleaved with 
io , — , i 2 , — , u , — , • • • at cell C6 and with — , i\ , — , i z , — , i$ , • • • 
at cell C3. Likewise, the sequence — , ji , — , j 3 , — , j$ , • • • is inter- 



Synthesis." Cellular structures which have crossing primitives and also per- 
mit the construction of zero-delay paths of unbounded (but finite) length are 
discussed in Burks, "Computation, Behavior, and Structure in Fixed and 
Growing Automata," Holland, "A Universal Computer Capable of Executing 
an Arbitrary Number of Sub-Programs Simultaneously," and Holland, "Itera- 
tive Circuit Computers." 

2 The crossing organ of Figure 42 was designed by J. E. Gorman. 



AUTOMATA SELF-REPRODUCTION 



263 



leaved with i 0 , — , i 2 , — , u , — , • • • at cell F6 and with — , i x , — , 
i a , — , % , • • • at cell F3. For example, the sequences entering and 
leaving cell C6 are: 



The sequences from cells C3 and F3 are combined in cell El to give 
the output jo , ji , j'2 , • • • delayed 15 units of time. Similarly, the 
sequences from cells F3 and F6 are combined in cell H5 to give the 
output to , ii , i 2 , • • • delayed 15 units of time. In this way informa- 
tion passes from a\ to 6i and from a 2 to 6 2 without any cross inter- 
ference. 

We shall now employ the crossing organ to solve the interference 
problem in von Neumann's design of the memory control MC. The 
interference occurs when n is very roughly 200. Under this condi- 
tion, a coded sequence U\ • a + , v\ • a+ , u 2 • a+ , or v 2 • a + which is to start 
a periodic pulser of RWE interferes with a coded sequence <i> 2 *6 or 
$i-6 which should eventually (via RWEC) stop this same periodic 
pulser. By means of the crossing organ the signal <J> 2 -b can be sent 
directly from <f> 2 to stop an upper periodic pulser of RWE and can 
then be sent into the coded channel to signal RWEC that this phase 
of the lengthening or shortening of loop Ci is finished. Likewise, the 
signal can be sent directly from $i to stop a lower periodic pulser 
of RWE and thence into the coded channel. This modification of 
von Neumann's design greatly reduces the amount of delay needed 
in the delay areas D (of a CO) and W (of MC). 

The simplest way to arrange for a direct stop of a periodic pulser 
by a triple-return counter is this. 

(1) In Figure 39a, exchange the pulser labeled "0.0" with 
PP (110000), and invert PP (TlOllO). The stop inputs of both peri- 
odic pulsers can now be connected; call the point of connection 
Ui-Vi'd- . Similarly, in Figure 39b exchange the pulser labeled "0.4" 
with PP (110000) and invert PP(110110); call the common stop 
input of these two periodic pulsers u 2 • v 2 • a_ . 

(2) Replace the single column of U states located between the 
upward and downward portions of the main channel near the center 
of MC by 12 columns of cells. This room will be used for the channels 
and crossing organs of (3) and (4) below. 

(3 ) Then take the primary output b of <f>i across the main channel 
(by means of a crossing organ), down, back across the main channel 
(by means of a second crossing organ), into u 2 -v 2 -a- , and also into 



From the left : 
From below : 
Output : 



0 0 10 1 to I 4 1 U • • • 

o o o i jo i h i U 1 • • • 

0 0 0 0 0 0 jo io j 2 i 2 j 4 U " - 



264 



THEORY OF SELF-REPRODUCING AUTOMATA 



a pulser of CCi (which will signal the appropriate CO of RWEC 
that this phase of the lengthening or shortening of loop C 2 is finished ) . 

(4) Finally, take the primary output b of $> 2 across the main channel 
(by means of a third crossing organ), up and outside of the channel 
of (3), back across the main channel (by means of a fourth crossing 
organ), into Ui-Vi-a,-, and also into a pulser of CCi (which will 
signal the appropriate CO of RWEC that this phase of the lengthen- 
ing or shortening of loop Ci is finished). 

These arrangements take only four crossing organs and considera- 
bly reduce the delay circuitry of von Neumann's design. But they 
raise a problem concerning the construction of the memory control 
MC, and hence of the construction of any secondary automaton which 
contains an indefinitely extendible linear array L. The construction 
process von Neumann envisaged has two stages (cf. Sees. 1.5.2 and 
1.7.2.1). First, the primary (constructing) automaton puts each 
cell of the secondary area in 1 of the 10 unexcited (quiescent) states 
U, T wa0 (u = 0, 1; a = 0, 1, 2, 3), and Coo (see Figs. 9 and 10). Von 
Neumann called the result of this process the initial state of the second- 
ary automaton. Second, the primary automaton stimulates the sec- 
ondary automaton at an appropriate point on its periphery, so that 
the secondary may begin its intended activity. Von Neumann called 
this starting signal the secondary automaton's starting stimulus. 

Thus the secondary automata von Neumann wished to have con- 
structed by the primary constructing automaton (i.e., from within 
his cellular structure) are all of a special kind: each cell of the second- 
ary is initially in 1 of the 10 quiescent states U, T Ma0 (u = 0, 1 ; a = 
0, 1, 2, 3), and C 0 o ; and the automaton may be started by a stimulus 
on its periphery. With this in mind, we define an initially quiescent 
automaton to be a finite area of von Neumann's 29 -state cellular 
structure every cell of which is in 1 of the 10 quiescent states U, 
T Ma o , and Coo • 

Since the crossing organ contains excited ordinary transmission 
states and excited confluent states, any automaton containing it is 
not initially quiescent. Consequently, the design of the memory 
control MC must be further modified so that MC will contain no 
excited cells initially, but that before it is used MC will contain the 
four crossing organs described above. 

The following procedure will work. Further enlarge the area in the 
center of MC to allow for the four constructing devices and four de- 
coders described below. Modify each of the four crossing organs of 
MC as follows: replace each active state (Ton , Cio , C 0 i) of Figure 42 
by the corresponding passive state (Toio , C 0 o , Coo), and delete cells 



AUTOMATA SELF-REPRODUCTION 



A5, B5, C5 (replace them by U's) to make the center clock accessible 
from the outside. Then provide, for each modified crossing organ, a 
special-purpose constructing device and a decoder to start it. The 
constructing device will employ the general procedure for modifying 
a remote cell which was described on p. 155 and illustrated in Figure 
14; by means of this procedure it can start the five clocks of the cross- 
ing organ in phase and rebuild cells A5, B5, and C5. The constructing 
device itself will be started by the decoder on signal from the main 
channel. 

Now arrange for the secondary automaton's starting stimulus to 
put a (new) coded sequence into the main channel of MC. This 
sequence will be sensed by each of the four decoders, which will in 
turn start the four constructing devices. Each constructing device 
will send a constructing arm to its crossing organ, start each of the 
five clocks of the crossing organ in proper phase, rebuild cells A5, B5, 
C5 of Figure 42, and leave the neighborhood of the crossing organ 
in the proper state. The memory control MC is then ready to operate, 
and the secondary automaton containing it can proceed in its normal 
way. 

This completes the second solution of the interference problem in 
von Neumann's design of the memory control MC. Neither solution 
is ideal: the first solution is inelegant, and the second solution is 
rather complicated. A radically different and much better approach to 
the design of MC will be suggested in Section 5.2.2. However, the 
inelegance or complexity of the final design of MC is not directly 
relevant to von Neumann's central purpose. Von Neumann was 
seeking an existence proof of self -reproduction, that is, a proof that 
self -reproduction is possible in his cellular structure (Sec. 1.1.2.1). 
The construction of the memory control MC is a step towards this 
proof, and for this purpose it suffices that there exists a workable MC. 

Let us summarize the results which have been achieved so far. 
The unlimited linear array L together with its control MC is a tape 
unit with an unlimited memory capacity. It may be constructed 
(embedded) in the 29-state cellular structure as an initially quiescent 
automaton. Hence, an initially quiescent automaton which performs 
the junctions of a tape unit with unlimited memory capacity can be 
embedded in von Neumann's 29-state cellular structure. 

5.1.3 Logical universality of the cellular structure. Let us next re- 
view briefly how von Neumann planned to use a tape unit with un- 
limited memory capacity in his cellular structure. 

He discussed Turing's universal automaton in the Second Lecture 
of Part I of the present work. Early in Part II he stated the five main 



266 



THEORY OF SELF-REPRODUCING AUTOMATA 



questions considered in this part: (A) Logical universality: can any 
single automaton perform all the logical operations which are per- 
formable with finite (but arbitrarily extensive) means? (B) Con- 
structibility: can automata be constructed by other automata? (C) 
Construction universality: can any single automaton construct every 
other automaton? (D) Self -reproduction: can any automaton con- 
struct copies of itself? (E) Evolution: can the construction of automata 
by automata progress from simple automata to increasingly com- 
plicated automata? Von Neumann stated that Turing had answered 
the first question. That is, Turing's universal computing automaton 
(machine) is logically universal. Von Neumann then promised to 
establish affirmative answers to questions (B)-(D). 

In discussing general construction schemes, von Neumann intro- 
duced the unlimited memory array L and its "ancillary observation, 
exploration, and construction facilities" (Sec. 1.4.2.3). The latter is 
the memory control MC, the design of which we have just completed 
(Sec. 5.1.2). Let us call the complex L + MC a "tape unit." In 
Section 1.4.2.2 von Neumann stated in effect that this tape unit can 
be used as the unlimited memory of a constructing automaton, in 
Section 1.5 he indicated how to use it as the unlimited memory of a 
universal constructing automaton, and in Section 1.6 he indicated 
how to use a universal constructing automaton to obtain self-repro- 
duction. We will show in Sections 5.2 and 5.3 how these results may 
be achieved. 

In Section 1.4.2.3 von Neumann stated that the tape unit can be 
used as unlimited memory of a logically universal automaton or 
universal Turing machine. Earlier, in the Second Lecture of Part I 
above, he explained how a universal Turing machine works. And in 
Sections 4.1.3 and 4.1.4 he outlined how the constructing unit CU 
can operate the tape unit MC + L. Putting all these ideas together, 
we will show how to design a universal Turing machine in von Neu- 
mann^ cellular structure. 

A Turing machine has two main parts: a tape unit with an indefinite 
memory capacity, and a finite automaton which can interact with 
this tape unit. As outlined in Sections 4.1.3 and 4.1.4 above, a con- 
structing automaton has two corresponding parts: a tape unit MC + 
L and a constructing unit CU which directs the construction of a 
secondary automaton on the basis of the information stored in L. 
Thus CU is a finite automaton which interacts with the tape unit 
and also performs the function of construction. Hence our task is to 
adapt von Neumann's outline for the interaction of CU and MC + L 
to cover the operation of a Turing machine. 



AUTOMATA SELF-KEPKODUCTION 



A finite automaton FA has a finite number of states a = 1, • • • , a. 
The automaton FA and the tape unit MC + L operate in a succession 
of steps 0, 1, 2, 3, • • • , s, s + 1, • • • . Let the state of FA at step 
number zero be state number one, let x n s be the cell of L under scan 
at the beginning of step s, and let £n« (= 0, 1) represent the content 
of x n » at this time. The finite automaton FA is then defined by three 
functions A, X, and E: 

The function A specifies the next state a s+1 as a function of 
present state a and the contents of x n » at the beginning 
of step s: a s+1 = A (a, 

The function X specifies the number to be written in x n * 
as a function of a and £n* , or equivalently, as a function 
ofa' +1 :& 1 = X(a s+1 ). 

The function E specifies the value of the lengthening-short- 
ening parameter e s+1 as a function of a s+1 : e s+1 = E{a s+l ). 
The ranges of arguments and function values are : 
a and a s+1 range over the finite automaton states 1,2, • • • , a; 
£n« and range over 0 (representing "0" on x n ) and 1 (representing 
"1" on x n ); 

e s+1 ranges over +1 (lengthening) and —1 (shortening). 

An "initially quiescent automaton" is one in which every cell is 
initially in 1 of the 10 quiescent states U, T Ma0 , and C 0 o (Sec. 5.1.2). 
We will show next how to embed any given finite automaton FA 
in von Neumann's cellular structure, that is, how to construct an 
initially quiescent automaton which will simulate FA. 

Each of the a states of FA will be represented by a copy of the state 
organ SO of Figure 43. The periodic pulsers PP(1) of this figure are 
the alternate periodic pulsers of Figure 20, which can be turned off 
without harm even when they have not been turned on. These PP (1 ) 
are not shown to scale in Figure 43; each is actually 13 cells long and 

4 cells high. As in Figure 42, single arrows are used to represent 
ordinary transmission states. 

The finite automaton FA consists of a copies of SO interconnected 
by means of a coded channel (Sec. 3.6 and Fig. 27). The specific 
interconnections within an FA are determined by its three functions 
A, X, and E. Within FA, control is shifted from one state organ SO a 
to another state organ SO«' in accordance with the information FA 
receives from MC. The automata FA and MC are interconnected 
by eight channels. The inputs ii , i 2 , i$ of FA come from the outputs 
of MC which are so labeled (Fig. 37) ; the outputs oi , 02 , o 3 , o 4 , and 

0 5 of FA go to the inputs of MC which are so labeled. 

The relation of the SO's of FA to the coded channel of FA is very 



268 



THEORY OF SELF-REPRODUCING AUTOMATA 



much the same as the relation of the CO's of MC to the coded channel 

of MC (Fig. 37). 

We will explain how the composite FA + (MC + L) operates. 

At the beginning of step s the following conditions obtain: 

(la) A stimulus is arriving at input % of FA, signifying that step 
.9—1 has been completed and it is time to begin step s. 

(lb) Cell x n » of L is under scan; i.e., MC is connected to x n » via 
the connecting loop Ci . The contents of the cell x n » are desig- 
nated to be (= 0, 1). 

(lc) FA is in state a; i.e., the state organ SO a « of FA is in control. 
More specifically, the upper PP(1) of Figure 43 is active. This 
periodic pulser has been on (active) while loops Ci and C 2 
of Figure 37 were being shortened or lengthened at the end of 
step s — 1. 

The reading process is now inaugurated and control is shifted to 
the lower PP(T) of SO a * : 

(2a) The pulse entering input % of FA goes via the coded channel 
to the input b of each SO of FA. It affects only SO a « , turning 
off its upper PP(1), and passing through the confluent state 
turned on by this PP(1) to exit from outputs j and k. 

(2b) The stimulus from exit j goes via the coded channel of FA to 
output Oi of FA, and thence to input Oi of MC, where it starts 
the process of reading cell x n8 of L. 

(2c) The stimulus from exit k of SO a « goes via the coded channel of 
FA to input / of this same SO«» , where it starts the lower 
PP(T). This PP(I) will be active while cell x n * is being read 
by MC. 

The tape unit MC + L then reads the contents of cell x n » and 
sends the result to FA : 

If £n« = 0 (i.e., x n s stores a "zero")> then a stimulus goes 

from output i x of MC to input ii of FA. 
If £n« = 1 (i.e., x n s stores a "one")> then a stimulus goes 
from output i 2 of MC to input i 2 of FA. 
We state next how these signals affect FA, using brackets to indi- 
cate the effects of £n« = 0 and braces to indicate the effects of = 1. 
(3a) The [input ii] {input i 2 ] goes via the coded channel of FA to 
enter [input d] {inputs e and g) of every SO. These signals 
affect only SO a « . 
(3b) The pulse from [ii] {i 2 } turns off the lower PP(l) of SO a « . 
(3c) The pulse from [ii] [i^] into [d] {g} is passed by the [upper] 
{lower} confluent state which is turned on by the output of the 
lower PP(1), and exits from outputs [Z, m, n] {p, q, r} . 



AUTOMATA SELF-REPRODUCTION 



269 



The pulses from [Z, m, n] {/>, g, r} will determine the next state of 
FA, what is to be written in x n s , and whether loops Q and C 2 are to 
be lengthened or shortened. 

The specific connections of these outputs Z, m, n, p, q, and r are 
determined by the three functions A, X, and E which characterize 
the given finite automaton FA. For each state organ SO«« (a = 
1,2, • • • , a), these connections are made as follows: 
(4a) The pulse from [I] {p} goes via the coded channel of FA to 

input c of SO«*+i , where a s+1 = A (a 8 , £n*)- 
(4b) The pulse from [m] {q} goes via the coded channel and outputs 

of FA to: 

input o 2 of MC if £„ s+1 = 0 (i.e., "0" is to be written in x n «)> 
input o 3 of MC if £ n s+1 = 1 (i.e., "1" is to be written in x n *), 
where £ n s+1 = X(a s+1 ). 
(4c) The pulse from [n] {r} goes via the coded channel and outputs 
of FA to: 

input 04 of MC if e s+l = 1 (i.e., loops Ci and C 2 are to be 
lengthened), 

input o b of MC if e s+1 = —1 (i.e., loops Ci and C 2 are to 

be shortened), 

where e^ 1 = E(a 8+1 ). 
This completes our construction of an arbitrary finite automaton 
FA in von Neumann's cellular structure, except for specifying the 
state for step s = 0. This state was stipulated to be state number one 
and hence is represented by the state organ SOi . If the confluent 
cell in the upper right-hand corner of SOi were initially in the state 
Cio , starting pulses would emerge from exits j and k of SOi . These 
pulses would stimulate pulsers of the coded channel which in turn 
would inject coded sequences yj and y k into the main channel, and 
FA would then operate as described above. However, such a device 
would not be an initially quiescent automaton, for its initial state 
would contain one cell in a state other than U, T wa o , and Coo . To 
make FA initially quiescent, we merely arrange for its starting 
stimulus to stimulate pulsers of the coded channel which will also 
inject the coded sequences yj and yu into the main channel of FA. 

This concludes our demonstration that an arbitrary finite auto- 
maton can be embedded in von Neumann's cellular structure as an 
initially quiescent automaton. We saw at the end of Section 5.1.2 
that a tape unit with unlimited memory capacity can be embedded 
in this cellular structure as an initially quiescent automaton. Hence 
any arbitrary Turing machine can be embedded as an initially 



270 



THEORY OF SELF-REPRODUCING AUTOMATA 



quiescent automaton, and a fortiori a universal Turing machine can 
be embedded as an initially quiescent automaton. 

It should be noted that all these embedded devices will operate 
slowly with respect to the temporal reference frame of the cellular 
structure. This temporal reference frame consists of the discrete time 
steps t = 0, 1, 2, • • • , and the fundamental operations of the 29-state 
automata of the cells take place in this reference frame (Sees. 1.2.1 
and 1.3.3.5). The finite automata and Turing machines which are 
embedded in the cellular structure operate in this temporal reference 
frame too, but the succession of steps s = 0, 1, 2, • • • takes place 
more slowly. In general, each of the steps s takes several time units 
t. In the case of a tape unit MC + L, the steps s take longer and 
longer as the loops Ci and C 2 get longer and longer. 

Finite automata and Turing machines are usually regarded as 
devices which accomplish one step s in one unit of time t. In other 
words, s = t, and the machines perform their computations in "real 
time." The initially quiescent automata of von Neumann's cellular 
structure which simulate finite automata and Turing machines do 
not operate at the same rate as the devices they simulate, but they 
compute the same results. 3 

Von Neumann's main aim was to achieve construction, construc- 
tion universality, and self -reproduction in his 29-state infinite cellular 
structure (Sec. 1.1.2.1). Let us summarize what has been accom- 
plished so far, and how it bears on his main aim. 

We have shown how to embed in von Neumann's 29-state cellular 
structure an initially quiescent automaton which performs the computa- 
tions of a universal Turing machine. Hence this cellular structure is 
logically universal. 

In planning his universal constructing automaton, von Neumann 
was guided by the parallel notion of a universal computing machine. 
His universal constructing automaton will operate like a universal 
computing machine, the chief difference being that the output of the 
computing machine is a computation, while the output of the con- 
structing automaton is a sequence of signals which constructs an 
initially quiescent secondary automaton. A universal computing 
machine M u is a complex FA + (MC + L) with this property: 

3 The real time performance of an automaton is usually called its "behavior" 
and is to be contrasted with the computed answer or "computation" of an 
automaton. For a discussion of behavior and computation in finite and infinite 
systems see Burks, "Toward a Theory of Automata Based on More Realistic 
Primitive Elements," McNaughton, "On Nets Made up of Badly Timed 
Elements," and Holland, "Universal Embedding Spaces for Automata." 



AUTOMATA SELF-REPRODUCTION 



271 



for each Turing machine M, there is a coded description C £>'(M ) 
such that, when £)'(M) is stored on L, M u will simulate M, that is, 
M u will compute the same result that M computes. Analogously, the 
universal constructor M c is a complex CU + (MC + L) with this 
property: for each initially quiescent secondary automaton M, there 
is a coded description £)(M) such that, when 3D(M) is stored on L, 
M c will construct M. 4 

Thus the essential step remaining in von Neumann's program to 
achieve construction, construction universality, and self -reproduction 
in his cellular structure is to design the construction unit CU. 

5.2 The Universal Constructor CU + (MC + L) 

5.2.1 The constructing arm. Von Neumann's universal constructing 
automaton consists of a constructing unit CU combined with a tape 
unit MC + L (Sec. 4.1.1). He calls this the "primary automaton/' 
and he calls the initially quiescent automaton to be constructed the 
"secondary automaton" (Sec. 1.4). Since the tape unit has been 
designed (Sec. 5.1.2), it remains to design the constructing unit. 

We will first discuss the overall arrangements for construction. See 
Figure 50. This figure is not drawn to scale, the universal constructor 
being very much larger than shown here (cf. Fig. 37). Also, the loop 
for reading L is not shown, nor the arrangement for lengthening and 
shortening the loop (cf. Figs. 37 and 51). 

Some point on the universal constructor is designated as the origin 
(0,0) of a coordinate system for locating the secondary automaton. 
The secondary automaton occupies a rectangular area of width a 
and height (3, the lower left-hand corner of which is located at Xi , yi 
with respect to the origin (Fig. 50). 5 For the sake of simplicity we 
will confine the secondary automaton to the first quadrant, so that 
Xi ^ 0 and yi ^ 0. The internal structure of the secondary automaton 
is completely characterized by the sequence for i = 0, • • • , 
a — 1 and j = 0, • • • , /3 — 1, where each A t y specifies one of the 
quiescent states U, T Ma0 (u = 0, 1; a = 0, 1, 2, 3), and Coo . 

4 Compare the discussion of a universal Turing machine in the Second Lec- 
ture of Part I with the discussion of the universal constructor in Sec. 5.3.1 
below. See also Burks, "Programming and the Theory of Automata. " 

The description of 3D'(M) which is given to a universal Turing machine is 
usually coded differently than the description 3D(M) which is given to a uni- 
versal constructor. is coded in terms of the states of the finite part of 
M, whereas 3D(M) is coded in terms of the initial states of the cells of M. See 
the next section. 

5 Cf. Sec. 1.5.2. In von Neumann's reference system, a point such as (xi , 
yi) is located at the center of a cell rather than at a corner of a cell. 



272 



THEORY OF SELF-REPRODUCING AUTOMATA 



The general procedure for construction is this. A coded representa- 
tion of Xi , yi , a, 0, and the X tj is placed on the linear array L. The 
constructing unit CU reads this information with the aid of the 
memory control MC, interprets the information, and acts on it. The 
primary automaton operates on the (possibly remote) secondary 
area by means of a "constructing arm" or information path which 
extends from the primary automaton to the secondary area. The 
constructing unit CU first builds the constructing arm out to the 
secondary area. Then CU sends signals down the arm which con- 
struct the secondary automaton and provide its starting stimulus. 
Finally, CU withdraws the constructing arm. 

It is clear that the first step in designing the constructing unit 
CU is to design the constructing arm which will be operated by CU 
under control of the information received from the tape unit MC + L. 
Actually, we have essentially used such a constructing arm before. 
Figure 14 illustrates a procedure for extending a constructing path, 
modifying a remote cell, and withdrawing the constructing path. 
This same procedure is used for lengthening and shortening the loops 
Ci and C 2 and writing in cell x n of the linear array L (Sec. 4.2 and 
Figs. 32-36). In this case, the upper half of each loop constitutes 
the constructing path. 

In both of these cases a single path of transmission states (some- 
times ordinary, sometimes special) goes from the constructing device 
to the area of construction. For this reason we will call this von Neu- 
mann's "single path construction procedure." Let us look at this 
procedure in detail to see if it can be employed for the constructing 
arm needed now. 

The construction of the secondary automaton takes place at the 
end of the constructing path, in accordance with the rules summarized 
in Figure 9. The two processes involved are the direct process (for 
construction) and the reverse process (for destruction). Destruction 
is as necessary as construction, for the operating terminus of the 
construction path can be withdrawn only by changing a transmission 
state T wa o into an unexcitable state U. Construction can be accom- 
plished in a given cell by feeding pulses into this cell from either an 
ordinary or a special transmission state, but destruction requires the 
proper kind of transmission state: special kills ordinary but not 
special, and ordinary kills special but not ordinary. This distinction 
is necessary if information signals are to be distinguished from de- 
struction signals (Sec. 2.6.2.2). 

Thus the operations at the end of the constructing path sometimes 
require special transmission states and sometimes require ordinary 



AUTOMATA SELF-REPRODUCTION 



273 



transmission states. In Von Neumann's single path construction 
procedure this is accomplished as follows. The beginning of the 
construction path is fed by both an ordinary transmission state (e.g., 
cell Bl of Fig. 14) and by a special transmission state (e.g., cell BS 
of Fig. 14). Whenever the cell at the operating end (head) of the 
constructing path needs to be changed from an ordinary transmission 
state to a special transmission state (or vice versa), the whole path 
is changed from ordinary transmission states to special transmission 
states (or vice versa). Compare Figure 32a with Figure 32b (and 
Figure 32c with Figure 32d). 

Each change of the single construction path from ordinary trans- 
mission to special transmission states (or vice versa) requires a pulse 
sequence of length proportional to the length of the path. For example, 
each such modification of the upper half of loop Ci (and of loop C 2 ) 
was accomplished with a sequence of length 6n, where n is the number 
of cells to be changed. The number n was represented by the loops 
Ci and C2 , each of these loops being approximately 2n cells long. 
A delay of 6n units was obtained by using a triple return counter 
with a loop (Ci or C 2 ) as its responding organ (Sec. 4.2). 

This single path construction procedure could be used for the 
constructing arm which is operated by the universal constructor. 
The arm would consist of a single path of transmission cells (some- 
times ordinary, sometimes special) going from the universal con- 
structor to the secondary area, and fed by both ordinary and special 
transmission states. Let I be the number of cells in this single path. 
To change this path from ordinary to special transmission states 
(or vice versa), CU would send a sequence of length 6^ into it. CU 
could determine I from the numbers x\ , yi , a, 0, i, j, which specify 
the location and size of the secondary automaton as well as the precise 
place within the secondary where the constructing arm terminates. 
The numbers Xi , y x , a, and (i are stored as L in explicit form, and 
CU could infer i and j by counting the position of X tJ in the sequence 
of X's. 

Though von Neumann could have used the single path construc- 
tion procedure for the constructing arm, he actually planned to use a 
different, and better, procedure. Four pages of rough notes accom- 
panied his manuscript "Theory of Automata: Construction, Repro- 
duction, Homogeneity,' ' which constitutes the present Part II of the 
present volume. These pages contain his design for a constructing 
arm and an outline of a program for controlling it. His program is too 
sketchy to be reconstructed, but his intended design of the construct- 



274 



THEORY OF SELF-REPRODUCING AUTOMATA 



ing arm is plain enough, and once this is known it is not difficult to 
write a program for it. 

Von Neumann's constructing arm is shown in Figures 44-50. 
Ordinary transmission states are represented by single arrows, and 
special transmission states are represented by double arrows. This 
constructing arm consists of two adjacent, parallel paths, terminating 
at a head in which one path normally points at the other path. One 
path is of ordinary transmission states, and the other path is of special 
transmission states. Thus, both ordinary and special transmission 
states are always available at the head for the reverse (destruction) 
process. This being so, it is never necessary to change a whole path 
from ordinary to special transmission states or vice versa, as in the 
single path construction procedure. 

The constructing arm may be fed by the constructing unit CU 
either from the left (using inputs s and o of Fig. 44) or from below 
(using inputs s and o of Fig. 44). The arm may turn a corner, as in 
Figure 50. 

Von Neumann's procedures for operating the arm, slightly modi- 
fied, are shown in Figures 44-50. We use a new method of symbolizing 
pulse sequences in these figures. A pulse sequence which destroys 
and constructs is represented by a sequence of symbols indicating 
the quiescent states produced by this pulse sequence. 

We will explain this method of symbolizing pulse sequences in 
connection with the transition from Figures 45a-45b. If the following 
pulse sequences are fed into s or s from special transmission or con- 
fluent states, they produce the effects indicated. 

1110 changes cell CI from U to H 
1101 changes cell C2 from U to <= 
I changes cell B2 from f to U 
10000 changes cell B2 from U to 

Thus 11101101110000 into s or s is indicated by 

II <= U — » into s or s . 

This method of symbolizing pulse sequences must be used with the 
constraints of von Neumann's transition rule in mind (Ch. 2). For 
example, the sequence 

II H U into s or s 

is not an allowable sequence for Figure 45b, because one special 
transmission state cannot kill another special transmission state. 
However, if the appropriate conditions are satisfied, a single pulse 



AUTOMATA SELF-REPRODUCTION 



275 



will change a cell into a U. Likewise, if the appropriate conditions are 
satisfied, 

10000 changes U into -> 

1 0001 changes U into ] 
1001 changes U into <— 

1010 changes U into [ 

1011 changes U into =» 

1100 changes U into tt 

1101 changes U into 

1110 changes U into Jl 

1111 changes U into C. 

Figures 45 and 46 show the procedure for advancing the construct- 
ing arm 1 unit, either horizontally or vertically. Figure 47 gives the 
procedure for withdrawing the arm 1 unit horizontally and leaving 
the two vacated cells in the desired quiescent states y and 5. The 
pulse sequences used for y and 8 depend, of course, on the quiescent 
states desired. For example, if the variable y has the value C, the 
sequence fed into Figure 47e becomes 

U => U C into s or s. 

This expression represents 

1101111111 into s or s. 

Figure 48 gives the procedure for withdrawing the constructing arm 
one unit vertically and leaving the vacated cells in the desired quies- 
cent states 7 and 5. Figure 49 shows how to inject a starting stimulus 
into the secondary automaton. 

This completes the description of von Neumann's five operations 
for the constructing arm: horizontal advance, vertical advance, 
horizontal retreat with y-5, vertical retreat with y-d, and injection 
of the starting stimulus. These operations suffice for the construction 
of any initially quiescent automaton in the first quadrant (i.e., with 
^i^0 and yi ^ 0). We will now state an algorithm composed from 
these operations which achieves this result. 

This construction algorithm presupposes that /3 is an even integer. 
If p is odd, a row of U's can be added to the secondary to make 
even, or the algorithm may be modified slightly. The algorithm also 
presupposes that the secondary's starting stimulus is to be injected 
from below into the cell whose center is at (xi + \ , yi + \). Other- 
wise, the instructions must be modified. 



270 



THEORY OF SELF-REPRODUCING AUTOMATA 



The algorithm for constructing and starting a secondary automaton 
on the plan of Figure 50 is : 

(1) The constructing arm is extended from the primary automaton 
to the upper left-hand corner of the area of the secondary autom- 
aton. This requires x\ + 2 horizontal advances and then yi + f3 
vertical advances. 

The constructing arm is now ready to advance to the right of the 
secondary area and then to retreat, constructing two rows of the 
secondary as it retreats. 

(2) The following sequence of operations is repeated times. 

(a) The horizontal advance is repeated a — 2 times. 

(b) The horizontal retreat with y-d is repeated a — 2 times. 

(c) The vertical retreat with y-d is repeated twice. 

At the end of operation (2) the secondary automaton is complete 
and may now be started. 

(3) A starting stimulus is injected from below into the cell located 
at Oi + }). 

The secondary automaton now begins to operate, and the construct- 
ing arm may be withdrawn. 

(4) The constructing arm is withdrawn to the primary automaton by 
yi vertical retreats with 7-5 followed by X\ + 2 horizontal re- 
treats with 7-5, both y and b always being the unexcitable state 
U. 

This concludes the algorithm. The constructing arm could now be 
used to construct another secondary automaton. 

The pulse sequences which execute this algorithm are injected into 
inputs s and o of the constructing arm. These pulse sequences are a 
function of the information stored on the linear array L. The way 
the universal constructor transforms the passive information on L 
into the correct pulse sequences will be explained in Section 5.2.3. 

Von Neumann's five constructing arm operations are adequate for 
construction in the first quadrant, but not for construction in the 
other quadrants. However, it is not difficult to redesign the head and 
to program operations for advancing to the left, retreating from the 
left, advancing down, etc. With these additional operations, the uni- 
versal constructor could construct a secondary automaton in any 
quadrant of the plane. This presupposes, of course, that the area for 
the secondary automaton consists of unexcitable states, that there 
is a sufficiently wide path of unexcitable states from the universal 
constructor to this area, and that no other automaton interferes with 
the construction process. 

It is clear from Figures 45-49 that the execution of each of von 



AUTOMATA SELF-KEPRODUCTION 



277 



Neumann's five constructing arm operations is accomplished by a 
finite pulse sequence. The longest sequence needed is that for a 
vertical retreat with y-8 ; if y and 5 are either — -> or f , this operation 
requires a sequence of length 47. Thus the length of the pulse sequence 
required for each of these five operations is a constant, independent 
of the length of the constructing arm. In contrast, von Neumann's 
one-path construction procedure requires a pulse sequence whose 
length depends on the length of the path; if i is the length of the path, 
a sequence of length bt or greater is required. In this respect, von 
Neumann's two-path construction procedure is far superior to his 
one-path construction procedure. This superiority can have a pro- 
found effect on the organs supplying the pulse sequences for construc- 
tion, as we shall now see. 

5.2.2 Redesign of the memory control MC. Von Neumann's two- 
path construction procedure can be used for operating the linear array 
L. The new arrangement is shown in Figure 51 ; it should be compared 
with the old arrangement of Figure 37. 

The path for reading cell x n starts at input v, extends along row 
1, passes through cell x n , returns along row 4, and terminates at 
output w. Row 1 is also the ordinary transmission path of the two- 
path constructing arm. Row 2 is the special transmission path of the 
constructing arm. The head of the constructing arm consists of cells 
C2, Dl, and D2\ it differs slightly from the head of Figure 44a. The 
stimuli into inputs u and v must come from ordinary transmission 
states. Note that cells Al and A2 do not affect each other, since a 
confluent state does not feed a confluent state. 

The reading process is exactly the same as before (Sees. 4.1.1 and 
4.1.5). The sequence 10101 is injected into v from a pulser P (10101). 
This sequence will travel along row 1, down column D, and into 
cell x n . What happens next depends upon whether cell x n is in state 
U, signifying a "zero," or in state j , signifying a "one." 

(1 ) If cell x n is in state U, the first part 1010 of the sequence changes 
x n into \ , and the remaining part T of the sequence travels 
back along row 4 and out at exit w. 

(2) If cell x n is in state j , the complete sequence 10101 travels 
through x n , back along row 4, and out at exit w. 

Thus a T at w signifies a "zero," while a 10101 at w signifies a "one." 
These two sequences may be discriminated by means of the T vs. 
10101 discriminator (Sec. 3.5 and Fig. 25), as before. 

At the end of the reading process, the reading and constructing 
paths are left as in Figure 51b, with cell x n in state [ . The next step 
is to lengthen or shorten the reading and construction paths of L, 



278 



THEORY OF SELF-REPRODUCING AUTOMATA 



and to leave cell x n in state U (representing a "zero") or state [ 
(representing a "one"). There are four alternatives: 

(LO) Lengthen and leave cell x n in U 
(LI ) Lengthen and leave cell x n in [ 

(50) Shorten and leave cell x n in U 

(51 ) Shorten and leave cell x n in J, . 

These operations are similar to the operations of horizontal ad- 
vance (Fig. 45), vertical advance (Fig. 46), horizontal retreat with 
y-8 (Fig. 47), and vertical retreat with y-d (Fig. 48). We will give 
the pulse sequences for only one case, that of (LI), i.e., lengthening 
and leaving cell x n in { . These are given in Figure 52, where the 
method of symbolizing sequences is the same as in the earlier figures. 
For example, starting with the situation of Figure 52a, the sequence 
1111011110110111001 of ordinary stimuli into u passes through the 
confluent state into the row of special transmission states and travels 
down this row. This sequence changes cell x n , the cells above and 
below it, and the cell below x n + 1, into II , Ji , =>, and <— , respec- 
tively. Figure 52b shows the situation produced by this sequence. 

Thus this operation (LI) is accomplished by sending a sequence 
into u, then a sequence into v, another sequence into u, and another 
sequence into v. Since the absence of a stimulus is represented by 
"0," this operation can be accomplished by sending simultaneously 
a sequence into u and a sequence into v\ see the example of Figure 
14 and Section 2.8.3. These two sequences can be produced by pulsers 
which feed into u and v and which are stimulated in the proper phase 
relative to one another (Sec. 3.2.1). The other lengthening, shorten- 
ing, and writing operations (L0), (SO), and (SI) can be handled 
similarly. 

It is important that each of the operations (L0), (LI), (SO), and 
(SI) may be accomplished by pulse sequences whose lengths are 
independent of the lengths of the construction paths, i.e., independent 
of the index n of the cell x n which is under scan. In contrast, von 
Neumann's method of operating the linear array L requires pulse 
sequences whose lengths depend on the length of the construction 
path (Ch. 4). As a consequence, the memory control for the new 
method of operating L can be much simpler than von Neumann's 
memory control MC (Figs. 37, 39-41). In particular, those aspects 
of von Neumann's MC concerned with obtaining and controlling a 
delay of 6n (where n is the subscript of x n ) are unnecessary in the 
new method. 

Since von Neumann's design of MC, as modified in Section 5.1.2, 



AUTOMATA SELF-REPRODUCTION 



279 



does work, we will not redesign MC here. The read-write-erase unit 
RWE for the new method of operating L can be constructed from 
nine pulsers and a I vs. 10101 discriminator, suitably arranged. By 
recoding and simplifying the control signals which pass between MC 
and the constructing unit CU, one can eliminate the read-write- 
erase control RWEC, so that MC will consist only of the simplified 
RWE unit and a coded channel. 6 

At this point I should like to speculate on von Neumann's thoughts 
concerning his design for a self -reproducing automaton at the time 
he stopped working on the manuscript reproduced as Chapters 1-4 
above. His design had turned out to be much more complex than 
he had anticipated. 7 After developing his two -path construction 
procedure, he must have realized that it could be used on L, and 
that this would greatly simplify the design of MC and of the whole 
machine. Realizing this, he would have wanted to redesign his self- 
reproducing automaton along new lines. This redesign would have 
entailed revising Chapter 3 above and starting Chapter 4 afresh. 
Von Neumann never found the time to revise and complete the 
manuscript in this way. 

Even though von Neumann's design of the memory control MC 
can be greatly improved, it is nevertheless important. Historically, 
it constitutes the first proof that a tape unit with unlimited memory 
can be embedded in his 29 -state cellular structure. Moreover, it con- 
tains many ingenious design techniques for parallel data processing 
in this cellular structure. 

5.2.3 The constructing unit CU. After he had designed the con- 
structing arm, there remained for von Neumann the task of designing 
the constructing unit CU itself. As he recognized, CU is a finite 
automaton which interacts with the tape unit MC + L and also 

6 The complete design of a memory control for a variant of the new method 
of operating L is given in Thatcher's "Universality in the von Neumann Cellu- 
lar Model." 

7 In a letter to Miodrag Muntyan of the University of Illinois Press, dated 
Nov. 4, 1952, von Neumann says of his manuscript: 

I have written so far the first chapter, which amounts to about 40 type- 
written pages. ... I am now working on the second chapter which, I 
expect, will be somewhat longer, perhaps about twice as long. I will 
also have a third chapter and possibly a fourth one, but the length of 
these is still uncertain. Also, when the whole is finished, I will have to 
go over the entire text once more, and this operation is likely to in- 
crease the length some more. 

Compare this with von Neumann's letter to Goldstine in Sec. 1.1.2.3 above. 
These statements show that after completing Ch. 1 above, von Neumann 

thought he could develop the design (starting with Ch. 2 above) in a chapter 

about twice as long as Ch. 1. 



280 



THEORY OF SELF-REPRODUCING AUTOMATA 



performs the function of construction. Hence the same kinds of 
organs and design principles used in the design of the memory control 
MC (Ch. 4) and an arbitrary finite automaton FA (Sec. 5.1.3) can 
be used to design CU. Since von Neumann was a skilled designer and 
programmer, 8 he undoubtedly saw how to design the constructing 
unit CU, and he may even have had a quite specific design plan in 
mind. 

While it is not appropriate for us to work out the full design here, 
we will say enough to show that a workable constructing unit CU 
does in fact exist. A complete design for a constructing unit (as well 
as for a universal constructing machine) is given in James Thatcher's 
"Universality in the von Neumann Cellular Model." 9 

Von Neumann discussed the process of constructing several sec- 
ondary automata (Sec. 1.7), but it will suffice here to explain the 
construction of a single secondary automaton in the first quadrant. 
The information concerning the location and size of the secondary 
automaton, as well as a complete description of the secondary autom- 
aton, is stored on the linear array L, as in Figure 50. First comes a 
period. Then come the location and size parameters Xi , yi , a, and /3, 
each followed by a comma. Next comes the sequence of X f /s describing 
the secondary automaton cell by cell, for i = 0, • • • , a — 1 and 
j = 0, • • • , j8 — 1. For the sake of simplicity we assume that the 
X»/s are stored in the order in which they are used by CU. The se- 
quence of X t /s is terminated by a period, which also marks the end of 
the information on the tape. 

This information may be coded in an alphabet of 14 characters: 
zero, one, comma, period, and 10 values of the . These 10 values 
of the \{j correspond to the 10 quiescent states T ua o {u = 0, 1 and 
a = 0, 1, 2, 3), C 0 o , and U. These 14 characters may be represented 
by four binary digits (bits). It is convenient to avoid the all zero 
sequence "0000." It is also convenient to employ a fifth bit position 
for marking purposes. Thus each character is represented by a total 
of five bits and is stored in five successive cells of the linear array L. 

The number x\ will be represented by a sequence of X\ + 1 of the 
five-bit characters representing "one," and similarly for the numbers 

8 See pp. 6-15 of the "Introduction" to the present work, as well as Chs. 
3 and 4. 

9 See also Codd, "Propagation, Computation, and Construction in Two- 
Dimensional Spaces." This contains a design for a universal constructing 
machine in a cellular system in which each cell has four immediate neighbors, 
but only 8 states, as contrasted with von Neumann's 29 states. Dr. Thatcher 
and Dr. Codd were acquainted with von Neumann's manuscript "The Theory 
of Automata: Construction, Reproduction, Homogeneity" in manuscript form. 



AUTOMATA SELF-REPRODUCTION 



281 



7/1 , a, and 13. This method of representation can clearly be improved, 
but it will simplify our discussion to represent all information in the 
same alphabet. 

Let 0 0 , 0i , 02 , #3 , 04 be the five bits of a character, with 0 O being 
the marker bit. The constructing unit CU must be capable of inter- 
preting the sequence 0i , 0 2 , 03 , 04 as a unit. Suppose, for example, 
that Coo is represented by -1010 on L, where the dash indicates the 
position of the marker bit. If the character X t y on L is -1010, CU must 
put cell (i, j) of the secondary automaton in state C 0 o at the appro- 
priate stage of the construction process. CU will accomplish this by 
replacing y or d in the sequences for horizontal or vertical retreat 
(Figs. 47 and 48) by the sequence 1111, which is the sequence re- 
quired by the transition rule (Fig. 10) for constructing C 0 o . Hence 
CU must interpret the tape character -1010 as calling for the sequence 
1111 in the appropriate context. 

Now the four bits of 0i , 0 2 , 03 , 04 are stored in successive cells 
Xn+i , x n+2 , £ n +3 , #n+4 . When CU so instructs MC, MC will read a 
cell and advance the reading loop to the next cell. The amount of 
time required for this process is a linear function of the index n. 
Consequently, the four bits 0 X , 0 2 , 03 , and 0 4 are received by CU at 
widely varying times. In von Neumann's terminology (Sec. 3.1.1), 
the sequence 1010 comes to CU freely timed, and in response CU 
must send the rigidly timed sequence 1111 down the construction 
arm. The four pulses of 1111 must enter the constructing arm at 
successive times (r, r + 1, r + 2, r + 3), since rigid timing is re- 
quired by the direct process. 

Hence the constructing unit CU must be able to convert the freely 
timed sequence 0i , 0 2 , 03 , 04 (e.g., 1010) into a rigidly timed sequence 
(e.g., 1111). This may be accomplished in either of two ways. 

The first method employs state organs like Figure 43. Associate 
with the 16 characters 0i0 2 0 3 04 30 state organs SO 0 , SOi ; SOoo , • • • , 
SOn ; SOoooo , • • • , SOmi , which are to be interconnected by the 
coded channel of CU. These organs are activated (take control) 
under the influence of the bits 0 X , 0 2 , 0 3 , 04 in the following manner. 
When 0i is transmitted by MC to CU, it has this conditional effect: if 
0i is zero, SOo is activated; while if 0 X is one, SOi is activated. Next, 
the bit 0 2 shifts control from SO 0 or SOi to one of the state organs 
SOoo , SOoi , SOio , SOn according to the rule: if 0 2 is zero, SO^o is ac- 
tivated, while if 0 2 is one, SO^i is activated. That is, after CU has 
received the two bits 0 X and 0 2 from MC, the state organ SO^t, 
will be "in control. " This process is repeated for the remaining bits 0 3 
and 0 4 , so that after all bits of 0i0 2 0 3 0 4 have been read from L, ex- 



282 



THEORY OF SELF-REPRODUCING AUTOMATA 



actly one of the 16 state organs SOe^e^ of CU will be activated. 
This state organ then controls the selection of a rigidly timed se- 
quence. For example, the state organ SO1010 will cause 1111 to be 
substituted for y or 8 in one of the constructing sequences of Figures 
47 and 48. 

The conversion of the freely timed sequence 1010 into the rigidly 
timed sequence 1111 is a static -dynamic conversion. Our second 
method of making this conversion employs the static -dynamic con- 
verter of Figure 53, together with four state organs SO 1 , SO 2 , SO 3 , 
and SO 4 to keep track of the four digit positions 0i , 0 2 , 03 , and 0 4 , 
respectively. The inputs and outputs of the static-dynamic converter 
and of these four state organs are all connected to the main channel 
of CU. The periodic pulsers of Figure 53 are copies of the alternate 
periodic pulser PP(T) of Figure 20. The organs in the left-hand col- 
umns of Figure 53 are all decoders which receive their inputs from 
the main channel. The figure is not, of course, drawn to scale. 

The static -dynamic converter and the four state organs SO 1 , SO 2 , 
SO 3 , SO 4 convert a freely timed sequence 0i , 0 2 , 03 , 04 into the 
corresponding coded rigidly timed sequence in the following way. 
At the beginning of this conversion the state organ SO 1 is in control 
and directs the reading of 0i . After 0i is read by MC from L it is 
transmitted to CU: a pulse from exit ii of MC (Fig. 37) signifies that 
0i is "zero," a pulse from exit i 2 of MC signifies that 0i is "one." 
Under the control of SO 1 , the pulse signifying "one" is coded into a 
sequence which is recognized only by the Starti decoder of Figure 53 
(input di) and control is transferred to SO 2 . Hence if 0i is "one," the 
PP(T) for 0! is activated, while if 0i is "zero," the PP(1) for 0 X is left 
in the inactive state. 

Similarly, bit 0 2 is transferred to the PP (T) for 0 2 by means of state 
organ SO 2 and the Start 2 decoder of Figure 53. The bits 0 3 and 0 4 are 
handled in the same way. Hence, after the character -01020304 has 
been read by MC from L and transmitted to CU, the periodic pulsers 
of the static -dynamic converter represent 0i0 2 0304 , the j'th periodic 
pulser being off or on according to whether 0y is "zero" or "one" 
( j = 1, 2, 3, 4). The control unit CU next directs the conversion of 
this static information into a rigidly timed sequence. 

The static representation of the character — 0i0 2 0304 is converted 
into the corresponding dynamic sequence 0i0 2 0 3 0 4 by a single pulse 
sequence of the main channel which is recognized by the conversion 
decoders Di , D 2 , D 3 , and D 4 of Figure 53. The operation of each 
decoder Dy is as follows, for,; = 1, 2, 3, 4. Dy emits a pulse which is 
delayed in Ay and then impinges on the confluent state of column A, 



AUTOMATA SELF-REPRODUCTION 



283 



row j. If the PP (1 ) for 6j is on, this pulse is passed and enters channel 
B; otherwise, this pulse is blocked and does not enter B. The pulse 
sequence of the main channel which is recognized by the decoders 
Di , D 2 , D 3 , and D 4 enters these four decoders at different times. But 
the delays Ai , A 2 , A 3 , and A 4 can be adjusted so that the pulses 
which do enter channel B are in proper phase. For example, if 61 , 
62,63, and 64 are all one, the sequence 1111 will be emitted from 
output g. 

Since the all-zero sequence is not used to represent any character, 
61 , 6 2 , 03 , and 64 cannot all be zero, and at least one stimulus will be 
emitted from g. Hence, stimulation of the conversion decoders Di , 
D 2 , D 3 , and D 4 will cause the rigidly timed sequence 61626264 to leave 
the static-dynamic converter at exit g. The periodic pulsers of the 
static -dynamic converter can then be "cleared" back to the inactive 
state by stimulating the four stop decoders, whose inputs are labeled /. 

This completes the discussion of the two methods for converting a 
static tape character into a dynamic sequence. The second method 
requires less apparatus than the first. 

We now have a method for reading characters from the tape. At 
the end of Section 5.2.1 we gave an algorithm for constructing and 
starting a secondary automaton. This algorithm describes the pulse 
sequences to be injected into the constructing arm as a function of the 
information X\ , yi , a\ , j8, A 0 o , ■ • • , Aa-i, 0-1 stored on L. The design 
of the constructing unit CU now becomes the task of translating this 
algorithm into a machine design. We restate this algorithm here, 
showing how CU obtains the necessary information for the construc- 
tion from the linear array L. We assume that initially the reading 
loop of L passes through cell x 0 , i.e., through the marker bit of the 
leftmost period on L. 

The revised algorithm for constructing and starting a secondary 
automaton (see Fig. 50) is: 

(1) CU extends the constructing arm from itself to the upper left- 
hand corner of the area of the secondary automaton. This is 
accomplished by the following two sub-operations: 
(a) CU sends pulse sequences into the constructing arm for 
Xi + 2 horizontal advances. The pulse sequences for each 
horizontal advance are given in Figure 45. CU senses a 
character (tally) of Xi on L, sends the sequences of Figure 
45, and then moves right to the next character on L. When 
CU reads the comma on L, it sends pulses for two more 
horizontal advances, and then goes to execute sub-opera- 
tion (b). 



284 THEORY OF SELF-REPRODUCING AUTOMATA 

(b) CU sends pulse sequences into the constructing arm for 
+ vertical advances. It does this as follows. CU senses 
each character (tally) on L, sends the sequences of Figure 
46 to the constructing arm, and advances to the next 
character on L. When CU reaches the comma on L, it 
passes over a to 0, sends the sequences of Figure 46 for 
each character of 0, and then returns to the comma pre- 
ceding a. 

The constructing arm is now in position to begin construction. 
CU constructs two rows of the secondary automaton at a time, and 
must do this 0/2 times. CU keeps count of these operations by mark- 
ing the marker positions of the characters of 0. 

(2) CU senses if 0 contains any unmarked characters. If 0 contains 
no unmarked characters, CU unmarks all the characters of 0 
and all the X's, and then proceeds to operation (3). If 0 contains 
an unmarked character, CU marks two characters of 0 and exe- 
cutes sub -operations (a), (b), and (c) in that order. 

(a) CU repeats the horizontal advance a — 2 times. It accom- 
plishes this by passing over the first two tallies of a and 
then sending the sequences of Figure 45 into the construct- 
ing arm for each of the remaining tallies of a. 

(b) CU repeats the horizontal retreat with y-8 for a — 2 times. 
It does this by marking two characters of a, and then exe- 
cuting the following operation until all characters of a 
are marked : mark an unmarked character of a and execute 
the horizontal retreat with y-8 of Figure 47. 

The horizontal retreat with y-8 requires the substitution 
of pulse sequences for y and for 8 according to the states 
\n and to be constructed in the cells at the terminus 
of the constructing arm. To obtain these X's, CU must find 
them on L and move the reading loop from a to them; 
later CU must return the reading loop to a. In both cases, 
CU can sense the place to stop by means of markers: each 
time it uses a character of a or a X, CU will mark it. We are 
assuming that the X's are placed on L from right to left in 
order of use. 

After CU has executed the horizontal retreat with 7-5 
for a — 2 times, it removes the markers from a and proceeds 
to sub-operation (c). 

(c) CU repeats the vertical retreat with y-8 twice, using and 
marking the next two X's on L. 

At the end of operation (2), CU has completed the construction of 
the secondary automaton and proceeds to start it. 



AUTOMATA SELF-REPRODUCTION 



285 



(3) CU injects the starting stimulus into the secondary automaton 
by means of the sequences of Figure 49. Exit e of this figure is 
an input to the cell located at (zi + |, yi + §), and so this 
presupposes that the secondary automaton is designed to receive 
its starting stimulus through the bottom of this cell. 

The secondary automaton now begins to operate and CU proceeds 
to withdraw the constructing arm back to itself. 

(4) CU withdraws the constructing arm by sending sequences into 
it for yi vertical retreats with 7-5 followed by X\ + 2 horizontal 
retreats with 7-5. The required sequences are given in Figures 
48 and 47, respectively. In every case both y and 5 are to be U, 
which means that single pulses are to be used for each occurrence 
of 7 and of 5. CU keeps count of these retreats by means of the 
numbers yi and X\ on L. 

This concludes the algorithm for constructing and starting a secondary 
automaton. Note that at the end of the algorithm the universal 
constructor CU + (MC + L) is again in its starting state. 

Now that this algorithm is formulated, the design of the construct- 
ing unit CU reduces to the problem of translating this algorithm 
into a machine design. This can be done by using state organs SO 
like Figure 43, interconnected by a coded channel, with the specific 
interconnections between each state organ and the other state organs 
reflecting the content of the algorithm. Control within CU is shifted 
from one state organ SO to another under the influence of the infor- 
mation on L according to the content of the algorithm. Note that 
the memory control MC operates in a similar manner, with the con- 
trol organs CO playing the role of state organs. 

In Section 5.1.3, we noted the resemblance of the constructing 
unit CU to a finite automaton FA, and the similarity of von Neu- 
mann's universal constructor to Turing's universal computing ma- 
chine. Let us look at these resemblances more closely. 

A universal computing machine M u has two parts: a tape unit 
MC + L and a finite automaton FA which interacts with this tape 
unit. Correspondingly, the universal constructor M c has the two 
parts MC + L and CU. The constructing unit CU performs two 
interrelated functions: it interacts with MC + L and it constructs 
the secondary automaton whose description is stored on L. The 
processes involved in constructing a secondary automaton are not 
novel: these processes are already employed in the tape unit MC + 
L. More specifically, the reading loop is moved from one cell to 
another cell on L by means of the same kind of construction and 
destruction steps used in constructing a secondary automaton. Thus 
the constructing arm of Figures 44-50 is very similar to the construct- 



286 



THEORY OF SELF-REPRODUCING AUTOMATA 



ing arm of Figures 51-52, and either arm could be used in place of 
the other. 

This comparison shows that the constructing unit CU is really a 
special kind of finite automaton, and that the same kinds of organs 
and design principles used in the design of the memory control 
MC (Ch. 4) and an arbitrary finite automaton FA (Sec. 5.1.3) can 
be used to design CU. This comparison shows also that in the con- 
text of von Neumann's cellular structure, the output of the universal 
computing machine M u is not as different from the output of the 
universal constructor M c as it might seem. The output of M u is a 
computation, while the output of M c is a construction, but both are 
accomplished by sending signals into constructing arms. 

This concludes our discussion of the constructing unit CU. We 
have shown that it can be embedded in von Neumann's 29-state 
cellular structure, and we have stated the general principles of its 
design. 

The constructing unit CU together with the tape unit MC + L 
constitutes a universal constructor. Hence there can be embedded in von 
Neumann's 29-state cellular structure a universal constructor M c with 
this property: for each initially quiescent automaton M, there is a coded 
description £>(M) of M such that, when £)(M) is placed on a tape L 
attached to M c , M c will construct M. 

This answers von Neumann's question about construction uni- 
versality : can any single automaton construct every other automaton 
(Sec. 1.1)? The only question remaining is his question about au- 
tomata self -reproduction: can an automaton construct copies of 
itself? We will present an affirmative answer to this question also, 
after first summarizing von Neumann's accomplishments in the 
present work. 

5.3 Conclusion 

5.3.1 Summary of the present work. Von Neumann began the pres- 
ent Part II by asking five main questions, some of which contain 
sub-questions (Sec. 1.1.2.1). The first main question concerns 
(A) Logical universality: 

(Al) When is a class of automata logically universal? 
(A2) Is any single automaton logically universal? 
A finite automaton with an indefinitely extendible tape is called a 
Turing machine (see the end of the Second Lecture of Part I above). 
Turing showed that the class of Turing machines is logically universal 
in the sense that any logical process (computation) that is at all 
performable by finite but arbitrarily extensive means can be per- 



AUTOMATA SELF-REPRODUCTION 287 

formed by a Turing machine. Turing further showed that there is a 
universal Turing machine, that is, a Turing machine which can 
perform any given computation. 

Thus, as von Neumann stated, Turing has answered these two 
questions about logical universality. Von Neumann then posed 
analogous questions about construction. 

(B ) Constructibility: 

(Bl) Can an automaton be constructed by another autom- 
aton? 

(B2) What class of automata can be constructed by a single 
suitable automaton? 

(C) Construction universality: Is any single automaton construction 
universal? 

(D ) Self -reproduction: 

(Dl) Is there a self -reproducing automaton? 
(D2) Is there an automaton which can both reproduce itself 
and perform further tasks? 
Von Neumann promised to answer all these questions affirmatively 
by constructive means, that is, by designing various kinds of con- 
structing and self -reproducing automata. 

Questions (C) and (D) lead to his last main question. 

(E) Evolution: 

(El ) Can the construction of automata by automata progress 
from simpler types to increasingly complicated types? 

(E2) Assuming some suitable definition of efficiency, can this 
evolution go from less efficient to more efficient au- 
tomata? 

Von Neumann made a few remarks relevant to evolution in Sections 
1.7 and 1.8 but never returned to this topic. 10 

After formulating the five main questions, von Neumann proceeded 
to make questions (B)-(D) more precise. In effect, the rest of Chapter 
1 and all of Chapter 2 are directed to this task. We will give a brief 
summary of the development of these chapters. 

Idealized neurons are adequate to handle the purely logical func- 
tions of automata, but construction requires organs which can per- 
form those non-logical functions which are required for the acquisition 
and combination of the organs of which the constructed automata 
are composed (Sec. 1.2). In his kinematic model, von Neumann 
introduced girders, sensing elements, kinematic elements, joining 
elements, and cutting elements to achieve these non-logical functions ( — 

10 See also our discussion of his "probabilistic model of self-reproduction 
and evolution'* at the end of Sec. 1.1.2.3. 



288 



THEORY OF SELF-REPRODUCING AUTOMATA 



(Fifth Lecture, Part I, and Sec. 1.1.2.3). In the present attempt, he 
preferred to avoid the kinematieal aspects of self -reproduction so that 
he could concentrate on the logical-combinatorial aspects of this 
topic (Sec. 1.3.1.1). He was thus led to work with a space (frame- 
work) in which the distinction between rest and motion is replaced by 
a distinction between quiescent states and active states (Sees. 1.3.2 
and 1.3.4.1). 

Von Neumann then put various restrictions on the space (frame* 
work) in which he would carry out his automata constructions. He 
wanted it to have a high degree of regularity. He required functional 
homogeneity, though not total homogeneity, because the latter is 
incompatible with computation and construction (Sec. 1.3.3.2). He 
^ further required isotropy, and selected a 2-dimensional space (Sec. 
1.3.3.3). Because of the difficulties in modeling automata construction 
in a continuous space, he decided to work with a discrete space (Sec. 
1.3.3.4; cf. Sec. 1.1.2.3). In sum, he decided to carry out his automata 
designs in a 2-dimensional, regular, cellular structure which is func- 
tionally homogeneous and isotropic. 

He then decided to model the growth of neurons (excitable cells) 
by the transformation of existing, unexcitable cells into excitable 
cells. Such a transformation cannot be induced by ordinary stimuli 
(the ordinary active states of neurons) since these control the logical 
functions. Hence, to accomplish construction in his cellular structure 
von Neumann introduced special stimuli, which cause transitions 
from the unexcitable state to different species of excitable states. Thus 
growth is modeled by the transformation of unexcitable cells to ex- 
citable cells by special stimuli (Sec. 1.3.4.2). This distinction between 
ordinary and special stimuli, though modified later (Sees. 1.3.4.3, 
2.5, and 2.6), is the basis for an answer to question (Bl), Can an 
automaton be constructed by another automaton? 

The next question is (B2), What class of automata can be con- 
structed by a single suitable automaton? Von Neumann referred to 
the constructing and constructed automata as the "primary" and 
"secondary" automata, respectively. In Section 1.4 he planned the 
general organization and mode of operation of a primary automaton 
which can construct any member of some infinite class of secondary 
automata. A description of the desired secondary automaton is to be 
given to the primary automaton. The main problem concerns the 
exact way this is to be done. Since there is no bound on the size of the 
secondary to be constructed by a single primary, these descriptions 
cannot be stored in the primary proper. Working with the universal 
Turing machine in mind, von Neumann introduced an indefinitely 



A I J TOM ATA SELF-REPROD l rCTION 



281) 



extendible linear array L, on which the description of any secondary, 
or series of secondaries can be stored. 

Thus the primary (constructing) automaton will consist of a 
finite part plus an indefinitely expandible linear array L. A construct- 
ing automaton is analogous to a Turing machine, which consists of a 
finite automaton plus an indefinitely expandible tape. Indeed, as von 
Neumann noted, the linear array L could also serve as an indefinitely 
expandible tape for a Turing machine if the finite automaton part of 
a Turing machine could be embedded in the cellular structure (Sec. 
1.4.2.3; cf. Sec. 5.1.3). The detailed problem of actually designing, for 
the cellular structure, finite automata which can interact with the 
linear array L and carry out constructions and computations on the 
basis of the information obtained from L, is solved in Sections 3.1 
through 5.2. Thus, by the end of Section 5.2, question (B) has been 
answered affirmatively. Moreover, question (A), as applied to von 
Neumann's cellular system, has also been answered affirmatively. 

In the balance of Chapter 1, von Neumann reduced questions (C) 
and (D) to question (B). He reduced question (C) to question (B) 
by outlining a plan for converting the primary (constructing) autom- 
aton into a universal constructor (Sec. 1.5). This plan is illustrated 
in Figure 50 for constructions in the first quadrant. The secondary 
automaton is a cells wide and 0 cells high, and its lower left-hand cell 
is located at (xi , yi). Let I be the number of states that each cell of 
the secondary is to assume, and let X = 0, 1, • • • , I — 1. The state 
of cell (i, j) is specified by A;y , where i = 0, 1, • • • , a — 1 and j = 
0, 1, • • • , j8 — 1. Hence the plan of any secondary automaton may be 
given to the universal constructor by placing the sequence X\ , yi , ex, 
iS, Aoo , • • • , Aa-i,,3_i on the tape L. The universal constructor can 
construct the secondary on the basis of the information contained in 
this sequence. This reduces question (C) to question (B). 

Von Neumann then reduced question (D) to question (C) by 
showing how to make the universal constructor reproduce itself (Sees. 
1.6, 1.7). In essence, he accomplished this by placing a description of 
the universal constructor on its own tape L. He discussed two interre- 
lated points in this connection. 

First, there is an apparent difficulty in using L for self -reproduction. 
A self -reproducing automaton must contain a complete description 
of itself. This might seem a priori impossible, on the ground that the 
constructing automaton must contain a complete plan of the con- 
structed automaton, and in addition the ability to interpret and exe- 
cute this plan (Sec. 1.6.1.1; cf. pp. 79-80 of Part I). The difficulty 
is circumvented by designing the universal constructor so that it uses 



290 



THEORY OF SELF-REPRODUCING AUTOMATA 



the information on L twice; once to construct a secondary, and once 
to make a copy of L which is attached to the secondary (Sec. 1.6.1.2; 
cf. pp. 84-87 of Part I and Sec. 5.3.2 below). In this way, a self- 
reproducing automaton stores a complete description of itself in a 
proper part of itself, namely, on the tape L (cf. Fig. 55). Likewise, 
an automaton which is both a universal constructor and a universal 
computer can store a complete description of itself in a proper part 
of itself (Fig. 56) n 

The second point about using L in self -reproduction concerns some 
alternatives. The universal constructor M c constructs a secondary 
automaton G whose description £)((?) is stored on L. Might a univer- 
sal constructor be designed which could directly copy automaton G 
itself (Sec. 1.6.2.3)? Alternatively, might an automaton be designed 
which could explore an automaton G and construct its description 
£)((?) (Sec. 1.6.3.1)? Von Neumann argued that these alternatives 
would be difficult, if not impossible, to carry out. In exploring one 
part of an active automaton (7, the exploring automaton might modify 
an as yet unexplored part of G. More generally, the active G might 
actively interfere with the exploratory activities of the exploring 
automaton. This difficulty would be particularly acute in the case of 
self -reproduction. If a universal constructor were to work directly 
from the secondary G, when the universal constructor attempted 
to reproduce itself it would be trying to explore itself. Von Neumann 
thought such an attempt would probably lead to paradoxes of the 
Richard type (Sec. 1.6.3.2). None of these difficulties arise when the 
universal constructor works with the quiescent description £>((?) 
(Sec. 1.4.2.1). 

There is a parallel problem with respect to the construction of the 
secondary automaton. If part of the secondary automaton were 
active during construction, it could interfere with the construction 
process. Von Neumann solved this problem by stipulating that the 
initial state of the secondary automaton is to be composed entirely 
of quiescent states (Sec. 1.7.2.1). In terms of the 29-state transition 
function developed in Chapter 2, this means that the X*/ are limited 
to the 10 values U, T Ma0 (u = 0, 1; a = 0, 1, 2, 3), and C 0 o . After 
the secondary automaton has been completed, it may be rendered 

11 We have already noted the parallelism between Turing machines and 
constructing machines. This parallelsm extends to the present point, for a 
Turing machine can contain a description of itself. See C. Y. Lee, "A Turing 
Machine Which Prints its Own Code Script" and James Thatcher's "The 
Construction of a Self-Describing Machine." 



AUTOMATA SELF-REPRODUCTION 



2\)\ 



active by a starting stimulus injected into its periphery. We have 
already called such automata "initially quiescent automata" (Sec. 
5.1.2). 

Thus the class of automata constructible by the universal con- 
structing automaton is a proper subclass of the automata which can be 
specified as part of an initial cell assignment (i.e., at time zero) of 
von Neumann's cellular structure (Sec. 1.3.3.5). Actually, it is pos- 
sible to design in the 29 -state cellular structure constructing automata 
which could construct every initially quiescent automaton and many 
active automata as well. There is no need to do this, however, since 
both the universal constructor and a universal Turing machine can 
be designed as initially quiescent automata (Sees. 5.3.2, 5.1.3). More- 
over, not all automata which can be specified as part of an initial cell 
assignment of von Neumann's cellular structure can be constructed 
from within the cellular structure. For example, the configuration of 
Figure 13b (time zero) when surrounded by a wide band of unexcita- 
ble cells U cannot be so constructed. A constructing arm can 
construct the quiescent states Tioo and T020 , but after it activates 
them it cannot withdraw from the surrounding area before Tioi and 
T021 kill each other. A simple example of a non-constructible autom- 
aton is the 3X3 configuration consisting of the sensitized state So 
surrounded by cells in state C 0 o - 12 

This concludes our summary of Chapter 1. To make specific his 
questions about automata construction, von Neumann had to choose 
a particular cellular structure. Chapter 2 is devoted to this task. 
In Section 2.1.2 he selected a discrete temporal reference frame and 
decided that the state of a cell at time t + 1 will depend only on its 
own state and the states of its four immediate neighbors at time t. 
In the remainder of the chapter he developed a set of 29 states and 
their transition rule. We refer to the result as "von Neumann's 
29-state cellular structure." It is summarized in Section 2.8 and 
Figures 9 and 10. 

12 Thatcher, "Universality in the von Neumann Cellular Model," Sec. 2.3. 

Moore, "Machine Models of Self-Reproduction," called a configuration 
which can exist only at time zero a "Garden-of-Eden" configuration. Every 
Garden-of-Eden configuration is non-constructible, though not conversely. 
Moore established a necessary condition for Garden-of-Eden configurations 
to exist in a cellular structure in which information requires at least 1 unit of 
time to pass from a cell to its neighbors. Myhill, "The Converse of Moore's 
Garden of Eden Theorem," showed that this condition is also sufficient. This 
condition is essentially that the cellular structure be non-backwards deter- 
ministic in the sense of Burks and Wang, "The Logic of Automata," Sec. 3.3. 



292 



THEORY OF SELF-REPRODUCING AUTOMATA 



We now reformulate questions (A)-(D) so they apply to von Neu- 
mann's 29-state cellular structure, at the same time modifying them 
somewhat. 

(A) Logical universality: Can an initially quiescent automaton which 
performs the computations of a universal Turing machine be 
embedded in von Neumann's 29-state cellular structure? 

(B) Constructibility: Can an automaton be constructed by another 
automaton within von Neumann's 29-state cellular structure? 

(C) Construction-universality: Can there be embedded in von Neu- 
mann's 29-state cellular structure a universal constructor M c 
with this property: for each initially quiescent automaton M, 
there is a coded description £>(M) of M such that, when £>(M) 
is placed on a tape L attached to M c , M c will construct M? 

(D) Self -reproduction: 

(Dl) Can a self -reproducing automaton be embedded in von 

Neumann's 29-state cellular structure? 
(D2) Can there be embedded in von Neumann's 29-state 

cellular structure an automaton which can perform the 

computations of a universal Turing machine and can 

also reproduce itself? 
All these questions are answered affirmatively in the present work. 
The answer to the constructibility question (B ) is given at the end of 
Section 2.8.3 and in Figure 14. For each initially quiescent automaton 
(2,, there are two binary (stimuli, no stimuli) sequences which, when 
fed into inputs i and j of Figure 14, will construct the automaton 
These two sequences can be produced by two linear arrays of Too* 
cells, with the e's properly chosen. Hence, these two linear arrays, 
together with the cells in columns A and B of Figure 14, will con- 
struct &. This shows that for each initially quiescent automaton Q 
there is an active automaton which will construct it. 

Affirmative answers to the remaining questions are based on the 
constructions of Chapters 3 and 4. 

In Chapter 3 von Neumann designed the basic organs to be used. 
Let i 1 • • • i n be an arbitrary finite binary sequence, with "1" repre- 
senting a stimulus and "0" representing the absence of a stimulus. 
i l • • • i n represents an indefinite repetition of the sequence i 1 • • « i n . 
Von Neumann developed design algorith ms for a rbitrary pulsers 
P(^i ... z , n) j arbitrary periodic pulsers PP^' 1 • • • i n ) y and arbitrary 
decoders D^' 1 % n ). He designed two specific organs: the triple- 
return counter <f> and the 1 vs. 10101 discriminator He concluded 
Chapter 3 with an algorithm which will design an arbitrary coded 
channel or wire crossing device. 



AUTOMATA SELF-REPRODUCTION 



Chapter 4 is devoted to the design of the memory control MC. 
MC reads and writes on the tape L under the direction of the con- 
structing uni + CU. A block diagram of MC is given in Figure 37; 
the operation of MC is outlined in Section 4.3.1. The basic operation 
of reading cell x n of L, writing on x n , and changing the connection 
to cell x n +\ (lengthening) or cell av-i (shortening) is carried out in 
two stages. First, when a pulse comes from the output of CU labeled 
"oi" to the input of MC having this same label, MC reads the cell 
x n by means of the connecting loop Ci . If x n stores "zero," MC sends a 
pulse to input i x of CU; while if x n stores "one," MC sends a pulse to 
input i 2 of CU. Second, CU sends a pulse to input o 2 or o 3 of MC 
according as "zero" or "one" is to be written in cell x n ; CU also sends 
a pulse to input o 4 or o 6 of MC according as the loops Ci and C 2 
are to be lengthened or shortened. MC executes these operations, and 
when they are finished it sends a pulse to input is of CU signifying 
completion. 

The parts of the memory control MC are: the read-write-erase unit 
RWE; the read-write-erase control RWEC; the delay area W; the 
transfer area Y; and the coded channel, consisting of the main chan- 
nel together with X, Z, CCi , CC 2 , and CC 3 . 

Von Neumann did not quite complete the design of MC, and what 
he did finish contained many errors. We have corrected all but one 
error as we have gone along (Ch. 4 and Sec. 5.1.1). This last error is 
corrected and the design of MC is completed in Section 5.1.2. A 
much improved design of MC is suggested in Section 5.2.2. 

The memory control MC, the indefinite linear array L, the con- 
necting loop Ci , and the timing loop C 2 together constitute a tape 
unit with unlimited memory capacity. Moreover, MC is an initially 
quiescent automaton which is started by a stimulus impinging on its 
periphery (input Oi). Hence an initially quiescent automaton which 
performs the functions of a tape unit with unlimited memory capacity 
can be embedded in von Neumann's 29-state cellular structure. 

A Turing machine consists of such a tape unit together with a 
finite automaton which can interact with this tape unit. In Section 
5.1.3 we showed how to simulate an arbitrary finite automaton by an 
initially quiescent cellular automaton. Combining these results and 
applying them to the specific case of a universal Turing machine, we 
obtained a positive answer to von Neumann's question (A): An 
initially quiescent automaton which performs the computation of a 
universal Turing machine can be embedded in von Neumann's 
29-state cellular structure. 

Von Neumann's universal constructor M c consists of the construct- 



294 



THEORY OF SELF-REPRODUCING AUTOMATA 



ing unit CU combined with the tape unit (MC + L). See Figure 50. 
The constructing arm is designed in Section 5.2.1, and the design of 
the constructing unit CU is sketched in Section 5.2.3. Hence there can 
be embedded in von Neumann's 29-state cellular structure a universal 
constructor M c with this property: for each initially quiescent autom- 
aton M, there is a coded description 3D(M) of M such that, when 
is placed on a tape L attached to M c , M c will construct M. 

This answers von Neumann's question (C) and brings us to ques- 
tion (D), concerning self -reproduction. 

A comparison of the universal computer M u and the universal 
constructor M c shows that in von Neumann's cellular structure, com- 
putation and construction are similar activities. Both M c and M u 
are finite data processors which can interact with an indefinitely ex- 
tendible tape. Suppose the universal computer M u is designed to 
write its output answer on a fresh tape L. Then both M u and M c 
produce initially quiescent automata. The universal constructor M c 
produces rectangular initially quiescent automata based on the 10 
states U, Tuao (u = 0, 1; a = 0, 1, 2, 3), and Coo • The universal com- 
puter M u produces a linear initially quiescent automaton based on 
the two states U and T 0 3o ( I )• 

5.3,2 Self -reproducing automata. Our task is now to convert the uni- 
versal constructor M c of Section 5.2.3 and Figure 50 into a self-repro- 
ducing automaton. 

Note first that the universal constructor M c is in fact an initially 
quiescent automaton. Consequently, a description £>(M C ) can be 
placed on the tape L attached to M c . When this is done and M c is 
started, the complex M c + £)(M C ) will produce a copy of M c as 
the secondary constructed automaton. This is not yet self -reproduc- 
tion, however, for the constructed automaton M c is smaller than the 
constructing automaton M c + 3D(M C ). 

In this case the constructing automaton is larger and, in a sense, 
more complex than the constructed automaton because the construct- 
ing automaton contains a complete plan £>(M C ) of the constructed 
automaton and, in addition, a unit M c which interprets and executes 
this plan (Sec. 1.6.1.1, pp. 79-80 of Part I). To obtain a primary 
automaton which will construct a secondary as large as itself, we 
make some modifications in the universal constructor M c (compare 
Sec. 1.6.1.2 and pp. 84-86 of Part I). 

Let the secondary automaton to be constructed consist of an 
initially quiescent automaton M together with a tape L which stores 
some tape content 3(M) initially. See Figure 54. Place the following 
information on the tape L of the universal constructor: a period, the 



AUTOMATA SELF-REPRODUCTION 



295 



description 3>(Af), a second period, the tape content 3(M), and a 
third period. As a special case we allow 3(M) and the third period 
to be omitted. It is easy for the universal constructor to detect that 
3(M) is omitted since 3(Af ) is written in a five-bit code that does not 
include the all-zero character (Sec. 5.2.3). 

Now change M c into a modified universal constructor M* which 
executes the following three steps. First, M* uses £>(M) to construct 
M, as before (Sees. 5.2.1 and 5.2.3). Second, M* produces a tape L 
attached to M which stores a period, 3(M), and a second period. If 
M c * finds that there is nothing stored on its tape beyond the second 
period (i.e., 3(Af) is missing), M* will then copy a period, 3)(M), 
and a second period onto the tape attached to M. Third, M c * will 
give a starting stimulus to M. 

The second of these steps is a simple tape-copying operation. It can 
be carried out by the constructing unit CU and the constructing arm 
in a way analogous to the way these organs construct M. The coding 
is different in the two cases, of course. A cell (i, j) of M is described 
in 3D (M) by a five-bit character which is stored in five successive cells 
of the tape of M c *. In contrast, each cell of M's tape is the same as the 
corresponding cell of M c *'s tape. 

For each choice of location (x x , y x ) of a copy of M* there is a 
description £>(M C *). Place this description £>(M C *) on the tape of 
M c * itself. The complex M c * + 3D (M c * ) will construct M c * + £> ( Af c * ) . 
This is self -reproduction. See Figure 55. 

Hence, a self-reproducing automaton can be embedded in von Neu- 
mann's 29-state cellular structure. This answers question (Dl). 

Iterated construction and self -reproduction may be achieved by 
further modification of the universal constructor (Sec. 1.7). 

The initially quiescent automaton M u is a universal Turing machine 
(Sec. 5.1.3). Place the description S>(M U + M c *) on the tape L of 
M c * . The primary automaton M e * + D(M tt + M e *) will then 
construct as secondary the automaton {M u + M c *) + £>(Mu + 
M c *). In this case the constructed automaton is larger, and in a 
sense more complicated, than the constructing automaton. 

Next, attach M u to M* as in Figure 56. Place the description 
£>(M W + M c *) on the tape of M u + M c *. The automaton (M u + 
M*) + <£>{M U + M c *) will construct (M u + M c *) + 3D(M W + 
M c ), and hence is self -reproductive. After it is completed, the second- 
ly (M u + M c *) + 3D(M W + M c *) can carry out a computation. 
Alternatively, if M u is supplied with its own tape, each (M u + 
M c *) + £>(M U + M c *) can compute and construct simultaneously. 
Hence, there can be embedded in von Neumann's 29-state cellular 



296 



THEORY OF SELF-REPRODUCING AUTOMATA 



structure an automaton which can perform the computations of a univer- 
sal Turing machine and can also reproduce itself. This answers question 
(D2). 

All of von Neumann's questions about automata construction and 
computation (Sees. 1.1.2.1 and 5.3.1) have now been answered 
affirmatively. His 29-state cellular structure is computation-universal, 
construction-universal, and self-reproductive. In this cellular struc- 
ture, self -reproduction is a special case of construction, and con- 
struction and computation are similar activities.] 



BIBLIOGRAPHY 



Birkhoff, Garrett. Hydrodynamics, A Study in Logic, Fact, and Similitude. 
Princeton: Princeton University Press, 1950. 

and John von Neumann. See John von Neumann. 

Boltzmann, Ludwig. Vorlesungen iiber Gas Theorie. Two volumes. Leipzig: 
Johann Barth, 189G and 1898. 

. Wissenschaftliche Abhandlungen. Edited by Fritz Hasenohrl. Three 

volumes. Leipzig: Johann Barth, 1909. 

Booth, Andrew D. "The Future of Automatic Digital Computers." Com- 
munications of the Association for Computing Machines 3 (June, 1960) 
339-341, 360. 

Brainerd, J. G., and T. K. Sharpless. "The ENIAC." Electrical Engineering 

67 (Feb., 1948) 163-172. 
Brillouin, Leon. Science and Information Theory. New York: Academic Press, 

1956. 

Bulletin of the American Mathematical Society. "John von Neumann, 1903- 
1957." Vol. 64, No. 3, Part 2, May, 1958, 129 pages. This is a memorial 
issue. Besides the articles by Ulam and Shannon listed separately in this 
bibliography, it contains articles on von Neumann's work in lattice 
theory, theory of operators, measure and ergodic theory, quantum 
theory, and theory of games and mathematical economics. 

Burks, Arthur W. "Computation, Behavior, and Structure in Fixed and 
Growing Automata." Behavioral Science 6 (1961) 5-22. Original version 
and discussion in Self -Organizing Systems (edited by Marshall Yovits 
and Scott Cameron). New York: Pergamon Press, 1960, pp. 282-311, 
312-314. 

. "Electronic Computing Circuits of the ENIAC." Proceedings of the 

Institute of Radio Engineers 35 (August, 1947) 756-767. 

. "Programming and the Theory of Automata." Pp. 100-117 of Computer 

Programming and Formal Systems, edited by P. Braffort and D. Hirsch- 
berg. Amsterdam: North-Holland Publishing Company, 1963. 

. "Super Electronic Computing Machine." Electronic Industries 5 (July, 

1946) 62-67, 96. 

. "Toward a Theory of Automata Based on More Realistic Primitive 

Elements." Pp. 379-385 of Information Processing 1962, Proceedings of 
IFIP Congress 62, edited by C. M. Popplewell. Amsterdam: North-Hol- 
land Publishing Company, 1963. 

and John von Neumann. See John von Neumann. 

and Hao Wang. "The Logic of Automata." Journal of the Association 

for Computing Machinery 4 (April, 1957) 193-218 and 4 (July, 1957) 
279-297. (Reprinted in Wang, Hao, A Survey of Mathematical Logic, pp. 
175-223. Peking: Science Press, 1962. Distributed by North-Holland 
Publishing Company, Amsterdam.) 

and Jesse B. Wright. "Theory of Logical Nets." Proceedings of the Insti- 
tute of Radio Engineers 41 (October, 1953) 1357-1365. Reprinted in Se- 



297 



298 



THEORY OF SELF-REPRODUCING AUTOMATA 



quential Machines — Selected Papers, pp. 193-212. Edited by E. F. Moore. 

Reading, Mass.: Addison Wesley, 1964. 
Church, Alonzo. "Applications of Recursive Arithmetic to the Problem of 

Circuit Synthesis/ ' Summaries of Talks Presented at the Summer Institute 

for Symbolic Logic, Cornell University, 1957. Princeton: Institute for 

Defense Analysis, 1960. 
Codd, Edgar Frank. "Propagation, Computation, and Construction in Two- 

Dimensional Cellular Spaces." 152 pp. Ph.D. Dissertation, University 

of Michigan, 1965. 

i/ Eccles, J. C. The Neurophysiological Basis of Mind. Oxford: Oxford Univer- 
sity Press, 1953. 

Estrin, Gerald. "The Electronic Computer at the Institute for Advanced 
Study." Mathematical Tables and Other Aids to Computation 7 (April, 
1953) 108-114 and frontispiece. 

Godel, Kurt, "tiber formal unentscheidbare Satze der Principia Mathematica 
und verwandter Systeme I." Monatshefte fur Mathematik und Physik 38 
(1931) 173-198. (English translation by Elliott Mendelson in The Un- 
decidable, edited by Martin Davis, pp. 4-38. Hewlett, New York: Raven 
Press, 1965.) 

. "On Undecidable Propositions of Formal Mathematical Systems." 

Mimeographed notes on lectures delivered at the Institute for Advanced 
Study, Princeton, New Jersey. February-May, 1934. 31 pages. (Re- 
printed as pp. 39-74 of The Undecidable, above.) 

. "tiber die Lange der Beweise." Ergebnisse eines mathematischen Kol- 

loquiums, Heft 7, pp. 23-24. (English translation by Martin Davis in 
The Undecidable, above, pp. 82-88.) 

Goldstine, H. H., and Adele Goldstine. "The Electronic Numerical Integrator 
and Computer (ENIAC)." Mathematical Tables and Other Aids to Com- 
putation 2 (July, 1946) 97-110 and frontispiece. 

and John von Neumann. See John von Neumann. 

Goto, Eiichi. "The Parametron, a Digital Computing Element which Utilizes 
Parametric Oscillation." Proceedings of the Institute of Radio Engineers 
47 (August, 1959) 1304-1316. 

Hamming, R. W. "Error Detecting and Error Correcting Codes." Bell Sys- 
tem Technical Journal 29 (1950) 147-160. 

Hartley, R. V. L. "Transmission of Information." Bell System Technical 
Journal 7 (1928) 535-563. 

Holland, J. H. "Concerning Efficient Adaptive Systems." Pp. 215-230 of 
Self-Organizing Systems — 1962. Edited by M. C. Yovits, G. T. Jacobi, 
and G. D. Goldstein. Washington, D. C: Spartan Books, 1962. 

. "Outline for a Logical Theory of Adaptive Systems." Journal of the 

7 Association for Computing Machinery 9 (July, 1962) 297-314. 

. "Iterative Circuit Computers." Proceedings of the 1960 Western Joint 

Computer Conference, pp. 259-265. Institute of Radio Engineers, 1960. 

. "Universal Embedding Spaces for Automata." To be published in 

Progress in Brain Research, a festschrift for Norbert Wiener; Amsterdam: 
Elsevier Publishing Company. 

. "A Universal Computer Capable of Executing an Arbitrary Number 

of Sub-Programs Simultaneously." Proceedings of the 1959 Eastern Joint 
Computer Conference, pp. 108-113. Institute of. Radio Engineers, 1959. 

Kemeny, John. "Man Viewed as a Machine." Scientific American 192 (April, 



BIBLIOGRAPHY 



299 



1955) 58-67. This is based on von Neumann's Vanuxem Lectures, de- 
livered at Princeton University in March, 1953. 
Keynes, John Maynard. A Treatise on Probability. London: Macmillan and 
Co., 1921. 

Kleene, S. C. Introduction to Metamathematics. New York: Van Nostrand, 
1952. 

. " Representation of Events in Nerve Nets and Finite Automata." Pp. 

3-41 of Automata Studies, edited by C. E. Shannon and J. McCarthy. 
Princeton: Princeton University Press, 1956. (This appeared originally 
as Rand Corporation Memorandum RM-704, 101 pp., dated December 
15, 1951.) 

Laplace, Marquis Pierre Simon de. A Philosophical Essay on Probabilities. 
Translated by F. W. Truscott and F. L. Emory. New York: Dover Pub- 
lications, 1951. First French edition 1814. 

Lee, C. Y. "A Turing Machine Which Prints Its Own Code Script." Pp. 155- 
164 of Proceedings of the Symposium on Mathematical Theory of Automata, 
New York, April, 1962. Brooklyn, New York: Polytechnic Press, 1963. 

Metropolis, N., and Ulam, S. "The Monte Carlo Method." Journal of the 
American Statistical Association 44 (1949) 335-41. 

McCulloch, W. S., and W. Pitts. "A Logical Calculus of the Ideas Immanent 
in Nervous Activity." Bulletin of Mathematical Biophysics 5 (1943); 
115-133. " 

McNaughton, Robert. "On Nets Made up of Badly Timed Elements, Part I; 
Slow but Perfectly Timed Elements." Mimeographed, 30 pages. Phila- 
delphia: Moore School of Electrical Engineering, University of Penn- 
sylvania, 1961. 

Moore, E. F. "Machine Models of Self-Reproduction." Pp. 17-33 of Mathe- 
matical Problems in the Biological Sciences. Proceedings of Symposia in 
Applied Mathematics, Vol. XIV. Providence, Rhode Island: American 
Mathematical Society, 1962. 

Morgenstern, Oskar, and John von Neumann. See John von Neumann. 

Myhill, John. "The Converse of Moore's Garden-of-Eden Theorem." Pro- 
ceedings of the American Mathematical Society 14 (August, 1963) 685-686. 

Nyquist, H. "Certain Factors Affecting Telegraph Speed." Bell System Tech- 
nical Journal 3 (1924) 324-346. 

Rajchman, J. A. "The Selectron — A Tube for Selective Electrostatic Stor- 
age." Mathematical Tables and Other Aids to Computation 2 (October, 
1947) 359-361 and frontispiece. 

Richard, Jules. "Les principes des math&natiques et le probleme des en- 
sembles." Revue generate des sciences pures et appliquees 16 (1905) 541- 
543. 

Russell, Bertrand. "Mathematical Logic as Based on the Theory of Types." 

American Journal of Mathematics 30 (1908) 222-262. 
Shannon, C. E. "A Mathematical Theory of Communication." Bell System 

Technical Journal 27 (1948) 379-423, 623-656. (Reprinted in C. E. 

Shannon and W. Weaver, The Mathematical Theory of Communication, 

pp. 3-91. Urbana: University of Illinois Press, 1949.) 
. "Von Neumann's Contributions to Automata Theory." Bulletin of the 

American Mathematical Society, Vol. 64, No. 3, Part 2, May, 1958, pp. 

123-129. 

Szilard, L. "Uber die Entropieverminderung in einem thermodynamischen 



300 



THEORY OF SELF-REPRODUCING AUTOMATA 



System bei Eingriffen intelligenter Wesen." Zeitschrift fur Physik 53 
(1929) 840-856. (English translation by Anatol Rapoport and Mech- 
thilde Knoller, "On the Decrease of Entropy in a Thermodynamic Sys- 
tem by the Intervention of Intelligent Beings." Behavioral Science 9 
(October, 1964) 301-310.) 

Tarski, Alfred. "The Concept of Truth in Formalized Languages." Pp. 152- 
278 of Logic, Semantics, M etamathematics (translated by J. H. Woodger) ; 
Oxford: Oxford University Press, 1956. (Original Polish version 1933, 
German translation 1936.) 

Thatcher, James. "The Construction of a Self-Describing Turing Machine." 
Pp. 165-171 of Proceedings of the Symposium on Mathematical Theory of 
Automata, New York, April, 1962. Brooklyn, New York: Polytechnic 
Press, 1963. 

. "Universality in the von Neumann Cellular Model." 100 pp. Technical 

Report 03105-30-T, ORA, University of Michigan, 1965. To be pub- 
lished in Essays on Cellular Automata, edited by A. W. Burks. 

Turing, A. M. "The Chemical Basis of Morphogenesis." Philosophical Trans- 
actions of the Royal Society of London. Series B, Biological Sciences, Vol. 
237, August, 1952, pp. 37-72. 

. "Computability and X-Definability." The Journal of Symbolic Logic 2 

(Dec, 1937) 153-163. 

. "On Computable Numbers, With an Application to the Entscheidungs- 

problem." Proceedings of the London Mathematical Society, Series 2, 42 
(1936-37) 230-265. "A Correction," Ibid., 43 (1937) 544-546. (Reprinted 
in The Undecidable, edited by Martin Davis, pp. 115-154, Hewlett, New 
York: Raven Press, 1965.) 

Ulam, S. M. A Collection of Mathematical Problems. New York, Interscience 
Publishers, Inc., 1960. 

. "Electronic Computers and Scientific Research." Pp. 95-108 of The 

Age of Electronics. Edited by C. F. J. Overhage. New York: McGraw- 
Hill, 1962. 

. "John von Neumann, 1903-1957." Bulletin of the American Mathemati- 
cal Society, Vol. 64, No. 3, Part 2, May, 1958, pp. 1-49. This contains 
valuable biographical information, a summary account of von Neumann's 
accomplishments, and a bibliography. 

. "On Some Mathematical Problems Connected with Patterns of Growth 

of Figures." Pp. 215-224 of Mathematical Problems in the Biological 
Sciences. Proceedings of Symposia in Applied Mathematics, Vol. 14, 
Providence, Rhode Island: American Mathematical Society, 1962. 

. "Random Processes and Transformations." Proceedings of the Interna- 
tional Congress of Mathematicians, 1950, Vol. II, pp. 264-275. Providence, 
Rhode Island: American Mathematical Society, 1952. 

John von Neumann — Collected Works. Six volumes. Edited by A. H. Taub. 
New York: Macmillan, 1961-63. Vol. V is entitled Design of Computers, 
Theory of Automata and Numerical Analysis. References to these volumes 
will be made as follows: ''Collected Works 5.288-328" refers to pp. 288^- 
328 of Vol. V. 

von Neumann, John. The Computer and the Brain. With a preface by Klara 
von Neumann, pp. v-x. New Haven: Yale University Press, 1958. This 
manuscript was prepared during 1955 and 1956 for the Silliman Lectures 



BIBLIOGRAPHY 



of Yale University, but von Neumann was never able to deliver the lec- 
tures and never completed the manuscript. 

. First Draft of a Report on the EDVAC. Contract No. W-670-ORD-4926, 

between the United States Army Ordnance Department and the Univer- 
sity of Pennsylvania. Pp. x + 101 (double spaced) with 22 figures and a 
table explaining the order code. Mimeographed. Moore School of Elec- 
trical Engineering, University of Pennsylvania, June 30, 1945. An earlier 
typed version had more material. Both versions are incomplete, lacking 
many cross references and terminating with the exposition of the order 
code. But the logical design is essentially complete except for the control 
and the input and output organs. 

. "The General and Logical Theory of Automata." Pp. 1-41 in Cerebral 

Mechanisms in Behavior — The Hixon Symposium, edited by L. A. 
Jeffress. New York: John Wiley, 1951. Collected Works 5.288-328. The 
paper was read on September 20, 1948. Von Neumann's comments on 
other papers are on pp. 58-63, 96, 109-111, 132, and 232 of Cerebral 
Mechanisms in Behavior. 

. M athematische Grundlagen der Quanlenniechanik. Berlin: Springer, 1932. 

(English translation by Robert Beyer, Princeton University Press, 1955.) 

. "Non-Linear Capacitance or Inductance Switching, Amplifying and 

Memory Devices. " Collected Works 5.379-419. This is the basic paper 
for United States Patent 2,815,488, filed April 28, 1954, issued December 
3, 1957. 

. "Probabilistic Logics and the Synthesis of Reliable Organisms from 

Unreliable Components." Pp. 43-98 of Automata Studies, edited by C. E. 
Shannon and J. McCarthy. Princeton: Princeton University Press, 1956. 
Collected Works 5.329-378. This paper is based on notes taken on five 
lectures given by von Neumann at the California Institute of Technology 
in January, 1952, which were issued in a mimeographed edition. 

. "Quantum Logics (Strict- and Probability- Logics)." Written in about 

1937. Collected Works 4.195-197. 

. Remarks in the discussion of Ralph Gerard's "Some of the Problems 

Concerning Digital Notions in the Central Nervous System." In Cyber- 
netics, edited by H. von Foerster, pp. 19-31. New York, Josiah Macy, 
Jr. Foundation, 1951. This is the transactions of a conference held in 
March, 1950. 

. Review of Norbert Wiener's Cybernetics, or Control and Communication 

in the Animal and the Machine. Physics Today 2 (1949) 33-34. 

. "Zur Theorie der Gesellschaftspiele," Mathematische Annalen 100 (1928) 

295-320. Collected Works 6.1-26. 

, and Garrett Birkhoff. "The Logic of Quantum Mechanics." Annals of 

Mathematics 37 (1936) 823-843. Collected Works 4.105-125. 

, Arthur Burks, and H. H. Goldstine. Preliminary Discussion of the 

Logical Design of an Electronic Computing Instrument. Part I of a report 
on the mathematical and logical aspects of an electronic computing in- 
strument prepared on Contract W-36-034-ORD-7481 for the Ordnance 
Department of the United States Army. 42 pp. Princeton: Institute for 
Advanced Study, 1946. Second edition, 1947, pp. vi + 42. Collected 
Works, 5.34-79. 

, and H. H. Goldstine. Planning and Coding of Problems for an Electronic 



302 



THEORY OF SELF-REPRODUCING AUTOMATA 



Computing Instrument. Part II of a report on the mathematical and 
logical aspects of an electronic computing instrument prepared on Con- 
tract W-36-034-ORD-7481 for the Ordnance Department of the United 
States Army. Princeton: Institute for Advanced Study. Vol. 1, pp. iv -f 
69, 1947. Vol. 2, pp. iv + 68, 1948. Vol. 3, pp. hi + 23, 1948. Collected 
Works 5.80-235. 

and Oskar Morgenstern. Theory of Games and Economic Behavior. Prince- 
ton: Princeton University Press, 1944. 2nd ed., Princeton, 1947; 3rd ed., 
Princeton, 1953. 

Wiener, Norbert. Cybernetics, or Control and Communication in the Animal 

and the Machine. New York: John Wiley & Sons, Inc., 1948. 2nd ed., with 

additions, New York, 1961. 
Wigington, R. L. "A New Concept in Computing." Proceedings of the Institute 

of Radio Engineers 47 (April, 1959) 516-523. 
Wilkes, M. V. Automatic Digital Computers. London: Methuen & Co., Ltd., 

1956. 

. "Progress in High-Speed Calculating Machine Design." Nature 164 

(August 27, 1949) 341-343. 
Williams, F. C. "A Cathode-Ray Tube Digit Store." Proceedings of the Royal 

Society of London, Series A, Vol. 195 (22 December 1948) 279-284. 



FIGURES 



FIGURES 



305 




Fig. 1. Neural network in which dominance is not transitive 



306 THEORY OF SELF-REPRODUCING AUTOMATA 




0 I I 0 I 0 0 I I 0 

Fig. 2. A binary tape constructed from rigid elements 



+ neuron: ^ _^ © — *~c 

• neuron*. J ^ Q — ^. c 
— neuron: a — - © — 

Fig. 3. The basic neurons 



<* a b ~® — -« 



FIGURES 



307 



-o o o- 

- o X o - 



(a) Nearest (O) and next nearest 
neighbors of X 






















• 


o 


• 










o 


X 


o 










• 


o 


• 





















(b) Unit vectors 



(c) Nearest ( O ) and next near- 
est (•) neighbors of X 



Fig. 4. The quadratic lattice 



308 



THEORY OF SELF-REPRODUCING AUTOMATA 







(a) 




T oo 


T oo 


T oo 


T oo 


T oo 








(b) 




T .o 




































































T .o 


































T.O 


































T.O 












































(a) 




















(b') 




t 


































t 


































t 


































t 


































t 











(c) 




T 20 


T 20 


T 20 


T 20 


T 20 








(d) 




T30 
































T30 
































T 30 
































T30 
































T 30 








































(c) 





























































































































































Fig. 5. Ordinary transmission states 



FIGURES 



309 







(e) 




T oo 


T oo 


T oo 


T 50 








vf ) 




T 30 


T 20 


T 20 


T 20 


















T 












T 
























T 

T 30 












T 

Too 


T oo 


T oo 


T oo 












































(e 1 ) 










J 








(f') 




1 
























I 












1 
























I 



























































(g) 











































t 
















t 
















t 

























Fig. 5. Part two 



310 



THEORY OF SELF-REPRODUCING AUTOMATA 























c 














t 














t 














t 























(a) Achievement of a • neuron by 
using confluent states 





























c 


















* 

































































( b) Achievement of a split line 
by using confluent states 



Fig. 6. Confluent states 



FIGURES 









































































i 


2 


3 






















4 


5 


6 






















7 


8 


9 
























t 


























t 












( 


r 




- 


- 


- 


- 


t 




























































Fig. 7. Organization of an area 





















1 




























1 




























B 




























t 
















A 






P 






t 












































* 


































































f- 












































t 








t 













































Fig. 8. The need for both even and odd delays 



S 



o3 



0 



S3 

a 

o 
O 



Symbol 



U 



C €€ , 



e = 0 (quiescent) 
e = 1 (excited) 



e' = 0 (next quiescent) 
€ r = 1 (next excited) 



to 



Toae 



10 



Tiae 



Tl 



11 



C7ass: 
w = 0 (ordinary) 
u = 1 (special) 



Excitation: 

e = 0 (quiescent) 
e=l (excited) 



Output direction: 
a = 0 (right) 

a = 1 (up) 
a = 2 (left) 
a = 3 (down) 



Summary of transition rule 



Direct process changes U into sen- 
sitized states and then into 
T ua o or Coo- 
Reverse process kills T ttae or C <e ' 
into U. 



Receives conjunctively from T 0a « 
directed toward it; emits with 
double delay to all T uat not 
directed toward it. 

Killed to U by Ti a i directed to- 
ward it; killing dominates 
reception. 



Receives disjunctively from T 0a « 
directed toward it and from 
C €e 'j emits in output direction 
with single delay 

(a) to Tea* not directed to- 
ward it and to C t€ ' 

(b) to U or sensitized states 
by direct process 

(c) to kill Ti a e by reverse 
process. 

Killed to U by Ti a i directed to- 
ward it; killing dominates re- 
ception. 



Receives disjunctively from T iat 
directed toward it and from 
C €€ 'j emits in output direction 
with single delay 

(a) to Ti ae not directed to- 
ward it 

(b) to U or sensitized states 
by direct process 

(c) to kill T 0 «« or C ee ' by 
by reverse process. 

Killed to U by T 0a i directed to- 
ward it; killing dominates re- 
ception. 



Si 



These are intermediary states in 
the direct process. T ua i directed 
toward U converts it to S$. 
Thereafter, Sz is followed by 

(a) Szi if some T tta i is di- 
rected toward the cell 

(b) Sso otherwise, 

until the direct process termi- 
nates in a Tuao or C C o. See 
Figure 10. 



Fig. 9. The 29 states and 



their transition rule 



FIGURES 



313 




THEORY OF SELF-REPRODUCING AUTOMATA 



u 


u 


C €€' 


T 00€ 


u 





\f 



mm 



Mil 



b b 
(a) Realization of /"(/+3) = [a{f) + £(/+!)] 



u 


U 


T 00€ 




U 


T 0I€ 



to 



d d 
(b) Realization of ?(/+3)= [c{f)-d{f)] 



u 




V 


T 00€ 


C €€' 


T o.e 


U 


T ,3« 


u 



w 

Wa 


to 


to 




c 


to 


W/, 


11 


w 



J 

(c) Conversion of ordinary stimuli into special stimuli 
by confluent states, and wire - branching 



Fig. 11. Illustrations of transmission and confluent states 



FIGURES 



315 





A 


B 


C 


D 




A 


B 


C 


D 


/ 


T ooo 




r ooo 


u 


/ 


T ooo 


c oo 


T 00l 


u 


2 


T oio 


T 020 


Time 0 


2 


T oio 


T 

021 


Time 1 




A 


B 


C 






A 


B 


C 


Z? 


1 


T ooo 


c oo 


T ooo 




1 


T 00l 


c oo 


T ooo 


s o 


2 




T 020 


Time 2 


2 


T oio 


T 020 


Time 3 




A 


B 


C 


D 




A 




C 


D 


1 


T ooo 


c o. 


T ooo 


s oo 


1 


000 


c .o 


T ooo 


s ooo 


2 


T oio 


T 020 


Time 4 


2 


T oio 


T 020 


r//776» 5 




A 


B 


C 


Z7 




A 


B 


C 


Z? 


1 


^00 


c 

00 


T 

001 


T ooo 


1 


T ooo 


C oo 


T 

000 


T ooi 


2 


T oio 


T 02l 


Time 6 


2 


T 

on 


T 020 


77>77e 7 



Fig. 12. Illustration of the direct process 



316 THEORY OF SELF-REPRODUCING AUTOMATA 

Time O Time / 



u 


u 


u 


u 


u 


T .0I 


C oo 


u 


u 


u 


u 


u 







? 


? 


7 


T I00 


U 


? 


? 


9 


P 


? 



(a) Special transmission state killing a confluent state. 



Time 0 



Time I 



u 


u 


u 


U 


u 


T IO« 


T 02l 


u 


u 


u 


U 


u 



(b) Special and ordinary transmission states killing each other. 



ABC 



ABC 



T ooo 


T ooo 


T ooo 




T .IO 




c 

Time 0 


A 


B 


c 


T OOI 


T ooi 


T ooo 




T lll 











b / 
2 



a 



b I 
2 



T OOI 


T ooo 


T ooo 








c 

Time 1 


A 


B 


C 


T ooo 


u 


T OOI 









b I 
2 



b I 
2 



Time 2 



c 

Time 3 



(c ) Killing dominates reception but not emission. 

Fig. 13. Examples of the reverse process 



FIGURES 



3K 



A 


B 


C 


D 








4 


5 


C 


Z? 




A 




10 






/ 




1 


U 


jo 






1 




u 


u 


u 


2 










JO 


10 


2 




tl 




u 


3 












c 


3 


A 


_0 


c 


(a) 
A 


4 




A 




c 


(b) 


4 






B 


C 


D 
















A 


£ 


10 






/ 
/ 






















u 




















»i 




c 


j 














A 




c 


(c) 


4? 











BCD 



A B C D 



i\ 


0 


10 






/ 


i\ 




10 






/ 








u 


u 


2 






u 


u 


u 


2 






fl 




c 


3 






tl 




c 


3 


A 




C 


(d) 


4 


A 


0. 


c 


(e) 


4 



Produces 
Fig. 14b 



Produces 
Fig. I4d 



Into / i o o oo i o oooiotoi i i r 0 0 0000000001 10 00 0 ? 



Into /: OOOOOOOOOOOOOOOOIIOIIIIOII 10 00 000010 

' V 



Y 

Produces 
Fig. 14c 



Produces 
Fig.l4e 



Fig. 14. Procedure for modifying a remote cell and returning the construct- 
ing path to its original unexcitable state 



318 



THEORY OF SELF-REPRODUCING AUTOMATA 







0 


0 


0 


0 


0 




£ 


C 


0 


C 


0 


c 



(a) P(lll) 



A 


B 


c 


D 


E 


F 






0 


0 


0 


0 


0 


\b 1 




to 


// 


to 




c 


2 




to 




to 


0 


to 


3 




to 


''A 


to 


to 




4 


1 


to 


0 


to 




to 


5 




to 


to 


0 


to 


0 


6 


0 


c 


0 


c 


0 


c 


7 



(b) P(IOOIOOOI) 



Fig. 15. Two pulsers 



FIGURES 



319 





0 


0 


0 


0 


0 


0 




0 


0 






















u 


TO 




TO 




TO 






TO 


u-J 


to 




TO 




TO 






to 






































2 


TO 




TO 




TO 






to 


J 


TO 




TO 




TO 






to 




C 


0 


C 


0 


C 


0 




0 

-♦ 


C 






J 




2 




J 






k 



(a) 



u 
u-1 



2 
1 



N i 



0 j c 



old a 



77 



N 2 



N 3 



2 3 



(b) 

Fig. 16. Design of a pulser 



\b 



N. 



320 



THEORY OF SELF-REPRODUCING AUTOMATA 



U 

u-1 



2 
1 



N h 



(c) 



s k -l 



2 
1 



0 


to 


To 




(d) 


u 


to 


u 


to 






u 


to 


u 


to 



(f ) 



u+2 



(g) 



Fig. 16. Part two 



FIGURES 



321 



A B 



C D 



E F G H 



















PI 


























0 


10 


11+ 






0 


y 








































; 

/ 


c 


to 


V/////////////////////////////////////// 




11 


to 












b'\ 


0 


0 


c 


to 














> 


to 


10 


to 
















c 


p 


// 






p 


(IC 


QIC 


)00 


1) 






10 


















JO 










////// //// 
















0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


o lo 


0 


0 















(a) Periodic Pulser PP(IOOIOOOI) 

B C D E 



°-\ 


0 


0 


0 


0 


0 


0 


0 


0 


0 


o 


o 


0 


0 


0 


c 


1 








11 


2 














P( 


\\\ 


H) 








b'\ 


0 


0 


I 


\*3 


"A 


\°' 


























c 




4 



( b) Special Periodic Pulser PP {]) 
Fig. 17. Two periodic pulsers 



2 
3 
\b4 

5 
6 
7 
8 
9 
10 



322 



THEOKY OF SELF-REPRODUCING AUTOMATA 



0 


10 


to 


0 

4- 



b 2 



i.0) 



0 

-+ 


0 




0 

-♦ 


40 




to 






0 

4— 


0 


0 

4- 




/ 


2 


1-1 J? 



c 


0 




0 


iO 








to 


0 




0 

4- 


0 

4- 


4- 




/ 


2 


J-l J? 



(b) 



c 


0 

-* 




0 


40 




to 


0 




0 


c 




/ 2 J-l J 



(c) 



(d) 



41 



c 


0 

-* 




0 

-4> 


40 




to 


0 

4- 




0 

4- 


X 





(e) 



/-// 



b'b 2 



b b 

1 2 



\ 


121 


0 

-> 


c 




u 


to 


40 


u\2 Pf/'-'/V 










u 


to 


40 




u 


X 


0 

4- 


^ — 1 2 k * 

♦ 





I 

J 



(f) 



Fig. 18. Design of a periodic pulser: initial construction 



FIGURES 




b 
\ 



~2 P (FT) (order 5p) 

|L J 0 p ^ 



12 



u 


u 




u 





u u u n 



(h) 



a_— a 



(i) 

Fig. 18. Part two 



324 



THEORY OF SELF-REPRODUCING AUTOMATA 



b* 
t 





i 
i 

3 










i 






♦ 

I 
l 
l 
I 

u+2 

i 
i 
I 
i 
i 






^_ i 

i 
i 

\ 







(j) 



• 4 "A 



(k) 



Fig. 18. Part three 



~r 

i 
i 
i 
i 
l 

u 



u 


u 


u 




u 


u 


0 


0 


0 




0 

-> 


0 




to 


u 


u 




u 


u 




to 







to 



TO 



iO 
iO 



u+2 



(a) 



to 
to 



"J 



to 
to 
u 

iO 
+0 
0 



I 
I 
I 

u+2 
i 
I 
I 
i 
I 
I 
i 
I 



(b) 



Fig. 19. Design of a periodic pulser: equalization of delays 



326 



THEORY OF SELF-REPRODUCING AUTOMATA 



t 

1 
1 

_ 1 

1 
1 


2k+4 


t 

I 
1 

J 1 
1 


1 
1 
1 

. 1 

u+5 

! 
1 




: . 

.AJ 
1 
1 
I 

* 


l 
1 
I 

♦ 








(c) 




1 

1 

1 
1 

1 


J0p + 3 

1 (In -*- 


t 

l 

i 


1 
1 
1 
1 

| 

u+5 

. ! 




i 

! 

/+J 

1 
1 
1 

i 


1 
1 
1 

t 








(d) 





b 



b 



Fig. 19. Part two 



FIGURES 



:i27 



1 
1 



1 


M+3 

M * 


1 
I 


1 
1 

. 1 
1 

. 1 

u+5 

■ 1 
1 




\ 
1 

1 ■ 

1 
1 
1 

t 


i 
1 
1 

I 








(e) 





(f) 

Fig. 19. Pari three 



0 


0 


0 




0 


0 


0 






— > 






-* 


u 


u 


u 




u 


u 


u 












14 









328 



THEORY OF SELF-REPRODUCING AUTOMATA 




(g) 




(h) 



Fig. 19. Part four 



FIGURES 



A 


B 


C 














D 


E 


F 


G 




0 


0 


c 


0 


c 


0 


c 


0 


c 


0 


c 


0 


c 


/ 


i 


i 


0 


0 


0 


0 


0 


0 


0 


0 


0 


to 


(1 


2 


10 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


10 


c 


b3 


0 


c 


0 


c 


0 


c 


0 


c 


0 


c 


1 


< 

0 


1 

to 


4 



Fig. 20. Alternate periodic pulser I > P(1) 



A 


B 


c 


D 


E 


F 






C 


0 


c 


0 


c 


\b 1 


1 


to 




to 




to 


2 




c 




to 




to 


3 


0 


to 




to 




to 


4 


to 


0 


y //< 


to 




to 


5 


0 


to 


0 


to 




to 


6 


to 


0 


to 


0 




to 


7 


0_ 


to 


0 


to 




to 


8 


to 


0 


to 


0 




to 


9 




c 


0 


c 


0 


c 


10 



Fig. 21. Decoding organ D (10010001) 



330 



THEORY OF SELF-REPRODUCING AUTOMATA 





4 


4 


b 3 




















\ 


J 


1 




i 




c 


0 


c 


0 


c 


0 

— ► 




0 


c 


u 


to 




TO 




TO 




TO 


u-1 


To 




TO 




TO 




TO 


2 


To 




TO 




TO 




To 


1 


To 




TO 




TO 




TO 




c 


0 


C 


0 


C 


0 




0 


C 




1 


2 


J 




k 



(a) 

Fig. 22. Design of a decoder 



FIGURES 



i i t 



u c 


0 

— ♦ 


c 


0 

— ♦ 


c 


0 




0 

— ♦ 


c 


*; 




N 3 






0^ c 


JO 


c 


0 

— ► 


c 


0^ 




0^ 


c 



/ 1 2 3 
old a 



(b) 




(0 



Fig. 22. Part two 



332 THEORY OF SELF-REPRODUC1 NO AUTOMATA 



A B C D E F G H I J 




Fig. 23. Triple-return counter <l> 



FIGURES 



333 




Fig. 24. Design of triple-return counter <f> 



334 



THEORY OF SELF-REPRODUCING AUTOMATA 




FTGTJRES 



335 



ABCDEFGHIJ 




Fig. 25. 1 vs. 10101 discriminator ^ 



A B C D E F G H I JKLMNOPORS T 




Fig. 26. Recognizer R (101001) 



THEORY OF SELF-REPRODUCING AUTOMATA 




Fig. 27. A coded channel 



338 



THEORY OF SELF-REPRODUCING AUTOMATA 




Fig. 28. Part two 



FIGURES 



339 




Fig. 28. Part three 



340 



THEORY OF SELF-REPRODUCING AUTOMATA 




Fig. 29. Pulsers and decoders of a coded channel 



341 




Fig. 30. Cyclicity in a coded channel 



342 



THEORY OF SELF-REPRODUCING AUTOMATA 



MC 







X 2 




X 


X 






x o 




n-l 


n. 







J 

(a) 



n 



»!fl 



* 














ot 


ot 


Ot 




0 










X o 








X . 
n-l 


x n 
n 






r 




0 


0 


0 




to 


0 






•4- 


4- 




4- 





.J 



(b) 



Fig. 31. The linear array L, the connecting loop Ci , and the timing loop C 2 



FIGURES 



r 



w i\ 



.J 



» I 



(c) 



4 



c 
11 


/ \ 


0 


0 


0 




0 


40 


*0 








X 4 
n-1 


A- 


0 

4- 


0 

4- 


0 

4- 




0 

4- 


0 

4- 













(d) 



K +1 



Fig. 31. Part two 



THEORY OF SELF-REPRODUCING AUTOMATA 



"A 



0 


0 


0 

-> 




0 

— ► 


10 


i 






X i 


X 2 




X 
n-1 


n 


/7+i 








0 

4- 


0 

4— 


0 

4- 




to 


H 











0 


0 


0 

. 




0 

— ► 


10 




0 

4- 


0 

4- 


0 

4- 




0 

4- 


0 

4- 





V A 



U A 



w 2 \ 



it 



it 



(e) 



(f) 



0 


ot 


0 




0 

— # 




a 




*i 






X 
n-1 


X 
n 


X 
n+t 




*0 


X 2 








to 


0 

4- 


0 

4- 




0 

4— 


to 













0 


0 


0 

-♦ 




0 


AO 




0 


0 

4- 


0 




0 


0 


4- 


4— 




4— 





Fig. 31. Part three 



u 2 



w 2 



w P \ 



c 

1 4 
11 




f\ 

u 


0 


0 




0 


40 








— ► 


u 


0 

4— 


0 

4- 




0 

4- 


0 

4- 




0 








/7-i 


n 






(a) 


c 

♦ 1 












1 


1 

— ► 


1 




1 


i 




0 

4- 


0 

4- 


0 

4- 




0 

4- 


0 

4- 




0 


1 


2 




n-1 


/7 



41 
v 2 \ 

w 2 \ 



w 2 



(b) 



c 












41 












1 


1 


1 




1 

— ► 


1 

— ► 


U 




0 


0 


0 




0 

4- 






4- 


4- 


4— 




0 


1 


2 


n-1 


n 


n+1 



(0 



c 

41 




0 

— ► 


0 


0 

-* 




0 

-> 


0 


41 




0 

4- 


0 

4— 


0 

4- 




0 


0 

4- 


0 




4- 


4- 


0 


1 






/?-/ 


n 


n+i 




(d) 


4.1 
















0 


0 




0 


0 


40 


2 


-¥ 








-♦ 


2 


0 

4- 


0 

4— 




0 

4- 


0 

4- 


0 

4- 




(9 


J 






n-1 


n 


n+1 



(e) 



Fig. 32. Lengthening the timing loop C2 



346 



THEORY OF SELF-REPRODUCING AUTOMATA 



1 c 
il 

1 o 

■ — ► 






0 


0 

-4 




0 

-4 


iO 








x 2 




x 4 

n-J 


iO 


A- . 








1 o 

1 4- 


0 

4- 


0 

4- 




0 


0 

4— 






1 c 
il 




(a) 


| 1 


1 

-4 


1 

-4 




1 

-4 


iO 








X 2 




X„-j 














1 o 


0 


0 

4- 




0 


0 

4- 


r 





(b) 



41 


c 
il 












1 


1 

—4 


1 

-4 




1 

—4 


il 1 i 


































x , 






0 

4- 


0 

4- 


0 

4- 




0 


1 10 

■ -» ' «- 1 









*4 



(c) 



c 














a 






















0 

-4 


il 


I 


0 


0 

-4 


0 

-4 










Xj 


x 2 




x„-i 


il 


X„,i 








0 

4- 


0 

4- 






0 

4- 


1 

1 1 
-* 


0 

4- 




0 

4- 




r 



(d) 



Fig. 33. Lengthening the connecting loop Ci 



FIGURES 



347 



C 1 

0 10 



0 I 0 

«- ' *- 



(e) 



c 






1 

— » 


1 


1^ 




1 

— ► 


1 

— » 


u 








X 2 




Xn-i 


iO 










to 


to 


0 

4- 




to 


to 


0 

4— 


c 





(f) 



c 
ii 






1 

— ► 


1 


1 


1 


1 

-4> 


iO 


c i 






X 2 




40 










0 

4- 


0 

4- 


to 


to 


to 


0 


r. 



(g) 



c 
il 






0 


0 

-*• 


0 

-*• 


0 


0 

— ► 


iO 




X o 


X J 


X 2 




iO 


X n +J 








0 

4- 


0 

4- 


0 

4- 


0 

4— 


0 

4- 


0 

4— 





(h) 



Fig. 33. Part two 



348 



THEORY OF SELF-REPRODUCING AUTOMATA 



v 2 
w 2 



" 2 \ 



c 










li 










1 


1 


1 

-4 




l 

-> 


iO 




0 

4- 


0 

4- 


0 

4- 




0 

4- 


0 

4- 




0 


1 


2 




n-i 




(a) 


c 










41 










1 

— ► 


1 


i 




1 

-4 






0 

4— 


0 

4- 


0 

4- 




0 


u 






i 


2 








0 


(b) 




c 










41 










0 

-4 


0 

-4 


0 

-4 




0 

-4 






0 

4— 


0 

4- 


0 

4- 




0 


u 

1 




0 


7 


2 


n-J 


n 



(c) 



Fig. 34. Shortening the timing loop C 2 



FIGURES 



3 19 



U 2 



c 












il 












0 


0 


0 




0 

— • 


U 








0 

4- 


0 

4- 


0 

4- 




0 

*- 1 


u 

1 


C 2 






0 


1 


2 


n-J 


n 


n+I 



(d) 



c 
il 






1 


1 

— » 


1 

-* 




1 


0 


U I 




to 


to 


to 




to 


— 1 

U I 

- J 





n-2 n-1 n n +1 



(e) 





c 












SI 




1 


1 




1 

-* 


iO 




0 

4- 


0 

4- 


0 

4- 




0 

4- 


^1 






0 


i 


2 


... n -2 


/7-i 



(f) 





c 
a 






0 


0 


0 




0 

—* 


10 




0 


0 

4— 


0 

4- 




0 






0 


1 






n-2 n-i 



(g) 

Fig. 34. Part two 



c 

Li 






1 


1 


1 

-> 




1 


40 






y 








* , 

n-1 










o 

4— 


0 

4- 


0 




0 

4- 


0 

4~ 


c 





(a) 



c 












41 












1 

— ► 


1 






1 


41 








x o 


X J 








41 










0 

4- 


0 


0 

4- 




0 

4- 


1 1 

U 
1 






r. 



(b) 



c 




41 


0 


0 

— ► 


0 




0 4i 




*0 








1 




0 

4" 


0 

4- 


0 

4- 




0 1 u 

«- 1 





n+1 



(c) 



4 



41 
0 



0 40 



0 1 u 



(d) 



Fig. 35. Shortening the connecting loop Ci 



c 
a 






1 

— » 


1 


1 

— ► 




1 

— » 


iO 




X, 






X„-j 


iO 


X n+1 








0 


0 

*- 


0 




0 

«- 


u 




c 
il 


(e) 


1 


l 


1 




1 


u 




*i 


** 




X„-j 


iO 










0 


0 


0 

4- 




0 

«- 


u 




c 
il 


If) 




0 


0 


0 




0 

-* 


1 

-♦ 


- -1 

u 




X o 


Xj 


x 2 




x„- 2 


X 4 


iO 






0 

4- 


0 


0 




0 

4- 


u 




1 c 

il 


(g) 


c 


1 


1 5 


0 


0 




0 


iO 






Xj 


*> 








10 


X„+j 




1 o 


0 


0 

4- 




0 

4- 


c. 



(h) 

Fig. 35. Part two 



u 
i 



10 



T 



*,_,U0 IX 



n-l 



n+J 



(a) 



l 11 



(b) 



c 
11 






ot 


0 


0 

-> 




0 

-♦ 


0 


n 

u 




*0 


Xj 






X „-i 


u 










to 


0 


0 

«- 




0 


0 

«- 


0 


c i 




C 
ii 


(c) 




0 


0 


0 

— ► 




0 

— * 


0 


10 


c. 














u 










0 


0 

4- 


0 

♦- 




0 

4- 


0 


0 







(d) 



Fig. 36. Writing "zero" in cell x n of the linear array L when lengthening Ci 



FIGURES 



353 



Inputs to and Outputs from CU 



/ 



\ 



cc„ 



RWEC 

(Lengthening) 



PP(1) 

(Write 0) 



PP(1) 

(Write 1) 



RWEC 

(Shortening) 



PP(1) 

(WriteO) 

1 PP(j) 

[(Write 1) 



cc 



V////////////////////A 




v r i\ 



w 

(Delay Area) 



1 P(IOIOI) 



si 



RWE 



c 
ii 




Pi 


o 


g 




0 


0 


iO 


X o 








X n-i 








^ 




0 


0 




0 


0 


0 




<: 

C 
H 




0 


0 




0 


0 




c 2 


0 


0 




0 


0 


0 



Memory Control MC 



Fig. 37. Tape unit with unlimited memory capacity. 
Note: the various organs and units are not drawn to scale. 



354 



THEORY OF SELF-REPRODUCING AUTOMATA 



/ 5 + 1 

(e = 



-1) 



01 



Inject the sequence 10101 at v\. 





Wi — * 


a of ^. 


(in = 0) 


6 of 


h. 


(in = 1) 


c of ^ — > 


12. 


(i'» +1 = 0) 


0 2 — > 


Starts a 0 . 


ttn +1 = 1) 


0 3 ~* 


Starts «i. 


= 1) 


0 4 -> 


(28 , .a)-(28 , .d) 



(29'.a)-(29'.d) 
I 



(ao active.) 

I 

Stops ao- 

(29'.e')-(29'.h') 



(ai active.) 

i 

Stops ai. 
(29'.e)-(29'.g) 



06 



(30'.a)-(30'.g) 
(31'.a)-(31'.c) 
I 



(ao active.) 
I 

Stops ao. 

(31'.d') 



(ai active.) 
i 

Stops OL\. 

(31\d) 



(31'.e)-(31'.h) 
4. 

Fig. 38. The logical structure of the procedure 
followed by the memory control MC. 



356 



THEORY OF SELF-REPRODUCING AUTOMATA 




Fig. 39. Read-write-erase unit RWE. (b) Lower part of RWE. Note: 
parts (a) and (b) of Figure 39 constitute one continuous drawing. 



FIGURES 



357 




To triple-return counter 
input {$-0 or $ -0) of 
RWE 2 



From triple-return counter 



output ifyb or^-^ofRWE 



This goes via delay area W 
to start a periodic pulserof 
RWE: 

To U 2 o+ot V 2 -a Jf if 
is used. 

To Uj • o+ or Vj • <7 + if * 
is used. 



To stop periodic pulserof 
RWE 

To U 2 - o_ or V 2 - 0. if $ y 
is used. 

Toty-<7_or V t -a_ if <f> 
is used. 



To stort pulser of RWE (if needed); and 
to either start next CO or send pulse to/j 



Fig. 40. Control organ CO 



Inputs to and Outputs from. CU 




CC 2 RWEC CC 3 



V\£. 11. Read-write-erase control RWEC. (a) Control organs for lengthen- 
ing C2 (COiandC02) and for lengthening the lower part of Ci (CQ 3 and CO4). 



FIGURES 



359 




Fig. 41. Read-write-erase control RWEC. (b) PP(1) to store the fact that 
a "zero" is to be written in x n , and control organs for writing a zero and 
lengthening the upper part of Ci . (This figure is continued from p. 358.) 




Fig. 41. Read-write-erase control RWEC. (c) PP(1) to store the fact that a 
"one" is to be written in x n , and control organs for leaving a one in cell x n and 
lengthening the upper part of Ci . (This figure is continued from p. 359.) 



CC 2 RWEC CC 3 




To input 
a of CO,: 



Fig. 41. Read-write-erase control RWEC. (d) Control organs for shortening 
C 2 • (This figure is continued from p. 360.) 




Fig. 41. Read-write-erase control RWEC. (e) Control organs and PP(1) 
for shortening the lower part of Ci and writing in cell x n . (This figure is con- 
tinued from p. 361.) 



FIGURES 




364 



THEORY OF SELF-REPRODUCING AUTOMATA 



A 


B 


c 










H 


So 


1 
f 






C 




So 


i 
? 


L 

• t 


c oi 


f 


'/// 




A 

T 


• t 


c oi 






C 






C 




1 


c 




t 


10 




t 


V/ 






1 


t 


•t 


c o. 


t 


1 


c 












c 






So 




t 


i 




t 


So 




• t 


c o. 


♦ 


c 






•t 


c o. 



°2 



(a) Crossing organ 
/ c 



H 

h I 



c 

10 


T 

030 


T on 


c o. 



g f 

Output from /' and / 
Output from e and f 



J 
2 
3 
4 
5 
6 
7 
8 



101 010 
010 101 



(b) Initial state of clock 



Fig. 42. Crossing organ 



o — 



0> 












3 


■o 




o» 




O O 


♦4— 


to 

3 


</> </> 


this 


O <i> 
CO t5 


JO Jh 


v»— 






o 


o x 


o o 


v. 


* = 


2 2 


3 
£L 


3 0> 
QL X 




C 


c *" 


^ <r 
















N 


5 



X> . X) 



tVi (VI 



2 2 



|2 £ 



T3 




0) a> 


o o 


a> 

</> 




_Q XJ 


a> a> 


3 




o o 


o o 


SO 


CL 


tn </> 


CVJ Csl 


it ste 


=° 


O u 


o 




o o 

♦4— *♦— 


Vi 
3 


<1> 

c 


O U 
2 2 


o b 
2 2 


Q. 


4) 

x: 


o o 


^ o 


e 


V- 






|2 


O 







fa 




O 

CD 



o3 



E * 



o 

2 a 

•4- cn 



E c Q. 

0 H- 

Ll in O 



° O 
01 CO 



p 

to 



CJ 

2 



ii 



o 
2 



£ 
o 
Ll 



— 3 



E £ 



C7> = 

o 

SE £ 

E 



366 



THEORY OF SELF-REPRODUCING AUTOMATA 



D E 



=> 




~o\ 


t 



o' 

(a) Head of constructing arm 



A B C D E 



s\ 


=> 


=> 








o\ 


— > 




-> 




t 



(b) Head fed from left 



D 


£ 




=> 


=> 




ft 


t 




ft 


t 


j 


ft 


t 




ft 


f 


5 









(c) Head fed from below 

Fig. 44. Constructing arm 



(a) 

Starting with 



(b) 

V<= U 



(c) 

u t u 



*l 

into s or s' produces 'I 



5| 

into o or o' produces I 



A 






=> 




u 


o\ 


t 


u 








A 


B 




=> 


=> 






-> 


<= 




<r 




A 


B 




=> 


=> 




^1 


-> 





2 



1 

2 



0' 



Fig. 45. Horizontal advance of constructing arm 



FIGURES 



367 



(a) 

Starting with 



(b) 

U t * ft 



u = 



into o or o' produces 



into 5 or 5' produces 



A 


B 


U 


U 




=> 




t 


A 




\ 




ft 


t 




t 

0' 


A 




=> 


=> 


ft 


t 




t 



Fig. 46. Vertical advance of constructing arm 



THEORY OF SELF-REPRODUCING AUTOMATA 



(a) 

Starting with 



(b) 

U ^ into o or o' produces 



U < 



u t 



into 5 or s' produces 



(d) 

U ~> U { U S into o or o' produces 



(e) 
u = 



U 7 into 5 or s' produces 









s 'o\ 












=> 






~o\ 


-> 


f 














^1 








0' 






-> 




Ol 


t 


8 












7 


5/ 1 

01 




8 









Fig. 47. Horizontal retreat of constructing arm with construction of y and 5 



FIGURES 



369 



(a) 

Starting with 



(b) 

u *■ u ^ u 



into o or o' produces 



-I 







ft 




^1 


t 




0' 


I 






t 


~o\ 


t 



(c) 



U ^ U <= U y into s or s' prod 



uces 



r 



o> 



(d) 

U t S 



into o or o' produces 





7 


s 


-1 


=> 


t 




^1 


t 









(e) 
u - 



into 5 or s' produces 





r 


s 




=> 








t 









Fig. 48. Vertical retreat of constructing arm with construction of y and 8 



370 



THEORY OF SELF-REPRODUCING AUTOMATA 



(a) 

Starting with 



s 'o\ 



(b) 
U f 



into o or o' produces 



s\ 





t 







O' 



e 



A pulse into o or o 1 exits at e 
as the stimulus to start the 
secondary automaton 





t 


*5i 


f 



U =t> into 5 or 5' produces 







^1 


t 



0' 



Fig. 49. Injection of starting stimulus into the secondary automaton 



.2 E 
o 



60 



n 



° 

s 1 

CL C 
O 

• °» 
I 5 



O 

2 



3 
O 



o 

2 



ft 
G 



372 



THEORY OF SELF-REPRODUCING AUTOMATA 



(a) Start with 





A 


B 








c 


D 


E 


v\ 


c 


-> 


-> 


-> 






-> 






u\ 


c 


=> 


=> 








=> 












X 


X 


X 




X 


X 


X 








VN 


0 


1 


2 




n-1 


n 


n+1 




w\ 


<- 


<- 

















1 

2 

L 3 

4 



in which the reading loop passes through cell X n . 



(b) The sequence 10101 into V produces: 

(1) lot W if X n is U (representing "zero") 

(2) 1 0 1 0 1 at W if X n is ♦ ( representing "one") 

(3) and leaves cell X n in state i: 

A B C D E 



V\ 

u\ 

w 



c 


-> 


-> 


-> 


-> 




-> 








c 


=> 


=> 




=> 




=> 








A' 


i 


A- 




A- 
/? -l 












<- 


<r 


<- 




<- 




<- 


<- 







J 

4 



Fig. 51. New method of operating the linear array L 



(a) Starting with 



"1 


c 


-> 


-> 


-> 






-> 










c 




=> 


=£> 




















X 


X 


X 




X 














0 


1 


2 




n-1 








w\ 




<- 






<- 






<- 





(b) U ^ U |f U 

V 

u\ 



into produces: 



W 



(c) U I U * u <- into ^ produces: 



1/ 



U 



w 



c 


->> 


-> 


-> 


-> 




-> 


1 






c 


=£► 






=> 




=> 












X 
2 






ft 


A- 










<- 


<- 


<- 








=> 









c 


-> 


-> 


-> 


-> 




-> 


1 






c 


=> 






=> 












0 


y 


A' 




X 
n-1 












<- 




<- 


«=- 






<r 


<- 


<- 






: U -» into £/ produces: 


1 c 




-> 


-> 






-> 


-> 


<= 






1 c 




=> 


=> 


=> 






=> 


it 










$ 






X 
n-i 




X 
n+i 








1 - 






<- 








<- 









(e) U i U i 

V\ 
U\ 

w 



into / produces: 



c 


-> 


-> 


-> 






-> 




1 






c 


=> 




=> 






=> 


















/?-/ 


A' 
/7 


A' 

/7+i 










<- 


<- 


<- 






<- 


<- 


<- 







Fig. 52. Writing "one" in cell x n and lengthening the reading loop 



SYMBOL INDEX 



Note: page references are to the text locations where symbols are denned 01 
formally introduced for the first time. 

A (a, £) (CU transition function), 206 
C €t f (confluent state), 148 
Ci (connecting loop), 208 
C2 (timing loop), 213 

CCi , CC 2 , CC 3 (coded channel of MC), 227 

CO (control organ), 244 

CU (constructing unit) , 205 

D (delay ar ea of CO), 245 

D^' 1 • • • i n ) (decoding organ), 176 

3D(M), SD'(M) (description of a Turing machine) 270-271 
E{a) (CU output function), 206 
FA (finite automaton), 267 

(cell vector), 133 
ii , ii , H (inputs to CU, outputs of MC), 232 
i 1 • • ■ i n (rigidly timed sequence), 157 
%i . . . {n (periodic repetition), 158 
L (linear array), 202 
M c (universal constructor), 271 
M c * (modified universal constructor), 295 
M u (universal computing machine), 270 
MC (memory control), 205 

(state of cell # at time t), 133 
n s (subsequent location to read value of £ n ), 204 
0 1,02,03, 0 4 , o 5 (outputs of CU, inputs to MC), 232-233 
P ffl ■ • • i n ) ( pulser), 159 
P P^ 1 - ■ • i n ) (periodic pulser), 163 
R^ 1 • • • i n ) (recognizer), 189 
RWE (read-write-erase unit), 226 
RWEC (read-write-erase control unit), 226 
Ss (set of direct-process states), 145 
SO« (state organ), 267 
Tuae (transmission state), 148 
U (unexcitable state), 140 
W (delay area of MC), 227 
Wi , W 2 , W 3 , W 4 (sub-areas of W), 256 
X (area X of MC), 227 

Xi f yi (secondary automaton coordinates) . 116 

x n (cell of L under scan), 203 

X(a) (CU output function), 206 

Y (transfer area of MC), 227 

Z (area Z of MC), 227 

Zi , Z 2 , Z 3 , Z 4 (sub-areas of Z), 256 



379 



380 



SYMBOL INDEX 



a (length of secondary automaton), 116 

/3 (width of secondary automaton), 116 

e s (lengthen-shorten parameter for G , C 2 ), 204 

\ij (state of cell 117 

£ n s (value of the n th cell of L), 204 

<£ (triple-return counter), 181 

• (1 vs. 10101 discriminator), 187 
ft (responding organ of $), 181 

[0] (ordinary stimuli), 148 
[1] (special stimuli), 148 

Z, to, 10 (ordinary transmission states), 152 
J>, ti, <L, |i (special transmission states), 152 

• ("and"), 100 
- ("not"), 100 
+ ("or"), 100 

^ , * , J, (ordinary transmission states), 262 
ft j ^> U (special transmission states), 262 

• t , • ft , etc. (ordinary or special transmission states initially active), 262 



AUTHOR. SUBJECT INDEX 



Acoustic delay line. See Storage de- 
vices 

Analog computers, 21, 22, 35-36, 68- 
70, 98 

Area X, 227, 252-253, 293 
Area Z, 227 , 253, 293 

construction of, 256-257 
Automata 

artificial, 21-25. See also Com- 
puters 

complex, 20, 32, 79 

complexity of, 36-37 

complicated, 20 

constructed. See Secondary (con- 
structed) automaton 

constructing. See Primary (con- 
structing) automaton 

construction aspects of, 92 et pas- 
sim 

efficiency of, 37-39, 91-92 
formalistic study of, 91, 102 
human nervous system and, 9-10, 
43-49 

logical aspects of, 92 et passim 
natural, 21-25, 64 
non-constructible, 291 
probabilistic, 99 

self-reproducing, 19, 21, 294-296. 

See also Self-reproduction 
single-cell, 111 

see also Computers; Finite au- 
tomata; Infinite cellular au- 
tomata 
Automata theory, 10, 17-28 

biology and, 21 

communication and control en- 
gineering and, 21 

continuous mathematics and, 25- 
27, 97 

mathematical logic and, 10, 19, 
25, 43-45, 47-48, 49-56 

thermodynamics and, 28 
Automatic programming, 5, 14-15 
Automaton milieu, 72-73 
Axiomatic method, 43-44, 76 



Balance, 28, 40-41, 63 
Base two, 114 
Behavior, 270 
Bigelow, J., 12, 105 
Birkhoff, G., 2, 34, 59 
Bitwise implication, 175 
Black box, 45 

Boltzmann, L., 26, 59, 60-61 
Boltzmann's constant, 66 
Boolean algebra, 100 
Booth, A. D., 16 
Brainerd, J. G., 6 
Brillouin, L., 68 

Burks, A. W., 6, 12, 37, 43, 126, 262, 
270, 271, 291 

Calculation chain, 24 

Cellular automata. See Cellular 
structure; Infinite cellular au- 
tomata 

Cellular model, description of, 106- 
108 

Cellular structure, 94, 103-106, 288 
construction-universality of, 92, 
116 

logical universality of, 265-271 

tape-reading method in, 283 

tape unit in, 26, 293 

von Neumann's 29-state, 291-292 
Church, A., 261-262 
Codd, E. F., 280 

Coded channel, 180, 190-200, 227, 
228, 239-243, 252, 293 
behavior described, 190-191, 227 
capacity of in MC, 239-241 
construction of, 192-194 
corruption problems in, 191, 196- 
198 

cyclicity in, 198-200 
dimensions of, 195, 242-243 
function of, 190, 191-192 
main channel of, 193 
rule for avoiding corruption in, 198 
timing considerations for, 195-196 
Coincidence organ, 81 



381 



382 



THEORY OF SELF- REPRODUCING AUTOMATA 



Collision in single reproduction, 

avoidance of, 120-121 
Combinatorics, 62 
Communication channel, 60 
Complexity, 22, 23, 54, 58, 65-73, 
79-80, 118 

reliability and, 23 
Complication, 47-48, 64-73, 78- 86 

complexity and, 79-80 

degeneration and, 79-80 
Component size, 20 
Computation, 24, 270 

quantity of, 26 

size and reliability related, 26 

speed of, 37-41 
Computer components and effi- 
ciency, 22, 66-67, 72-73 
Computers, 32, 35-41, 75 

applications of in science, 33-34 

circuits of, 15-17 

efficiency of, 28 

heuristic use of, 3-5, 33-35 

human nervous system and, 9-10, 
43-49 

mixed synchronous, asynchronous, 
8 

see also Analog computers; Auto- 
mata; Digital computers 
Confluent states C fe ', 107, 136-139, 
147 

and the -f neuron, 136 

and the • neuron, 136-137, 138 
Connecting loop G , 115, 180, 208, 
210-213, 228, 238, 246, 293 

lengthening of, 216, 218-220 

preliminary description of be- 
havior of, 210-213 

shortening of, 222-224 
Connecting organ, 80 
Constructibility, 92, 156, 292 
Constructing arm, 271-276 

design of, 272-276 

double path procedure, 273-277 

head of, 274 

operation of, 274-275 

single path procedure, 272-273, 277 
Constructing unit CU, 205-208, 238, 
293 

design of, 279-280, 295 
function of, 201-202 



input-output connections, 232-233 
interconnections with MC, 205- 

206, 228-229, 232-233 
postulates for, 207 
schematic description of, 206-207 
viewed as special type of finite 

automaton, 286 
Construction, 101-132, 288 et passim 
geometry and kinematics of, 101- 

102 

Construction-universality, 92, 1 1 6 

of cellular structure, 286, 292 
Constructive method, 91-92 
Control organ CO, 227-228, 229, 230, 
245 

delay adjustments in, 244-245 
operation of, 230 
see also Memory control unit 
Conversion of freely timed sequences 
to rigidly timed ones. See Static- 
dynamic converter 
" Copying," use of descriptions vs. 
originals in self-reproduction, 84, 
121-122 

Crossing lines and paths. See Wire- 
crossing 
Crossing organ, 262-263 

clock sequences in, 262 

used to solve interference problem 
for MC, 263-264 
Crystal, 103 
Crystal lattice, 104 
Crystalline regularity, 93, 94 
Crystalline structure, 132 
Crystalline symmetry, 103-104 
Cutting organ, 81 

Decision machine, 52 

Decoding organ D^' 1 • • • i n ), 175-179 

behavior described, 175 

characteristic of, 176 

construction of, 175-179 

dimensions of, 176-177, 178 

order of, 176 

timing considerations for, 176, 177, 
178-179. 
Degenerative processes, 62 
Delay area D, 246 

dimensions of, 256 
Delay area W, 227, 241-242, 243 



A.UTHOR SUBJECT INDEX 



construction of, 256-257 
delay considerations and, 257-258, 
293 

Delay line. See Storage devices 

Delay paths, 146 
odd, 146-147 

Delays, single, 147 

through confluent states, 147 

Descriptive statement L, for nu- 
merical parameters, 112-113 

Differential equations of self-repro- 
duction, 97, 106 

Diffusion processes, 97-98 

Digital computers, 21, 22, 35, 36, 69- 
70, 98 

Digital notation, 48 

Digital organ, 69-70 

Digitalization, 61-62 

Dimensionality, 104-105 

Direct process, 107, 111, 142-145, 
272 

need for control by fixed stimulus 
sequences, 143-145 
Directed process, 135 
Double line trick, 138 

Eccles, J. C, 97 
Eckert, J. P., 6, 8 
EDSAC, 9 

EDVAC, 9-11, 19, 158, 261 

Efficiency, 28, 40, 48, 67, 93. See also 
Computer components and effi- 
ciency 

Elementary parts, 77 

Energy and information, 66-67 

ENIAC, 6-10, 19, 37, 48, 65 

Entropy, 59-63, 67 

Entscheidnngsproblem, 49, 204. See 
also Halting problem 

Error-detecting and -correcting codes, 
61 

Error detection and correction, 24- 

25, 73 
Estrin, G., 12 

Evolution, 79, 92-93, 99, 131 
Excitation, 44, 97-98 



Failure, 58, 70-73 
Fatigue, 44, 96, 97-98 



Finite automata, 108, 114-115, 267, 
286 

embedded in cellular structure, 
267-268, 293 
Flip-flop, 174 
Flow diagram, 13-14, 84 
Free timing, 157 
Fusing organ, 81 

"Garden-of-Eden" configuration, 291 

Gene-function, 130 

Godel, K., 25, 53, 55-56, 125, 126 

Godel number, 55 

Godel's theorems, 47, 51, 53-56 

Godel's undecidable formula and 

self-reproducing automata, 126 
Goldstine, H. H., 4, 6, 12, 37, 95, 

105, 279 
Gorman, J. E., 262 
Goto, E., 16, 17 
Growth, 109, 110 
Growth functions, 139-142 

Halting problem, 52-53, 124-126 
undecidability of and Richard's 
paradox, 125-126 
see also Entscheidungsproblem 
Hamming, R. W., 61 
Hartley, R. V. L., 59, 61 
Heuristic use of computers, 3-5, 33, 
35 

Hixon Symposium, 53, 81 
Holland, J. H., 99, 262, 270 
Homogeneity, 72, 103-106 

functional or intrinsic, 103-104, 
106, 288 

total, 104 
Homogeneous medium, 103 
Hydrodynamics, 2-3, 34 

computers and, 3 

Idealized computing elements and 

computer design, 9-10 
Idealized excitation - threshold - fa- 
tigue neuron, 96 
Idealized neurons, 44 ff., 287 
Idealized switch-delay elements, 25 
Infinite cellular automata, 108 et pas- 
sim 



384 



THEORY OF SELF-REPRODUCING AUTOMATA 



spatial and temporal relations, 
132-134, 152 
Information, 18, 26, 57, 59-63, 66, 
67,78 

Information theory, 19, 27, 42, 60-63 

et passim 
Inhibitory organ, 81 
Initial cell assignment, 108, 152, 291 
Initially quiescent automata, 264, 

291 

Input direction, 135 
Isotrophy, 105, 106, 288 

JONIAC, 12 

Kemeny, J., 95-96 
Keynes, J. M., 59 
Kinematics, 101 

Kleene, S. C, 43, 101, 123, 125 

Language, complete epistemological 

description of, 55 
Laplace, P. S., 58 
Lee, C. Y., 290 

Linear array L, 112-116, 202-204, 
259, 293 

altering x n in, 210-212, 224-226 
described, 203 
function of, 203 

function of Q and C2 in lengthen- 
ing and shortening on, 214-216 

lengthening on, 216-220 

moving its connection with MC, 
214-226 

operations on summarized, 209- 
210 

read-write sequence of operations 

on, 209-210 
shortening on, 220-224 
use of for non-numerical (univer- 
sal) parametrization, 116 
Logic, formal, 42-43 
Logical depth, 24 
Logical operators, 42, 99-101, 111 
Logical organization, 20, 22, 23-24 
Logical universality, 92, 265-271, 
287, 292 

McCulloch, W., 9, 43 ff., 77, 100, 101 
McCulloch-Pitts neuron nets, 43-49, 
75, 80 



Machine language, 14-15 
Machine-man interaction, 5 
McNaughton, R., 270 
Main channel of coded channel, 193 
MASER, 16 

Mathematical logic, 10, 19, 25, 43-45, 

47-56 
Mauchly, J., 6 
Maxwell's demon, 60, 61 
Memory, 39-41, 67-68, 101, 203 

access to, 40-41 

capacity of, 40-41, 68 

hierarchical, 23-24, 41 

human, 39, 48-49 

unlimited, 113-114 

virtual, 68 

see also Storage devices 
Memory control unit MC, 115, 201- 
202, 205-206, 226-250, 251-252, 
293, et passim Ch. 4 
constructing devices in, 264-265 
control organ CO in, 243-246 
corruption considerations, 253-255 
delays in control processes in, 254- 
259 

design modifications of, 257, 264- 
265 

dimensions of, 244 
dimensions of areas X, Y, Z, W, 
253 

function of, 205-206, 231-238 

interconnections with CU, 205- 
206, 228-229, 232-233 

modu s operandi on L, 207-210 

1010 as no-response characteris- 
tic, 209 

operation of, 228-229 

organization of, 226-228 

postulates for, 207 

redesign using double path pro- 
cedure for reading L, 277-279 

solution of interference problem in, 
259-264 

Mixed analog-digital systems, 22, 27 

Monte Carlo method, 6 

Moore, E. F., 94, 291 

Morgenstern, O., 2, 59 

Morphogenesis, 99 

Motor organ, 80 

Muntyan, M., 279 

Muscle organ, 77, 80, 81-82 



AUTHOR-SUBJECT INDEX 



385 



Mutation, 87, 130 
Myhill, J., 291 

Natural selection, 131 
Negation, synthesis of, 138 
Neighbors, immediate, 133 
Nervous system, 9-10, 39, 42-48, 64 

complexity of, 37 

languages of, 15 

probablistic logics and, 15 
Neuron fatigue, 48-49 
Neuron response, 100 
Neuron stimulus, 100 
Neurons, 77, 99-101 

excited, 44 

quiescent, 44 
Non-Euclidean spaces, 103 
Non-linear partial differential equa- 
tions, 2, 19, 33-34, 97 

of self -reproduction, 97, 106 
Nyquist, H., 61 



1 vs. 10101 discriminator^, 187-189, 
209, 292 
behavior described, 187 
construction of, 188-189 
dimensions of, 188-189 
function of, 187 

timing considerations for, 188-189 

Optimality and minimality, 91-92 

Ordinary stimulus, 110-111 

Ordinary transmission states Toae > 
107, 134-136 
as connecting lines, 135-136 
logical-neural functions and, 134 

Output direction, 135 

Parallel processing, 7, 22, 23, 157- 
158, 261 

Parametron, 16 

Periodic pulser PPft 1 • • • i n ), 162- 
175, 238, 292 
alternate periodic pulser PP(1), 174 
characteristic of, 163 
construction of, 163-175 
corruption of by interference, 169- 
170 

dimensions of, 163-164, 170-171 
external characteristics sum- 
marized, 174-175 



operation illustrated, 162-163 

order of, 163 

phasing for, 173-174 

rules for avoiding corruption (final 
version), 172 

rules for avoiding corruption (ini- 
tial version), 169-170 

special periodic pulser PP(1), 163, 
167, 227 

special periodic pulser PP(1), 
shown defective and altered, 
174 

start mechanism of, 164 
stop mechanism of, 165-166 
timing considerations corrected, 
172-173 

timing considerations for, 168-169, 
170-171 
Periodic repetition, 158 
Pitts, W., 9, 43 ff., 77, 100, 101 
Primary (constructing) automaton, 

82, 111, 116, 271, 288, 289 
Probabilistic logics, 20, 26, 58-63, 99 
Probability theory, 99 
Programmer's language, 14-15 
Programming, 12-15 
Propositional functions, 100-101 
Pulser P(i* • • • 159-162, 292 

behavior described, 159 

characteristic of, 159 

construction of, 159-161 

dimensions of, 159-161 

external characteristics sum- 
marized, 162 

order of, 159 

timing for, 160-161 

Quadratic lattice, 132 
Quantum mechanics, 59, 62 
Quiescence, total, 104 
Quiescent states, 99, 103, 106-107 

Rajchman, J. A., 12 
Read- write mechanisms, 114-116 
Read-write-erase control unit RWEC, 
226, 227, 246-250, 251, 259, 293 

control organs CO of, 229-230 

dimensions of, 242-243 

function of, 246-247 
Read-write-erase unit RWE, 226- 



380 



THEORY OF SELF-UEPltODUCING AUTOMATA 



227, 230-231, 237-243, 251, 259, 
293 

construction of, 237-243 

dimensions of, 241 
Recognizer R(iH 2 • • • i n ), 189-190 
Recognizing device, 175. See also 

Recognizer 
Recursive functions, 25 
Redundancy, 60 

Refractoriness, 96. See also Fatigue 
Refractory period, 48, 96 
Regularity, 105 

Reliability, 22, 23, 24-25, 67, 70-73 
Reliable systems, and unreliable 

components, 19, 70-71 
Responding organ Q, 181, 215 
Reverse process, 107, 111, 140-142, 
272 

need for, 143 
Richard, J., 123 
Richard paradox, 123-124 

and self -reproduction, 122-123 
Rigid binary tape, 83 
Rigid member, 81 
Rigid timing, 157 
Russell, B., 125 

Scientific method, 3 
Secondary (constructed) automaton, 
111, 271, 288 
algorithm for constructing and 

starting, 276 
algorithm for constructing and 

starting revised, 283-285 
complete description of initial con- 
ditions for, 117 
construction of, 272, 280-285, 290 - 
291 

dimensions of, 116-117 
initial state of, 117, 127-128 
parametric form plan of, 112-113 
plan of, 111-112 
positioning of, 126-127, 129 
starting stimulus for, 128 
universal type of plan of, 116-118 
Self-reference and self-reproduction, 
125-126 

Self-reproducing cellular automaton, 
294-296 

Self -reproduction, 19-21, 78-86, 92, 
95, 106, 289-291, 292 



a priori argument against, 79, 118, 
121 

cellular model of, 93-95 

circumvention of a priori argu- 
ment against, 118-119 

conflicts and collisions of progeny, 
130 

continuous model of, 93, 95, 97-99, 

105-106, 288 
differentia] equations of, 97, 106 
cxcitation-threshold-fatigue model 

of, 93, 95, 96 
in a cellular structure, 294-296 
kinematic model of, 80-87, 93-94, 

287-288 

physiological aspects of for auto- 
mata, 129-131 

probabilistic model of, 93 

sequential, 128-129 

single-action, 128-129 
Sequencing, 111-112 
Serial processing, 22, 23 
Shannon, C. E., 27, 59-61 
Sharpless, T. K., 6 
Simulation, 14-15 
Size, 64-65 

Special stimuli, 110-111, 288 

origination of, 141-142 
Special transmission states Ti ae , 107 
Speed, 65-66 
Split, 139 
SSEC, 65 

State organ SO, 267-269 
State transition rule 

derived for confluent states, 136- 
138 

derived for ordinary transmission 

states, 135-136 
derived for sensitized states, 144- 

145 

derived for special transmission 

states, 140 
illustrated, 151-156 
modifications required by reverse 

process, 140-141 
modified for confluent states, 147- 

148 

rigorous description of, 148-150 
verbal summary, 150-151 

States 

active, 103 



AUTHOR-SUBJECT INDEX 



UH7 



duality of ordinary and special, 

142-143 
excitable, 107, 109, 288 
excited, 99 
initial, 108, 264 

of cellular automaton, verbal sum- 
mary, 150-151 

sensitized, 107, 145 

see also Confluent states; Ordinary 
transmission states; Special 
transmission states; Unexcita- 
ble state 

Static-dynamic converter, 281-283 

Stationarity, 103 

Stimuli producer, 81 

Stimulus organ, 81 

Stop at step /, 158 

Stop in period s, 158 

Stop in phase k in period s, 158 

Storage devices, 7, 10, 41 

acoustic delay line, 8, 67-68 

cathode ray tube, 68 

electrostatic, 11-12 

film, 75 

magnetic tape, 75 

punched cards, 75 

punched tape, 74 

vacuum tubes, 7-8, 67 

see also Memory 
Strict logics, 59 
Structure, 104-106 
"Structure of the vacuum," 102-106, 
109 

Subharmonic generator, 16-17 
Survival, 72, 73 
Synthesis, 75-86 
Szilard, L., 59, 60, 61 

Tape and tape control components, 
201 

Tape-copying operation, 295 

Tape, indefinitely extendible, 92, 

202-203, 289 
Tarski, A., 55 
Taub, A., 1 

Temporal reference frame, 108 
Tessellation model. See Cellular 

structure 
Thatcher, J., 279, 280, 290, 291 
Theory of games, 21 



Thermodynamic-based logics, 62-63 
Thermodynamics, 19, 26, 28, 59-63, 

66, 91 
Threshold, 97-98 
Threshold neurons, 96 
Threshold switches, 9 
Timing loop C 2 , 115, 180, 213-214, 
228, 238, 246, 293 

function of, 213, 215 

lengthening of, 216-218 

shortening of, 221-222 

timing considerations for, 213-214 
Tissues, 77 

Total numerical material produced 

in a process, 39 
Transfer area Y, 227, 252-253, 293 
Transition rule, 132, 134 
Transmission states T wae , 135. See 
also Ordinary transmission states; 
Special transmission states 
Triple-return counter <£, 179-187, 
215, 227, 238, 292 
behavior described, 180 
coding and decoding considera- 
tions for, 184-185 
construction of, 181-185 
corruption considerations for, 186 
dimensions of, 179, 185-186 
function of, 180 
function of in MC, 234 
primary and secondary input-out- 
put pairs in, 181 
timing considerations for, 181, 
185-187 

Turing, A., 14, 25, 43,49fT., 52, 83, 

92, 99, 122, 204 
Turing machine, 25, 45, 49-53, 54, 

74-75, 107, 114-115, 289, 293 
abstract, 52 

automatic programming and, 14- 
15 

circle-free, 52, 124 

circular, 52, 124 

concrete, 51-52, 124 

see also Universal Turing machine 
29-state finite automaton cell, 94, 

106-108 
Type theory, 51, 53-55, 125 

Ulam, S. M., 1, 2, 3, 5, 6, 28, 94, 95, 
102 



388 



THEORY OF SELF-REPRODUCING AUTOMATA 



Unexcitable state U, 107, 109, 139- 
140, 288 

Universal computing machine M u , 

270-271, 285-286, 294 
Universal construction, affected by 

interference problem for MC, 

260-261 

Universal constructor M c = CU + 
(MC + L), 271-286, 289-291, 294 
general construction procedure, 272 
modified to M c *, 295 
see also Constructing arm; Con- 
structing unit 
Universal Turing machine, 50-51, 
52, 56, 83, 288 
designed in cellular structure, 266- 
270 



operation of in cellular space, 268- 
269 

Vanuxem Lectures, 95-96 
Visual pattern, 46-47, 53-54 
Volume, 66 

von Neumann, Mrs. K., 1, 94 
von Neumann's five main questions, 
92, 292 

Wang, H., 291 

Wiener, N., 22, 27, 60, 91 

Wigington, R. L., 16 

Wilkes, M. V., 9 

Williams, F. C, 12 

Wire-crossing, 180, 183, 190, 191- 

192, 261-264 
Wright, J., 43 



LIBRARY U. OF I., UR8ANA-CHAMPAIGN 




/