Man-Computer Symbiosis

J. C. R. Licklider IRE Transactions on Human Factors in Electronics, volume HFE-1, pages 4–11, March 1960

Summary

Licklider describes symbiosis as an anticipated form of cooperative interaction between humans and electronic computers, involving "very close coupling between the human and the electronic members of the partnership." The two main aims: (1) letting computers assist formulative thinking the way they already help solve formulated problems, and (2) enabling cooperative decision-making and control of complex situations without rigid reliance on pre-set programs. Humans would set goals, formulate hypotheses, determine criteria, and perform evaluations. Machines would handle routinizable work that prepares the ground for insights and decisions. Preliminary analysis suggests such a partnership would perform intellectual operations far more effectively than humans alone. Prerequisites include advances in time-sharing, memory components, memory organization, programming languages, and input/output equipment.

1 Introduction

1.1 Symbiosis

Licklider opens with a biological analogy: the fig tree and the insect Blastophaga grossorun are interdependent—the tree cannot reproduce without the insect, and the insect cannot feed without the tree. This "cooperative 'living together in intimate association'" of two dissimilar organisms is called symbiosis.

He notes that man-computer symbiosis is a subset of man-machine systems, but "at present, however, there are no man-computer symbioses." The paper aims to present the concept and encourage its development by analyzing interaction problems, applying man-machine engineering principles, and identifying research needs. His hope is that "human brains and computing machines will be coupled together very tightly" and that the resulting partnership will think in ways no human brain has and process data beyond today's machines.

1.2 Between "Mechanically Extended Man" and "Artificial Intelligence"

Licklider distinguishes his concept from what North called "mechanically extended man." In past man-machine systems, the human supplied initiative, direction, integration, and criteria; mechanical parts were mere extensions of the human body. Those systems involved only one kind of organism—man—with the rest merely serving him.

He notes that "mechanical extension" has recently given way to automation and replacement of humans. In some large computer-centered systems, remaining human operators handle functions that proved infeasible to automate—these are "semi-automatic" systems, not truly symbiotic ones.

Licklider acknowledges that symbiosis may not be the ultimate paradigm. He concedes that eventually electronic or chemical "machines" may surpass the human brain in most functions, citing Gelernter's geometry theorem-proving program on the IBM-704 that "proceeds at about the same pace as Brooklyn high school students." He references multiple theorem-proving, problem-solving, chess-playing, and pattern-recognizing programs from the era, and notes Newell, Simon, and Shaw's "general problem solver." He "concede[s] dominance in the distant future of cerebration to machines alone" but argues there will be a significant interim during which the main intellectual advances come from humans and computers working in close association. A multidisciplinary Air Force study group estimated it would be 1980 before machines alone could do much militarily significant thinking. That would leave roughly five years to develop symbiosis and fifteen to use it—years he predicts will be "intellectually the most creative and exciting in the history of mankind."

2 Aims of Man-Computer Symbiosis

Present computers are designed to solve preformulated problems or process data via predetermined procedures. All alternatives must be foreseen in advance; if an unforeseen one arises, the process halts for a program extension. This requirement is sometimes no great disadvantage—programming forces clear thinking. But many problems that can be thought through in advance are still very difficult to do so. They would be easier and faster to solve through "intuitively guided trial-and-error procedure" with computer cooperation, revealing flaws or unexpected turns. Other problems simply cannot be formulated without machine aid. Licklider quotes Poincaré: "'The question is not, "What is the answer?" The question is, "What is the question?"'" One main aim is bringing computers into the formulative parts of technical problems.

The second main aim is bringing computers into real-time thinking—processes that move too fast for conventional computer use. He imagines directing a battle with today's slow cycle: formulate the problem today, spend tomorrow with a programmer, have the computer spend 5 minutes assembling the program and 47 seconds calculating the answer next week, producing a 20-foot sheet of numbers. The battle would be over before planning was begun. Thinking interactively with a computer as one would with a colleague will "require much tighter coupling between man and machine" than currently possible.

3 Need for Computer Participation in Formulative and Real-Time Thinking

3.1 A Preliminary and Informal Time-and-Motion Analysis of Technical Thinking

Unable to find time-and-motion analyses of technical mental work, Licklider studied his own activities during spring and summer of 1957. He found that "about 85 per cent of my 'thinking' time was spent getting into a position to think"—finding information rather than digesting it. Hours went into plotting graphs, calculating transformations, and performing clerical or mechanical operations. His choices about what to attempt were "determined to an embarrassingly great extent by considerations of clerical feasibility."

His conclusion: the operations filling most technical thinking time are ones machines can perform more effectively than humans. The challenge is that these operations involve diverse variables in unforeseen, changing sequences. If those problems can be solved to create a symbiotic relationship between a person and a fast information-retrieval and data-processing machine, "the cooperative interaction would greatly improve the thinking process."

He notes the term "computer" covers a wide class of machines whose capabilities increase almost daily. He identifies genotypic differences: humans are "noisy, narrow-band devices" with highly parallel nervous systems; computers are very fast and accurate but perform "only one or a few elementary operations at a time." Humans are flexible and self-programming contingently; computers are single-minded and constrained by pre-programming. Humans use redundant languages with 20–60 symbols; computers use nonredundant languages, typically with two symbols. A symbiotic partnership integrating these complementary strengths would be highly valuable, though differences in speed and language pose difficulties.

4 Separable Functions of Men and Computers in the Anticipated Symbiotic Association

Licklider anticipates that human and equipment contributions will blend together so completely in many operations that "it will be difficult to separate them neatly in analysis." For example, both might retrieve relevant precedents, and the computer might suggest a course matching the human's intuition (as already happens in theorem-proving and the SAGE system).

Nonetheless, some functions are separable. Humans will set goals, supply motivations, formulate hypotheses, ask questions, think of mechanisms and models, recall relevant prior work, and make approximate but leading contributions. They will define criteria, serve as evaluators, handle very-low-probability situations, and fill gaps when the computer lacks applicable routines.

The computer will convert hypotheses into testable models and test them against data. It will answer questions, simulate mechanisms, display results, transform data, plot graphs (in whatever way the operator specifies), interpolate, extrapolate, and convert static equations into dynamic models. It will "carry out the routinizable, clerical operations that fill the intervals between decisions." It will also serve as a statistical-inference, decision-theory, or game-theory machine for evaluating courses of action, and will do as much diagnosis, pattern-matching, and relevance-recognizing as feasible, while "accept[ing] a clearly secondary status in those areas."

5 Prerequisites for Realization of Man-Computer Symbiosis

5.1 Speed Mismatch Between Men and Computers

Any large-scale computer is too fast and costly for real-time cooperative thinking with a single user. The computer must time-share among many users. He envisions a "thinking center" 10–15 years hence that would combine library functions with advances in information storage and retrieval and symbiotic capabilities. This expands into a network of such centers connected by wide-band communication and leased-wire services, balancing computer speed and dividing the cost of large memories and sophisticated programs.

5.2 Memory Hardware Requirements

Storing any appreciable fraction of a technical literature would require billions of bits and, without changes, billions of dollars. Licklider argues we should not store entire papers—only parts that can be succinctly summarized (quantitative sections and citations). Books, he notes, are "among the most beautifully engineered" and human-engineered components and will remain functionally important.

He describes two important memory types: indelible memory (the computer can write once and read indefinitely but cannot erase) and published or "read-only" memory (introduced pre-structured, readable but not changeable). These types can be more compact and less expensive than core, thin-film, or tape memory. The main engineering challenges concern selection circuitry. He also anticipates memory elements potentially becoming as fast as processing elements, which "would have a revolutionary effect upon the design of computers."

5.3 Memory Organization Requirements

Information must be retrievable by name and by pattern, and accessible much faster than serial search. At least half the problem lies in the storage procedure; the remainder involves pattern recognition within the storage mechanism. He outlines Fredkin's "trie memory" concept, so named because "the branching storage structure, when developed, resembles a tree."

In trie memory, both arguments and functions are stored. An argument enters the memory one character at a time from a standard initial register. Each argument register has one cell per character in the ensemble, and each cell stores the address of the next register. The argument is stored by writing a series of addresses. An end-of-argument marker follows, then directions to the function.

The scheme's attractive features: (1) retrieval is extremely simple—enter with the first character, pick up the next address, and repeat. (2) Arguments sharing initial characters use the same storage for those characters. (3) Argument lengths need not be equal or specified in advance. (4) No storage is reserved for any argument until stored. (5) A function can serve as an argument for another function, enabling retrieval of an entire program by entering an argument like "matrix multiplication." (6) Examining storage at a given level reveals similar stored items.

These properties "bring computer storage into resonance with human operators and their predilection to designate things by naming or pointing."

5.4 The Language Problem

The fundamental dissimilarity between human and computer languages "may be the most serious obstacle to true symbiosis." Licklider notes progress through interpretive programs, assembly/compiling programs like FORTRAN, and the "Information Processing Language" of Shaw, Newell, Simon, and Ellis. ALGOL and related systems show humans adapting by adopting standard formulas translatable into machine language.

For real-time cooperation, a different communication principle is needed. Instructions to computers specify precise steps and sequences; instructions to humans present goals and criteria for knowing when a task is accomplished—"instructions directed to computers specify courses; instructions directed to human beings specify goals." Humans think more naturally in goals than courses. He uses Wiener's analogy: people bound for Los Angeles "try continually to decrease the amount by which they are not yet in the smog."

Computer instruction through goal specification is being approached via two paths: (1) problem-solving, hill-climbing, self-organizing programs, and (2) real-time concatenation of preprogrammed segments and closed subroutines that the operator can call by name. The first path has shown promising exploratory work. The second, while simpler and more readily realizable, has been relatively neglected. Fredkin's trie memory provides a promising paradigm. The lack of development is understandable since "it would be unrewarding to develop the language before there are any computing machines capable of responding meaningfully to it."

5.5 Input and Output Equipment

Licklider identifies input/output as the least advanced area for symbiosis. Generally available computers offer "almost no provision for any more effective, immediate man-machine communication than can be achieved with an electric typewriter." Displays are in a better state than controls—many computers plot graphs on oscilloscopes, and some use charactron display tubes. But nowhere is there "anything approaching the flexibility and convenience of the pencil and doodle pad."

He discusses three areas:

1) Desk-Surface Display and Control: The human and computer need to draw graphs and pictures and write notes and equations on the same display surface. A human should be able to present a function roughly by drawing a graph; the computer should read clear block capitals and post interpreted characters in precise type-face. The operator could compose instructions and subroutines, define new symbols (as Gilmore and Savell did at Lincoln Laboratory), sketch table formats for the computer to precision-shape, correct data, and instruct via flow diagrams—interacting much as with another engineer who is also "a precise draftsman, a lightning calculator, a mnemonic wizard, and many other valuable partners all in one."

2) Computer-Posted Wall Display: In systems where multiple people share control of interacting vehicles, some information must be presented simultaneously on a common grid, while other information is relevant only to specific operators. Manual plotting is too slow; the computer must post the display. He notes this is already a critical problem that will worsen. Designers believe displays with desired characteristics could use flashing lights and time-sharing viewing screens based on the light-valve principle.

Individual display-control units should supplement the large display, letting operators modify the wall display from their locations and perhaps communicate with the computer through them. Laboratory experiments suggest "informal, parallel arrangements of operators, coordinating their activities through reference to a large situation display, have important advantages" over individual-console arrangements.

3) Automatic Speech Production and Recognition: Licklider examines the desirability and feasibility of speech communication with computers. Engineers take a conservative view on desirability; speech recognition specialists are conservative about feasibility. Yet interest persists because "one can hardly take a military commander or a corporation president away from his work to teach him to type." For top-level decision makers, communication via the most natural means may be worthwhile even at considerable cost.

He suggests a corporation president would likely use a symbiotic association "only as an avocation" since business situations move slowly enough for briefings. Computer specialists would likely be the direct computer interactors in business.

A military commander, however, faces critical decisions in short intervals. Licklider warns against overdramatizing "the ten-minute war" but notes it would be dangerous to assume more than ten minutes for critical decisions. As military systems grow in complexity, a real requirement for speech production and recognition will likely develop.

On feasibility: speech production poses less severe technical problems. He cites examples including a digital voltmeter that reads aloud, and work at Bell Labs, the Royal Institute of Technology (Stockholm), the Signals Research and Development Establishment (Christchurch), Haskins Laboratory, and MIT by Dunn, Fant, Lawrence, Cooper, Stevens, and others. Recent Haskins work produced "a digital code, suitable for use by computing machines, that makes an automatic voice utter intelligible connected discourse."

Speech recognition feasibility depends on vocabulary size and diversity of talkers and accents. Ninety-eight percent correct recognition of spoken decimal digits was demonstrated years earlier at Bell Labs and Lincoln Lab. He suggests a recognizer for clearly spoken alphanumeric characters could almost surely be developed from existing knowledge, which would be "a convenient tool in almost any computer installation" since untrained operators can read at least as fast as trained ones type.

For truly symbiotic real-time interaction, a vocabulary of roughly 2,000 words (1,000 basic English plus 1,000 technical terms) would likely be required—a challenging problem not currently achievable. Several organizations would undertake the development on a five-year basis, provided speech is in clear dictation style without unusual accent.

He notes that computing machines themselves play a dominant role in developing automatic speech recognizers. Two to three years earlier, recognition of sizeable vocabularies seemed ten to fifteen years away. Now, many see a prospect of accelerating knowledge acquisition through computer processing of speech signals, with sophisticated programs potentially performing well as speech-pattern recognizers even without extensive substantive knowledge. Combining these considerations brings the estimate down to perhaps five years.

References

The paper includes 27 references spanning computer chess, pattern recognition, speech synthesis and recognition, theorem proving, self-organizing systems, trie memory, learning machines, and the "general problem solver" of Newell, Simon, and Shaw.

This landmark paper, published in March 1960, established the conceptual foundation for interactive computing, personal computing, and the entire field of human-computer interaction, anticipating time-sharing systems, graphical user interfaces, and collaborative human-machine problem-solving by decades.