conanxin

Abstract

Human-computer symbiosis is an anticipated development of cooperation and interaction between humans and electronic computers. This will involve a very close coupling between humans and electronic devices. The main objectives are 1) to enable computers to facilitate formalized thinking, as they currently assist in solving formalized problems; 2) to allow humans and computers to collaborate in decision-making and controlling complex situations without relying on predetermined programs. In the anticipated symbiotic partnership, humans will set goals, formulate hypotheses, establish criteria, and conduct evaluations. Computers will perform routine tasks to prepare humans for insights and decisions in technical and scientific thinking. Preliminary analysis indicates that the symbiotic partnership will be more effective in intellectual activities than individuals working alone. Prerequisites for achieving effective collaborative relationships include the development of time-sharing computers, memory components, memory organization, programming languages, and input and output devices.

1 Introduction

1.1 Symbiosis

Only the fig wasp (Blastophaga grossorun) can help fig trees complete pollination. The larvae of this insect live in the ovaries of the fig tree, where they also find food. In this way, fig trees and fig wasps are severely dependent on each other: without fig wasps, fig trees cannot bear fruit; without fig trees, fig wasps cannot obtain food. The combination of both not only allows for mutual survival but also creates a highly productive and vibrant cooperative relationship. "Two different organisms live together in intimate cooperation, even forming a close alliance," this mode of cooperation is called symbiosis.

Human-computer symbiosis is a subclass of human-machine systems. There are many human-machine systems. However, there are currently no human-computer symbionts. The purpose of this paper is to propose this concept and to draw attention to the applicable principles of human-computer engineering by analyzing some issues in human-computer interaction, and to point out some questions that need to be researched and answered, thereby promoting the development of human-computer symbiosis. We hope that, in not too long a time, the human brain and computers will be closely integrated, resulting in a partnership that will be considered as having capabilities that no human brain can think and process data in ways that today's information processing machines have not achieved.

1.2 Between "Machine-Enhanced Humans" and "Artificial Intelligence"

As a concept, human-computer symbiosis differs in an important way from what North refers to as "machine-enhanced humans." In past human-machine systems, the operator held the initiative, provided direction, integrated, and established standards. The mechanical parts of the system were primarily extensions of the human arm and then the eye. These systems were certainly not composed of "different organisms living together." There was only one organism—humans—while the rest were merely aids to that person.

In a sense, any artificial system is designed to assist humans, helping one or more individuals outside the system. However, if we focus on the operators within the system, we find that there have been significant changes in certain technical fields over the past few years. "Machine enhancement" has replaced humans, shifting towards automation, leaving humans more as helpers rather than being helped. In some cases, particularly in large information and control systems centered around computers, human operators are primarily responsible for functions where automation is not feasible. Such systems (which North might call "human-enhanced machines") are not symbiotic systems. They are "semi-automated" systems, initially designed to be fully automated but failing to achieve their goals.

Human-computer symbiosis may not be the ultimate paradigm of complex technical systems. At the appropriate time, electronic or chemical "machines" seem entirely capable of surpassing the human brain in most functions we currently consider. Even now, the progress of Gelernter's IBM - 704 plane geometry theorem proving program is comparable to that of a Brooklyn high school student, making similar mistakes. In fact, there are several theoretical proof, problem-solving, chess, and pattern recognition programs that can match human intellectual performance in restricted domains; while Newell, Simon, and Shaw's "General Problem Solver" may eliminate some limitations. In short, it seems worthwhile to avoid arguing with (other) artificial intelligence enthusiasts who believe that only machines will dominate in the distant future. However, in the meantime, the main intellectual advances will be made by closely collaborating humans and computers, which will be a rather long transitional period. A multidisciplinary research group studying the future research and development issues of the Air Force estimated that by 1980, the development of artificial intelligence would enable machines to think or solve military-relevant problems independently. This would lead, for example, to 5 years of development for human-computer symbiosis and 15 years of usage. Those 15 years could be 10 years or 500 years, but those years should be among the most creative and exciting periods in human history.

2 Goals of Human-Computer Symbiosis

Today's computers are primarily designed to solve predetermined problems or process data according to predetermined programs. The computational process may depend on the results obtained during the computation, but all alternatives must be anticipated in advance. (If unforeseen alternatives arise, the entire process halts, waiting for the necessary extensions of the program.) The requirement for predetermined or pre-established criteria sometimes has no significant drawbacks. It is often said that programming computers forces people to think clearly, standardizing the thinking process. If users can think through their problems in advance, then the symbiotic relationship with computers becomes unnecessary.

However, many problems... are difficult to think through in advance; recall the earlier description of emerging systems. If problems can be solved more quickly and effectively through collaboration with computers, guided by intuition and trial and error, exposing errors in reasoning or revealing unexpected twists in solutions, then problems can be resolved better. Without computer assistance, some problems cannot be solved at all. Poincaré foresaw the frustration of a significant number of potential computer users when he said, "The question is not, what is the answer? The question is, what is the problem?" One of the main purposes of human-computer symbiosis is to effectively incorporate computers into the formalized aspects of technical problems.

Another main goal is closely related. It is to effectively bring computers into the thinking process that must occur "in real-time," where the pace is too fast to allow for traditional computer usage. Imagine, for example, trying to command a battle with the help of a computer on such a timeline. You pose your question today. Tomorrow you spend time with a programmer. You receive a 20-foot-long paper filled with numbers that do not provide a final solution but propose a strategy that should be explored through simulation. Clearly, the battle will end before the second step of its plan begins. The way of thinking while interacting with a computer is the same as how you would interact with a colleague whose abilities complement your own, which will require a much closer coupling between humans and machines than this example suggests and the current situation allows.

3 The Need for Computer Participation in Formalized and Real-Time Thinking

The preceding paragraphs assume that if data processing machines can be effectively introduced into the thought process, the functions they can perform will significantly improve or facilitate thinking and problem-solving. This assumption may require justification.

3.1 Preliminary and Informal Ergonomic Analysis of Technical Thinking

Despite the vast literature on thinking and problem-solving, including numerous historical case studies of the invention process, I find nothing better than conducting ergonomic studies of the mental labor of people engaged in technological enterprises. Therefore, in the spring and summer of 1957, I attempted to record what a moderately technical person did during the time he believed he was focused on work. Although I realized the sampling was insufficient, I still made my research subject. It was clear that my main activity was record-keeping; if I preserved records according to the details envisioned in the original plan, the project would turn into an infinite regression. It did not. Nevertheless, I obtained a snapshot of activities that made me stop. Perhaps my scope is atypical—I hope not, but I fear it is.

I spent 85% of my "thinking" time on thinking, decision-making, and learning things I needed to know. The time spent searching for or acquiring information far exceeded the time spent digesting information. Several hours were spent drawing charts, and other hours were spent instructing assistants on how to draw charts. Once the charts were completed, two relationships became immediately apparent, but drawing was necessary to make them a reality. At one point, it was necessary to compare the six experimental measurements of voice clarity and voice noise ratio. No two experimenters used the same definition or measurement of voice noise ratio. It took several hours of computation to turn the data into a comparable form. When they were in a comparable form, I spent only a few seconds determining what I needed to know.

In short, throughout the study, my "thinking" time was primarily spent on activities that were essentially clerical or mechanical: searching, calculating, drawing, transforming, determining a set of hypotheses or the logical or dynamic consequences of hypotheses, paving the way for decisions or insights. Furthermore, my choices about what to try and what not to try were largely based on considerations of clerical feasibility rather than intellectual capability, which is embarrassing.

The main suggestion conveyed by the research results just described is that most of the operations referred to as technical thinking are operations that machines can perform more effectively than humans. These operations must be carried out in an unpredictable and constantly changing order across different variables, which presents serious challenges. However, if these challenges can be addressed in a way that establishes a symbiotic relationship between humans and fast information retrieval and data processing machines, then collaborative interaction will clearly improve the thinking process significantly.

At this point, it may be worth acknowledging that we are using the term "computer" to encompass a variety of computing, data processing, and information storage and retrieval machines. The capabilities of such machines are increasing almost daily. Therefore, making general statements about the functions of this class is perilous. Perhaps making general statements about human capabilities is equally perilous. Nevertheless, certain genotype differences in capabilities between humans and computers are indeed striking, and they have implications for the nature of possible human-computer symbiosis and the potential value of achieving such symbiosis.

As has been said in various ways, humans are noisy narrowband devices, but their nervous systems have many parallel channels that are simultaneously active. In contrast to humans, computers are extremely fast and precise, but they can only perform one or a few basic operations at a time. Humans are flexible, able to "continuously self-plan" based on newly received information. Computers are rigid, constrained by their "pre-programmed" nature. Humans naturally speak a redundant language, organized around single objects and coherent actions, using 20 to 60 basic symbols. Computers "naturally" speak a non-redundant language, typically with only two basic symbols, lacking inherent appreciation for single objects or coherent actions.

To be strictly correct, these characteristics must include many qualifiers. Nevertheless, the differences they present (and thus potential complementarity) are fundamentally accurate. Computers can easily, well, and quickly perform many tasks that are difficult or impossible for humans, while humans can easily and well perform many tasks that are difficult or impossible for computers, albeit not very quickly. This suggests that symbiotic cooperation, if it successfully integrates the positive features of both humans and computers, will have immense value. Of course, the differences in speed and language present challenges that must be overcome.

4 The Separability of Functions of Humans and Computers in the Anticipated Symbiotic Relationship

It seems that the contributions of human operators and devices in many operations will be so completely integrated that it will be difficult to neatly separate them in analysis. This is the case; for example, when collecting data to support decisions, both humans and computers find relevant precedents from experience, if the computer subsequently proposes an action plan that aligns with human intuitive judgment. (In theorem proving programs, computers find precedents in experience; in the SAGE system, they propose action plans. The above is not a far-fetched example.) However, in other actions, the contributions of personnel and devices are somewhat separable.

Certainly, at least in the early stages, humans will set goals and provide motivation. They will formulate hypotheses. They will ask questions. They will think of mechanisms, programs, and models. They will remember that such individuals did some potentially relevant work on a topic of interest as early as 1947, or at least shortly after World War II, and they will know which journals that topic might be published in. Overall, they will make approximate, erroneous, but leading contributions; they will define standards and serve as evaluators, judging the contributions of the devices and guiding the overall thinking.

Moreover, when such situations do arise, humans will handle extremely low-probability situations. (In current human-machine systems, this is one of the most important functions of the operator. The sum of the probabilities of extremely low-probability alternatives is often too large to ignore.) When computers lack applicable patterns or programs for specific environments, humans will fill in the gaps in problem-solving or computer programming.

The information processing devices themselves will convert hypotheses into testable models and then test the models based on data (the operator can roughly specify this data and determine its relevance when the computer submits it for his approval). These devices will answer questions. They will simulate mechanisms and models, execute programs, and display results to the operator. They will transform data, draw charts (in any way "cutting the cake" as specified by the human operator, or if the human operator is uncertain about what he wants, present several alternatives). The devices will insert, infer, and transform. They will convert static equations or logical statements into dynamic models for the operator to examine their behavior. Generally, they will perform routine clerical work to fill the gaps between decisions.

Additionally, as long as there is sufficient foundational support for formal statistical analysis, computers will act as machines for statistical inference, decision theory, or game theory, conducting preliminary evaluations of proposed action plans. Finally, they will conduct as many diagnostics, pattern matches, and correlation identifications as possible, but in these areas, they will accept a clearly subordinate position.

5 Prerequisites for Achieving Human-Computer Symbiosis

In the previous section, it was assumed that data processing devices are unavailable. Computer programs have not yet been written. In fact, there are several obstacles between the current non-symbiotic state and the anticipated symbiotic future. Let us examine some of these obstacles to gain a clearer understanding of what is needed and the feasibility of achieving this goal.

5.1 Speed Mismatch Between Humans and Computers

Current large computers are too fast and costly for real-time collaborative thinking with a human. Clearly, for efficiency and economy, computers must allocate time among many users. Time-sharing systems are currently under active development. There are even arrangements to prevent users from "disrupting" anything other than their personal programs.

In a period of 10 or 15 years, envisioning a "thinking center" seems reasonable, combining the functions of today's libraries with the anticipated advancements in information storage and retrieval and the symbiotic functions suggested earlier in this paper. This vision can easily be scaled into a network of such centers interconnected through broadband communication lines and connected to individual users via leased line services. In such a system, the speed of computers would be balanced, and the costs of massive storage and complex programs would be divided by the number of users.

5.2 Memory Hardware Requirements

When we begin to consider storing any known technical literature in computer memory, we encounter billions of bits of data that will cost billions of dollars unless significant changes occur.

The first thing to face is that we will not store all technical and scientific papers in computer memory. We may store the most concise summaries—the quantitative parts and references—but not everything. Books are one of the most exquisite and humanized components that exist, and in the context of human-computer symbiosis, they will continue to play an important role. (Hopefully, computers will expedite the searching, delivery, and return of books.)

The second point is that a very important part of memory will be permanent: part will be non-erasable memory and part will be published memory. Computers will be able to write to non-erasable memory once and read it indefinitely, but they will not be able to erase non-erasable memory. (It may also rewrite, turning all 0s into 1s, marking what has been written before.) Published memory will be "read-only" memory. It will be introduced into computers that have already been built. Computers will be able to reference it repeatedly but cannot change it. As computers grow larger, these types of memory will become increasingly important. They can be more compact and much cheaper than core, thin-film, or even tape storage. The main engineering problem will involve selecting circuits.

Regarding other aspects of memory requirements, we can expect the continued development of ordinary scientific and commercial computers. Storage elements may become as fast as processing (logic) elements. This development will have revolutionary implications for computer design.

5.3 Storage Organization Requirements

The concept of human-computer symbiosis implies that information can be retrieved by name and pattern and accessed through programs that are much faster than serial searches. At least half of the memory organization problems seem to exist in the storage process. The rest seem to be contained within the pattern recognition problems in the storage mechanisms or media. A detailed discussion of these issues exceeds the current scope. However, a brief overview of a promising idea, "trie storage," may help illustrate the general nature of anticipated developments.

Trie storage, as termed by its founder Fredkin, is designed to facilitate information retrieval, and the branching storage structure resembles a tree during development. Most common memory systems store the functions of parameters at specified locations. (In a sense, they do not store these parameters at all. In another, more realistic sense, they store all possible parameters within the framework structure of memory.) In contrast, trie storage systems store functions and parameters. Starting from standard initial registers, parameters are introduced into memory one character at a time. Each parameter register has a cell, and each character has a cell (for example, two for binary information), with each character cell having storage space for the address of the next register. This parameter is stored by writing a series of addresses, each telling where to find the next address. At the end of the arguments is a special "end parameter" marker. Following this is the function's instruction, which is stored in one way or another, with further trie structures or "list structures" typically being the most efficient.

Trie storage schemes are inefficient for small memory, but as memory size increases, they become increasingly efficient in utilizing available storage space. The appealing characteristics of this scheme are: 1) the retrieval process is extremely simple. Given a parameter, input the first character into the standard initial register and extract the address of the second character. Then go to the second register to obtain the address of the third register, and so on. 2) If two parameters have the same initial character, they use the same storage space for those characters. 3) The lengths of parameters do not need to be the same and do not need to be specified in advance. 4) No parameter will be retained or use storage space before actual storage. The trie structure is created when items are introduced into memory. 5) One function can serve as a parameter for another function, which can serve as a parameter for the next function. Thus, for example, by inputting the parameter "matrix multiplication," one can retrieve the entire program for performing matrix multiplication on the computer. 6) By examining the storage at a given level, one can determine what similar items have been stored so far. For example, if there is no reference to Egan, J. P., it would only take one or two steps to find traces of Egan James...

The properties just described do not encompass all desired attributes, but they resonate with human operators, who tend to specify things through naming or pointing.

5.4 Language Issues

The fundamental differences between human language and computer language may be the most serious obstacle to true symbiosis. However, it is reassuring that significant progress has been made in adapting computers to human language forms through interpretive programs, especially through assembly or compilation programs like FORTRAN. Shaw, Newell, Simon, and Ellis's "Information Processing Language" represents another form of reconciliation. Moreover, in ALGOL and related systems, flexibility has been demonstrated by adopting standard formulas that can be easily translated into machine language.

However, to achieve real-time collaboration between humans and computers, it is necessary to utilize a rather different set of communication and control principles. This idea can be highlighted by comparing instructions typically directed at intelligent humans with those usually directed at computers. The latter precisely specify the individual steps to be taken and the order in which to take them. The former propose or imply something about incentives or motivations, providing a standard by which the executor of the instructions will know when to complete the task. In short: instructions for computers specify routes; instructions for humans specify goals.

Humans seem to think more naturally and easily about goals than about routes. Indeed, they often know some information about travel or work routes, but few can start from a precisely formulated itinerary. For example, who would set out from Boston to Los Angeles with detailed route instructions? Instead, in Wiener’s words, the person heading to Los Angeles tries to minimize the degree to which they have not yet been shrouded in smoke.

There are two avenues for implementing computer instructions. The first involves problem-solving, hill-climbing algorithms, and self-organizing projects. The second involves real-time chaining of pre-programmed segments and closed subroutines, which operators can simply specify and call by name.

Along the first avenue, promising exploratory work has already been done. It is clear that working under the loose constraints of a predetermined strategy, computers will be able to design and simplify their programs to achieve established goals at appropriate times. So far, these achievements are not significant; they are merely "demonstrations in principle." However, their implications are profound.

Although the second avenue is simpler and clearly realizable sooner, it has been relatively overlooked. Fredkin's trie storage provides a promising example. We may see, at the appropriate time, a serious effort to develop computer programs that can connect like words and phrases in a language, enabling any computation or control. Clearly, considerations hindering this effort are that such efforts will not yield anything of significant value in the existing computer environment. Developing a language is undesirable until any computer can respond meaningfully to it.

5.5 Input and Output Devices

In terms of the requirements for human-computer symbiosis, the least advanced data processing sector seems to be the one dealing with input and output devices, or from the operator's perspective, the one dealing with display and control. Having said that, it is necessary to make qualifying comments, as the engineering of devices for high-speed input and extraction of information has been excellent, and some very sophisticated display and control technologies have been developed in research laboratories like Lincoln Laboratory. However, overall, in generally available computers, there is hardly anything more effective and immediate for human-computer communication than an electric typewriter.

Displays seem to fare somewhat better than control. Many computers draw graphics on oscilloscope screens, and a few computers utilize the superior graphic and symbolic capabilities of character display tubes. However, to my knowledge, in technical discussions, nothing comes close to the flexibility and convenience of a pencil and doodle board, or the chalk and blackboard used by people.

1. Desktop Display and Control: Certainly, for effective human-computer interaction, both humans and computers need to draw graphics and pictures on the same display surface and write annotations and equations on the same display surface. This person should be able to present a function to the computer by drawing a chart in a rough but quick manner. The computer should read the person's handwriting, perhaps under the condition of clear uppercase letters, and should immediately post the corresponding characters at each hand-drawn symbol's location, translating them into precise fonts. With such input-output devices, operators will quickly learn to write or print in a machine-readable way. They can write instructions and subroutines, format them appropriately, and check them before finally introducing them into the computer's main memory. They could even define new symbols, as Gilmore and Savell did at Lincoln Laboratory, and present them directly to the computer. They could roughly sketch the format of a table and then let the computer shape it precisely. They could correct the computer's data, guide the machine through flowcharts, and interact just as they typically would with other engineers, except that the "other engineers" would be precise draftsmen, fast calculators, mnemonic guides, and many other valuable partners.

2. Computer-Published Wall Displays: In some technical systems, several individuals share responsibility for controlling vehicles whose behaviors mutually influence each other. Some information must be presented to everyone simultaneously, preferably on a common grid, to coordinate their actions. Other information is relevant only to one or two operators. If all information is presented on one display to everyone, it will only create incomprehensible chaos. This information must be published by the computer, as manual drawing is too slow to keep up to date.

The issues just outlined are now even a critical issue, and over time, they seem certain to become increasingly critical. Some designers believe that displays with the desired characteristics can be constructed based on the principle of light valves, using pulsed light sources and time-sharing viewing screens.

Most people who have thought about this issue believe that large displays should be supplemented by separate display control units. The latter would allow operators to modify wall displays without leaving their positions. For certain purposes, it is hoped that operators will be able to communicate with the computer through auxiliary displays or even wall displays. At least one proposal for providing this communication seems feasible.

Of course, large wall displays and their associated systems are related to the symbiotic cooperation between computers and a group of people. Laboratory experiments have repeatedly shown that informal parallel arrangements among operators, coordinating their activities by referring to large location displays, have significant advantages over more widely used arrangements that position operators at various consoles and attempt to associate their actions through computer agents. This is one of several operational team issues that require careful study.

3. Automatic Speech Generation and Recognition: How ideal and feasible is speech communication between human operators and computers? This complex question arises whenever complex data processing systems are discussed. Engineers who work and live with computers tend to be conservative about this desire. Engineers experienced in automatic speech recognition are cautious about its feasibility. However, there remains interest in the idea of conversing with computers. To a large extent, this interest stems from the recognition that it is difficult to pull a military commander or corporate president away from their work to teach them to type. If computers could be directly used by high-level decision-makers, then providing communication in the most natural way might be worthwhile, even at considerable cost.

Preliminary analysis of the questions and time scales for corporate presidents suggests that they are only interested in a symbiotic relationship with computers as a hobby. Business situations typically progress slowly enough to allow time for briefings and meetings. Thus, it seems reasonable for computer experts to interact directly with computers in business offices.

On the other hand, military commanders are more likely to make critical decisions in a short time frame. It is easy to exaggerate the concept of a ten-minute war, but it is dangerous to expect more than ten minutes to make critical decisions. Therefore, as the capabilities and complexities of military systems' ground environments and control centers grow, the genuine need for automatic speech generation and recognition by computers seems likely to develop. Of course, if devices have already been developed, are reliable, and are available, they will be used.

In terms of feasibility, the technical issues posed by speech generation are less severe than those posed by automatic speech recognition. A commercial electronic digital voltmeter now loudly reads its indications one digit at a time. Over the past eight or ten years, Bell Telephone Laboratories, the Royal Institute of Technology (Stockholm), Signals Research and Development Establishment (Christchurch), Yale University's Haskins Laboratories, and MIT, along with Dunn, Fant, Lawrence, Cooper, Stevens, and their colleagues, have demonstrated generation after generation of comprehensible automatic generators. Research at Haskins Laboratories has developed a digital code suitable for computer use, enabling automatic speech to be fully understandable in relevant discourse.

The feasibility of automatic speech recognition largely depends on the vocabulary of words to be recognized and the diversity of speakers and accents. A few years ago, at Bell Telephone Laboratories and Lincoln Laboratory, it was demonstrated that 98% correct recognition of natural decimal digits was achievable. To further expand the vocabulary, it can be said that it is now almost certain that an automatic recognizer for clearly pronounced alphanumeric characters can be developed based on existing knowledge. Since untrained operators read at least as fast as trained operators type, such a device could be used in almost any computer installation.

However, to engage in real-time interaction at a truly symbiotic level, a vocabulary of about 2000 words may be required, such as 1000 basic English words and 1000 technical terms. This is a challenging problem. There is a consensus among acoustic experts and linguists that establishing a recognizer for 2000 words cannot yet be accomplished. However, several organizations are willing to commit to developing an automatic recognition system for such a vocabulary within five years. They would stipulate that speech must be clear, with a dictation style and no unusual accents.

While a detailed discussion of automatic speech recognition technology exceeds the current scope, it is worth noting that computers play a dominant role in the development of automatic speech recognizers. They provide the impetus for current optimism, or rather, for the optimism of some individuals at present. Two or three years ago, it seemed that automatic recognition of a large vocabulary would not be achievable within 10 or 15 years; it would have to wait for the gradual accumulation of knowledge about the acoustic, vocal, linguistic, and psychological processes involved in speech communication. However, now many see the prospect of accelerating the acquisition of this knowledge through computer processing of speech signals, and many workers believe that even without substantial knowledge of the acoustic signals and processes, complex computer programs can perform excellently in speech pattern recognition. Combining these two considerations can reduce the estimated time required to achieve practical speech recognition to five years, which is the five years just mentioned.