Research in Natural Language Processing

AlI research dealing with Natural Language Processing or NLP has been successful in the past, despite the fact that much of that success is unknown to the public. Many of the accomplishments are based on programming methods that can allow concepts and meaning to be translated in such a way, that a computer can use them to use language effectively. This paper analyzes the history of some of these programs and their developments, and the different focuses in NLP research. It ends by discussing some issues in having a correct simulation of human language, while proposing a slightly different focus to the research that is currently going on.

In terms of goals, there are two major paths taken by scientists and scholars in AI: the pragmatic approach and the scientific approach. The pragmatic approach has the sole objective of performing a task required by a product or a business, which eventually leads to profit. An example of this approach is GUS, a system developed by Xerox Palo Alto Research and which is designed to be an airline reservation assistant. In this area, scientists work along with businessmen to achieve intelligent programs which can have a friendly user interface, in such a way that a customer can speak to the program and request what they need, without requiring additional knowledge on the system being used. The simplification and the enhancement of performance by means of direct human-computer communication is undoubtedly one of the most common goals of language simulation, and perhaps the form that is most available to the public. While having a strong influence in media and products, these kinds of programs are the most unintelligent and the least capable of generating language.

The scientific approach has more potential than the pragmatic approach in terms of establishing effective NLP systems. What most people are not aware of is that these programs have had increasing success in research environments, but their language generation capabilities remain beyond public reach. Occasionally, some of these technologies infiltrate everyday life in the forms of computerized grammar checking, interactive help and troubleshooting tools, translator programs, and on the last levels, media constructs directed towards adolescents and youth (ie. online games with speaking characters, chatbot programs). Within the scientific approach, there are also subdivisions of language generation programs designed to perform an assortment of language-associated tasks, which are primarily focused on emulating different aspects of cognition. The subdivisions tend to separate into the different components of language generation, and are frequently studied apart from each other.

History of the Developments

At the surface level, syntax is the first and most evident feature in all language generation systems. At the same time, it is the simplest to achieve. A computer must contain a list of vocabulary words and grammatical paradigms. Given a particular ordering structure of a sentence, using variables that represent different parts of speech, the system must merely use a system of replacement in order to generate correct sentences. Some of the first successful attempts in simulating language syntax were Weizenbaum's ELIZA (1966) and Bobrow's STUDENT(1968), which had the added capability of responding to questions in correct English sentences. Both programs used heuristic rules to achieve an impressive performance in terms of language simulation. However, the domain and scope of the language in these programs was specific and narrow, in such a way that the programs were able to avoid the problems of language understanding. Any successful NLP system must, however, have an understanding of semantics.

During the early 70's an important improvement was made when Yorick Wilks created an English-French translation program that could effectively translate sentences from one language to the other, in a meaning preserving manner, through the use of what he called Preference Semantics. However, the translator program contained only capabilities of isolating and conserving different forms of meanings, but contained no real understanding. Later on, in 1975, there was a project called MARGIE developed by Roger Schank and several of his students in the Stanford AI laboratory, which gave birth to the Primitive Decomposition Hypothesis. This hypothesis declares that : "for any two sentences of identical meaning, in any language, a single underlying symbolic representation can be assigned, composed of structures encoded in terms of a relatively small set of 'primitive elements'"(Cullingford, 1986). A second hypothesis was derived with the implementation of this project. It was called the Understanding As Spontaneous Inference Hypothesis, which claimed that language understanding occurred after inferences that happen immediately after the semantic content of language input is assimilated.

Many of these breakthroughs in both syntactic and semantic computation occurred at the time when LISP was a popular programming language. LISP was based on lists and was commonly denominated the "language of artificial intelligence." There were many contributions that naturally arose from the creation of LISP. One of them was the possibility of tree structures. Not only was LISP able to categorize and organize data, but the source code itself was treated as data. In other words, LISP functions were treated as data objects and could be manipulated at runtime, allowing programs to be of unprecedented malleability and self-modifiability. Furthermore, the LISP programming language used nested objects as the main tool to separate one structure from another. This would eventually lead to the creation of Knowledge Structures (KS) which were the basis of several formal representational systems utilized by the most successful NLP system prototypes. Preceding the formalization of these representational systems, however, were several programming methods which revolutionized the cognitive handling of semantics in computerized systems. Following the conclusions of the Understanding As Spontaneous Inference Hypothesis, the Yale AI Project created the SAM (Script Applier Mechanism) in 1979. This project introduced the powerful AI concept of situational scripts. Situational scripts were programmed prototypical episodes that would allow a machine to understand such episodes if it encountered them in the future. The computer could then generate a summary of its understanding of that episode or answer questions about it, in order to demonstrate its understanding of it. Later on, an improved version of this program was able to understand newspaper stories and articles from different domains, and correctly answer a set of comprehension-based questions relating to these articles. This was a significant development in the field, which opened the possibility of several other programs to come into existence, using similar methods.

Mostly, the scriptal structures used in SAM were based on causal chains, which were organized in hierarchical fashion. This hierarchy included, as a primary node, the proposed objective or goal by the participants of the given event. Data structures within the program would be assigned to the several identifiable characters in the task, which would have a range of causal tree structures leading to the intended objective. From this framework, a computer could not only understand the logical link between events, but could predict what a particular character or entity was planning to do, the consequences of a particular event, or the means of achieving a goal. This procedure enabled a kind of goal-based cognitive language which led to two different developments; expert systems and frame systems.

Frame systems adopted sophisticated programming tools to produce a symbolic generalization of all scriptal structures. This allowed a wider variety of events to fit a single paradigm, and provided a firm base for the development of plan-based reasoning systems. These new systems caused programs to have increased understanding and deductive capabilities. Wilensky's PAM program and Ms. Malaprop were the first to introduce the concept of plan-based reasoning, but it was applied mainly by the program POLITICS, developed at Yale. This program could read newspaper headlines of foreign affairs and describe the possible motives of the participants. It could also understand the demands of certain parties and also deduce the actions most likely to be taken under a conservative or a liberal ideology. Subsequent projects included the IPP and BORIS, BORIS being one of the most ambitions projects to date, since it included data structures that made it possible to understand extremely complicated stories and situations, and functions to deduce the intended effects, the plan used, why it failed or succeeded, and ways to counteract that plan. The improved generation of these systems can be called the knowledge based processing systems. These systems are based on knowledge structures and the concept that "important processes in understanding and generation go on at a deep conceptual level, rather than, for example, at the more superficial levels of syntax and morphology." (Cullingford, 1986)

Conclusions

It seems that the main discovery that scientists in the field of AI are making are on programming techniques, and if there is any major realization to happen as a result of successful research, it would be that language is programmable. In itself, this tells us nothing. It has already been predicted that, given a wide enough knowledge base, a wide range of conceptual spaces, combined with the right linguistic and social rules, a computer theoretically can simulate language. In fact, there is more than one possible way of achieving it. "What if you could give the program every possible utterance by a human being, and tell it what to respond to it in each context?" Theoretically, it is possible. And discovering new programming techniques which could make this final goal happen is just a method to prove an existing theory. While being able to simulate language with a computer is quite an accomplishment by a programmer, it certainly does not give us insight into human beings. It does, at best, give insight on the methods which are effective and those that are not.

So, what is the whole objective of this discipline? The objective is entirely scientific, inspirational, and philosophical, driven to a large extent by curiosity and the need to accomplish new things. In many ways, this might imply that a great deal of the AI developments might remain on scientific grounds for indeterminate time, in which case most breakthroughs would not be open to the general public. Of course, there are technological advances which could be seen in the form of machines that can do a wide variety of automated tasks, and robots, which have not yet made a wide appearance.

For conventional purposes, though, any robots or "androids" that could make an appearance in the future are most likely to be used in a practical way, such that they do specified tasks or act as servants or assistants. This is where certain misconceptions occur, where an automaton could be regarded as intelligent if it can cook, say preprogrammed sentences, or say peculiar facts on demand. A robot that can follow human instruction is not intelligent, because this is what computers do. The majority of such robots will be merely computers that can do a greater variety of tasks. For all that matters, nobody wants a robot that can think, object to instructions, or make decisions based on no others' judgment. I would content myself with a robot that does my laundry without wondering about why I asked or even knowing what laundry is. The latter could happen if a robot was allowed to think in an intelligent way.

It was said before that for a machine to think intelligently it must have an understanding of concepts. This was explained through the semantics of language. However, a computer can have many representations of a concept, but it in fact does not know anything, because it hasn't seen, lived, or felt anything of things that human see, live, and feel. In fact, the computer only has a severely limited textual representation of any human concept. One of the things that semiotics tell us is that a sign, in this case a human concept, cannot be understood in any other than its original medium. For example, our understanding of abstract concepts has been achieved through several media, which include eyesight, logical deduction, emotions, chemical reactions in our body, and most importantly, our consciousness itself. By consciousness, I mean our awareness of the world, and our perspective of the world in the first person. These concepts, being in several different media which any computer is completely unable to reach, are thus completely isolated from what a computer can comprehend (this is comparable to trying to explain visual images to a blind man). Notice that even if computers had the necessary chemical reactions to produce emotion, there is no guarantee that there is "somebody in there" who feels that emotion. A computer is an object, and making an object understand something is very unlikely. The act of programming AI is, in short, attempting to make a computer appear to understand something, when in reality this is never the case.

Apart from the inability of computers to genuinely understand a concept, there lies the issue of unpredictability. Given a particular sentence from a human speaker, would a computer respond the same way each time? In many cases, this would be true. However, in the cases that this is not true, most likely the computer will have several options to respond to the given statement or question. The problem with this is that the amount of responses that the computer gives is always limited, and it is always possible to analyze. Given the correct amount of time and knowledge about the source code of a computer program, it is eventually possible to predict exactly what the computer will say in a particular concept. This is entirely untrue with human beings. In human discourse, there is an incalculable amount of factors that contribute in each instance. Conversational discourse contains factors that include mood set (which is in turn influenced by a myriad cognitive processes involving past memories, surface thoughts, prejudices and recent events or affairs going on in both local or broad psychological senses), what the person is thinking about at the moment, who the person is talking to, which are in turn influenced by stereotypes, and can have other factors such as short or long term goals regarding that person, unconscious desires of the ego, and others.

Now, a computer can be programmed to fool a person into believing that it has an understanding of these factors, and can indeed apply them in a conversation with a human, but human conversations are also about human ideas, and in this domain the computer is destined to lose. I don't mean to adopt a pessimistic attitude towards AI. If there is one conclusion I can reach is that the Extensional language can theoretically be simulated.

So far, many of the prototypes that have been created have been specifically directed at certain areas of cognition and language. Some programs were only able to read newspaper articles in certain genres, and offer only a certain kind of analysis. There are programs that are good at identifying meanings and translating sentences, but cannot analyze anything of what they read. Other languages are able to even write their own stories by means of hierarchical meaning and plot coding. In fact, it has been demonstrated that computer programs can demonstrate creativity, and in some cases, creativity that exceeds that of a human being in a particular area. There currently are programs that generate outstanding artistic pieces displayed worldwide, and there are some people who use computer programs to enhance their own creativity (Boden, 2002)

The techniques that currently have been researched regarding formal methods of representing semantics, are effective to the extent that they can translate a certain medium into another (text), and this is the only possible way for working with these media in a computational way. However, in terms of the research oriented in the computation based on conceptual spaces and semantics, I think it has a lot of potential for creating more order in society, in the sense that an "automated policeman" could enforce the law, for example. However, if the objective is merely curiosity and trying to completely simulate language without a direct application for society, I think the focus of that research should be modified.

For purposes of simulating language for the sake of simulation, I propose focusing on the factors which are entirely human, such as emotion, and consciousness itself. Researching a possible simulation of consciousness is more meaningful than the current Extensional approach to simulating language. How can instincts be simulated? I believe instincts, the psychological makeup of a person, the emotions that the person is capable of perceiving, along with how the person experiences life, are the most determinant factors of behavior. And behavior is the determinant factor when analyzing if a computer behaves like a human or not, even if it is in text-speech. I don't believe the understanding of specified concepts (such as the newspaper-analyzing programs) can lead to a computer that is more human. A general and less performance-guided approach could yield more enriching results, even if they were not as apparent to the eye. But then again, as David Gelernter said in a debate at the recent MIT conference on artificial intelligence, "You can simulate a rainstorm, but you won't get wet," nobody should forget that no matter how complex, all you are dealing with is a simulation.

References

Abdi, H. (1999). Neural networks. Thousand Oaks, CA: Sage Publications.

Bates, M. (1993). Challenges in natural language processing . Cambridge, MA: MIT Press.

Boden, M. A. (2004). The creative mind : myths and mechanisms . New York: Routledge.

Bringsjord, S. (2000). Artificial intelligence and literary creativity [computer file] : inside the mind of BRUTUS, a storytelling machine . NJ: Erlbaum Associates.

Cullingford, R. E.(1986). Natural language processing. Rowman & Littlefield, GA.

Gardenfors, P. (2000). Conceptual spaces [computer file] : the geometry of thought. Cambridge, MA: MIT Press.

Kurzweil, R. (2002). Are we spiritual machines? : Ray Kurzweil vs. the critics of strong AI . Seattle, WA: Discovery Institute Press.

Pereira, F. (1994). Natural language processing. Cambridge, MA: MIT Press.

Potts, T. C. (1994). Structures and categories for the representation of meaning. Cambridge University Press.

Ristad, E. S. (1993). The language complexity game . Cambridge, MA: MIT Press.

Springer, (2006). Research and development in intelligent systems XXII : proceedings of AI-2005, the Twenty-Fifth SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge, UK, December 2005 . Cambridge.