The Discipline
of Interdisciplinarity.
Introduction
to The Semiotics of the Visual - On Defining the Field.
A contradiction (it would sound better if I called it a paradox) dominates
the definition of the subject: Since semiotics is an integrative approach, to
section off a particular area of perception and declare it the valid object of
semiotic research means to deny semiotics its essential quality of
interdisciplinarity. Therefore, from an epistemological perspective, the
semiotics of the visual is not possible, and the attempt to define its
boundaries is alien to the nature of the concepts and methods of semiotics. One
more argument against the subject: Various pre-semiotic theories followed a
common pattern. That is, they attempted to apply concepts of great generality
(such as form and content, space and time, and more recently, structure and
function) to the main spheres of sensorial perception, sometimes producing pretentious
statements or apparent laws. (It is sufficient to cite here Fechner’s law
concerning the auditory, or the laws referring to color in respect to visual
phenomena.)
Semiotics
has not put any of these laws under discussion but has questioned the legitimacy
of specializing knowledge, of introducing distinctions that only a
gnoseologically rigid, clear-cut model of sensorial perception founded on the
psychological model sustains. Obviously, the ear does not see in the way the
eyes do as a sight organ; neither do fingertips hear; nor does the nose feel,
etc. But the fact is that each time a sign is perceived, a semiosis begins in
which the ‘absent’ accompanies the present. In other words, the sign presents
sets of complex mechanisms--the mechanism of memory in the first place--that
restore the syncretism of the real. The dominance of one sensation (visual,
auditory, olfactory, etc.) cannot be denied, but not to the extent of
eliminating all others or especially their integration in the whole idea or
feeling or both together. Semiotics does not introduce an integrating principle
from the sphere of the known into the sphere of the cognizable, but starts out
from the pragmatic observation according to which everything is in motion and
everything is interaction.
The
interdisciplinarity of semiotics is thus an epistemological condition deriving
from the need to consider interaction in its complexity. Semiotics proposes a
heterogeneous, instead of homogeneous, model and tries to describe it not
through reductions but through eliminating them. In the case mentioned above,
the elimination is of psychological reductions; but elimination is valid for
sociological, ideological, historical, etc. reductionist models.
Considering semiotics as the theory and practice of mediations
(Nadin 1981), I naturally had reservations when initiating semiotic
research in the visual. Although my research follows the line of Peirce’s
semiotic, it integrates results from Saussure’s pre-semiotic, structuralist
model and from his successors, especially the French semiologues. No matter how
much we think we can determine our own existence, there comes a moment--such as
the one when I started my work in America—when certain adaptations to reality
are necessary. Working at an excellent school, whose object is design (which
many in and outside the school consider a visual domain par excellence),
I came to understand that the chance to verify my own model was given a
rebours. Either the visual could be separated from the continuum of the
semiotic field - and in this case, semiotics would follow its traditional
course of specializations, denying its universality as envisioned by Morris -
or this separation would be impossible and research would be justified through
its negative results (in the Hegelian sense of negation). These appeared to be
the logical extremes.
I was still unaware of the fact that semiotics, in the sense founded by
Peirce, transcends the dualistic model, that a third is possible, and that this
third is itself a semiotic result: in the sphere of the visual, the nonvisual
whole is rediscovered. Different hierarchies between the visual and nonvisual
proposed in various cultural contexts or anthropological models correspond to
semioses dictated by pragmatic reasoning. The written word exercises a social
action of stabilization and simultaneously evidences an important political
function. Under conditions in which media are diversified, the word’s role
changes; images less associated with language play a more and more important
role. The written word blocks interaction but invites interpretation in time.
The image, transmitted with the aid of communication systems supported by
computer technology, reduces interpretivity but permits interaction in a way
never before attempted or utilized.
The fact
that the visual, like all other components of the semiotic field, is a
crossroads for all that the visual is not does not automatically mean that
specialization is reconfirmed as a paradigm of knowledge. The way in which the
nonvisual is known and understood from the perspective of the visual is
different from the way verbal language is known and understood by linguists, or
space by a geometrist, color by physicists, texture by chemists, etc.
Mediations in the field of the visual are nonvisual by implication: word
associations, olfactory, tactile, or other associations. Interpretation of the
sign by sign--in a process in which we ourselves become, in the act of
interpretation, signs and elements of mediations--brings the concrete visual
back in the universal perspective.
These methodological observations are intended only to explain the
general framework in which the articles that follow were conceived, the
framework of research, more precisely. It was inevitable that each author start
out from his or her specialty: linguistics, design, art history, philosophy.
The result is a departure point. The research, whose results are partially
presented in these articles, does not propose to negate semiotics from within
by application contrary to its nature, but to establish what happens when for
methodological or other reasons (such as those acting in a modern society that
is extremely fragmented due to specialization) one attempts to section the
object of study of semiotics.
Anticipating
several results to be presented herein, I can say that they converge toward the
conclusion that the visual, as an integral part of the general semiotic field,
evidences ever-greater importance in our time. The continuous deterioration of
language--which many, confusing cause and effect, still associate with
low-quality education--has as one of its necessary results the transition from
word-dominated to image-dominated communication. It would be excessive to enter
here into the details of the semiotic processes that mark the transition from
the civilization of literacy to what I call ‘the civilization of illiteracy’
(Nadin 1983). The social division of labor is only one of the factors
that need to be considered, and specialization (which the criterion of
productivity makes necessary) is a consequence of labor division. Thus in the
field of the visual, new specializations lead to segmentation that is deeper,
harder to overcome, making the integrating procedure of semiotics all the more
necessary--but all the more difficult.
Actually,
nothing would justify this entire project if not the facts that professionals
in the visual--graphic designers, architects, film/video artists, newspeople,
painters, computer graphics professionals, and others--sense the fragmentation
of their specialties and that mediation through visual signs often escapes
their control. Obviously, the solution is not the return to pictographic
culture or mytho-magical images, but the integration of complementary
perspectives, such as those of Western and Far Eastern cultures. In practice,
the techno-scientific activities in the two cultures often meet and continue to
influence each other more than their competition in the marketplace leads us to
think.
Working on the language of television (a nontraditional area for an art
historian), Gregor Goethals attempts to see to what extent semiotic concepts
are merely a new name given to traditional concepts in art history or if the
former represent a means of investigation and evaluation better adapted to her
object of study. It is easy to understand her insistence on the category of function,
which she applies almost in the structuralist sense used by Mukarovský,
while aiming to place the entire discussion in the semiotic perspective.
(Peirce is invoked as a terminological orientation point.) From its beginning,
television has had problems with formal categories. After McLuhan, it became
quite evident that this new medium presupposes its own evaluation criteria, and
that the influence it exercises would be extended to manifestations that are
not telegenic through their own nature. Politics discovered television’s
mediating (i.e., semiotic) action before semioticians determined the rules to
be applied. Electoral campaigns have been decided through television. Facts and
events are hidden or omitted. On the other hand, facts and events occurring
outside our direct sphere of action become familiar. As a medium, television
has assumed new aesthetic functions and will exercise direct influence on our
future life and work in its relation with computers (image manipulation,
retrieval from immense data bases). Although Gregor Goethals limits her study
to the relationship between the semiotic tactics of religious and political
‘communicators’, the possible implications in other spheres are easy to
discover and consider.
Having preoccupied herself for several years with the aspects of written
language, Naomi Baron extends the semiologic model of the sign, also
introducing the semiotic function but from a perspective different from that of
Gregor Goethals. She distinguishes between various levels of representation
while pursuing her argument, according to which iconicity is not a property of
the sign itself but a relational concept. She states: ‘Iconicity in any system
of representation--be it language or art--can only be defined modulo
another variable: the people producing or perceiving the sign.’ Baron’s
definition of representation is very encompassing. The bottom line is: ‘Words
in human languages represent experience.’ Since the definition is so general,
she must keep under control the concepts used in discussing differences between
various forms of representation. In future attempts to use the components she
identifies in representation (content, shape, participants), it will have to be
proven that they are not a remake of Saussure’s distinctions or of similar
semiologic paradigms. Naomi Baron understands that representation is tested in
communication. Applying her model to the typology of artistic representation,
she makes an attempt at interdisciplinarity, which is actually the only
characteristic shared with the other contributions to this issue. Here, I would
like to point out that while divergent in premises and conclusions, the results
of our activity are based on this shared understanding of interdisciplinarity.
It might not be enough to configure a school of thought--and there is no need
to regard this issue as representative of a newly formed group--but it is a
critical characteristic.
The questions that Nikhil Bhattacharya raises are of principle: How
adequate is verbal language for communication in a world of individual,
subjective experiences? What are the shortcomings of visual representations?
Will understanding visual language help us better understand verbal language?
He relates his inquiry to the increasingly important role of computers and
their languages and the need to understand the referential, iconic, and
symbolic aspects of verbal and visual constructions. While occasional
semioticians do not have time to explore basic issues, Bhattacharya positions
himself in the philosophical realm. It is not useless to say that, whether
following Saussure or Peirce, it is impossible to understand the perspective
from which they work without understanding the philosophical foundation.
Removed from the philosophical context, either of the two systems is only a
collection of strange words. Philosophical foundation should not be seen as a
goal in itself, and this makes Bhattacharya’s contribution distinctive. When
talking about iconic elements in visual language, he discusses the role of
convention in order to discover that iconicity is a relative quality of
representation. Previous discussions (especially Eco’s on iconic
representation) have missed this point. It is no accident that, although
computer scientists use icons for making their machines more user friendly,
Bhattacharya asks whether iconicity is available in human language and
furthermore, how iconic encoding-decoding takes place. No doubt that for
someone unfamiliar with ‘computerese’ or what are called ‘buzzwords’, his
discussion of the typical ‘garbage collector’ will present some problems. It
just happened that when LISP, the artificial intelligence computer language,
was first presented to the public, the expression ‘garbage collector’ produced
a big laugh. As a formal language device meant to help in controlling the
amount of memory used, the ‘garbage collector’ presents not only technical
aspects but also very important semiotic aspects. The example that Bhattacharya
discusses belongs to meta-language. Once again, interdisciplinarity was
accepted as a necessary premise.
An important segment of the entire research has concentrated on the
various aspects of graphic design education and activity. It should be pointed
out that Thomas Ockerse (in collaboration with Hans van Dijk) elaborated a
course in applied semiotics that has been a requirement in the Graphic Design
Program since 1977. Since its inception, the course has integrated the results
of research carried on in the United States, Europe, and Japan and has in turn
become a source of research, an experimental laboratory, and a viable context
for testing results. Thomas Ockerse is an example of an artist with high
semiotic awareness. The greatest part of his creative work deals with sign
processes. Ockerse has very good conceptual, semiotic discipline and an
original way of involving it in generating images with poetic value. One of his
concerns is the relation between various types of signs--visual, verbal,
musical. Attracted by the Peircean semiotic approach, Ockerse not only
classifies signs but also shows how classifications can be broken, proving that
dynamic continuity (what Peirce called synechism) is just as important a
part of his semiotics as his definition and typology of sign. It is, I believe,
rewarding to see how semiotics becomes alive. Even if, at the level of the
metaphor, conceptual discipline is no longer entirely possible, we can learn so
much more about semiotics.
Claire Taylor’s study concentrating on ‘noise’ in visual communication
is not surprising. Again, we have the occasion to observe that what we call noise
is, among other things, the interaction between visual and nonvisual,
between various codes of a given culture. The expression of the conflict of
values in this idea, together with the suggestion of the formative role played
by noise (to a certain extent, culture is the product of
restrictions--conscious or not--imposed by noise), is pursued mainly in the
printed media. The conclusion (slightly provocative) is not the result of
accepting noise as a disturbance in communication, but of understanding the potentiating
function that noise exercises. Instead of a septic, sterile, monotone image in
which the personality of the author, photographer, or illustrator disappears
behind depersonalized typography, Claire Taylor suggests expressive
spontaneity, the graphic ‘accident’, ‘imperfection’ (the latter graphically
controlled). Extending her research to other forms of visual expression (for
example, Times Square and ‘noise’), she observes the semiotic phenomena through
which noise is integrated in the message and accepted as a cultural value.
Contrary
to the tendency of many researchers to produce new taxonomies adapted to the
medium analyzed, Bethany Johns asks whether certain explicative models of
nonvisual origin (in this case coming from the analysis of poetry) can be
applied to the visual. Her procedure is obviously integrative, but it could not
be anticipated to what extent concepts so different in nature from the visual
(such as those concerning metaphor) can be applied. To read Vico, for instance,
after Peirce’s semiotic model is understood and his theory of logical nature
applied to the symbol means not so much to confirm Vico’s explanations but to
integrate important results that have been ignored, if not rejected, by
semioticians. At the level of reciprocal action between constituent parts,
Bethany Johns points out different levels at which the image is constituted,
the relation between the visual and nonvisual, the openness of processes
through which associations, superimpositions, disassociations, etc. are
produced. It is a domain of junction in which semiotics’ quality as meta-domain
becomes especially clear.
Finally, from among the research projects I have been engaged in since
the establishment of the Institute for the Semiotics of the Visual, I selected
one of the themes everyone talks about, but not always with enough professional
discipline. The meaning of the visual is part of the comprehensive subject
called the meaning of the sign, and it is not possible to avoid, as a premise
of research, the definition of the sign’s functions as they derive from the
definition I adopted. Semiotics has been literally invaded by all sorts of
specialists, scholars who never really succeeded in their respective domains.
They recycle some of their older articles or lectures, introducing two or three
terms with a semiotic flavor. (The magic word ‘meaning’ always shows up.) There
is no intention to deal with this phenomenon, first of all because I believe
that the best way to defend semiotics is to continuously improve the quality of
our research and of everything we decide to publish. However, not to be aware
of the confusion shadowing semiotics is as detrimental as contributing to the
problem. This issue would not be possible if the participants in the session The
meaning of the visual: On defining the field at the VII Annual SSA
Conference held in Buffalo had not taken part in discussing our theses as
generously as was done. It was our intention to allow as much time as possible
for discussion and the three hours open for discussion provided us with the
feedback from our colleagues that we wanted and looked for so much. Our special
thanks go to David Lidov, Wendy Holmes, Donald Preziosi, and Elaine Nardocchio,
to mention a few. The discussion encouraged us to propose to Thomas A. Sebeok
publication in a special issue of Semiotica. The fact that he overcame
an initial circumspection, as expressed in our beginning correspondence, is
just one reason for thanking him for making publication of this issue possible.
Mihai Nadin (1981). Zeichen und Wert. Tuebingen:
Gunter Narr Verlag.
--.
(1983). The civilization of illiteracy, Semiotics 1981. Deely and Lenhart
(eds.). New
York:
Plenum.