Interview with Aaron Marcus

Aaron Marcus spent a lot of time with his nose buried in comic books and science fiction stories. When it was time to learn things, he chose to learn…everything. His history reads like a who’s who, and a where’s where, or the early days of our field. My favorite part? He went into more than one job interview saying “I don’t know why you would want to hire me.” That makes one of us.

EXCERPTS FROM THE INTERVIEW

I have to admit: it’s a wise newbie or youth who actually looks to some older people and says, “Oh, they might have something there; let me go ask.” A more typical opinion is, “I think I know better. This is all new. Nothing like this has ever happened before, and I have unique insight into the situation.” That may be so. But I think we can learn from our elders. Especially now that I am an elder, I naturally favor that opinion.

Tamara Adlin: Hello everybody, and welcome to this newly rejuvenated set of interviews of UX Pioneers.

I’m Tamara Adlin. Today I have the honor of talking to Mr. Aaron Marcus, who is the President of Aaron Marcus and Associates, Inc., or AM+A. He was graduated with a degree in physics from Princeton, and in graphic design from Yale.

In 1967 he became the world’s first graphic designer to work full time with computer graphics at AT&T Bell Labs, where a lot of early work was done in the field of user experience.

In the early 1980s he was a staff scientist at Lawrence Berkeley Laboratory. He founded Aaron Marcus and Associates, and began research as a co-principal investigator of a project funded by the U.S. Department of Defense’s Advanced Research Project Agency, or DARPA.
In 1992 he received the National Computer Graphics Association’s annual award for contributions to the industry. In 2007 he was named an AIGA Fellow. In 2008 he was elected to ACM’s CHI Academy, and he serves as Editor-in-Chief Emeritus of User Experience Magazine, and is an editor of Information Design Journal.
Mr. Marcus has written over 350 articles, has published 15 books, and has lectured, tutored, and consulted internationally for 45 years.

He is a true UX Pioneer.

Welcome, and thank you, Mr. Marcus for being with me today.

Aaron Marcus: Thank you very much, Tamara. I’m so pleased to have this opportunity to speak with you.

TA: I want to mention that Mr. Marcus is also a big — if not the big — reason for the rejuvenation of the UX Pioneers project. I’m so delighted that he got in touch and helped me build a fire under this project once again.

I’m going to start this interview the way I’ve started all of my interviews: what is the first thing you can remember really fascinating you as a kid?

AM: Well, as a kid I really was interested/fascinated by astronomy, rocket ships, science fiction.

TA: How old were you? Were you reading books on these things?

AM: By the age of 10, I think I was reading comic books and science fiction magazines like Galaxy, and other of the pulp science fiction magazines, but I was also a comic book advocate and fan – addict, I guess, is the better word.

I collected myself about 18 different brands, and two of my friends and I had an agreement that we would each take a different portion of the complete comic book universe. Another of my friends actually took science fiction.

I took DC Comics and Marvel Comics, if I recall correctly.

TA: Wow. Do you still have those?

AM: I sold most of my collection [of 1500 comics] in 1972 but I still have about 300 comics from the 1950s and 60s. I had Action #82, I think, from 1938 – even before I was born – courtesy of my cousins. I had a lot of wonderful science-fiction magazines, which led to my drawing lots of rocket ships, space guns, men and women in space suits, future cities, life on other planets, etc. I still have a lot of those drawings, actually, and I’ve published a few of them in an e-book, available at our website, called The Past 100 Years of the Future: History of HCI or UX in Science Fiction, Movies, and Television.

TA: I’m excited to talk about that topic. But before we get to that, I want to come back to one thing, which is one of the things that truly fascinates me in these conversations, the mixture of science and art/design. You’ve already said you were fascinated by technology, like rocket ships and astronomy. And at the same time you were fascinated by a very artistic expression of those ideas and designs in comic books.

AM: That’s correct. In fact, I was blessed with abilities not only in science but also art and design. I loved to read about these things. I also drew them. I still have this collection of drawings from childhood.
I also made things. I guess it was an early version of the Maker phenomenon [the current interest catalyzed by 3D printers, for example], because I built pretend rocket-ship control-rooms in my basement. My late brother Stephen, friends, and I would “take trips to other planets.”…[I guess you could call it an early form of “trippin’out.”]

TA: Did you want to be an astronaut when you grew up?

AM: I’m not sure that I wanted to be an astronaut; I might have. I don’t recall that being an important matter.
I think what was intriguing to me was designing the control panels: the dashboards, the displays, which I made out of junk that I found in alleyway trashcans. You know: dials and gauges and knobs and old radio parts.

TA: Where was this? Where did you grow up?

AM: This was Omaha, Nebraska in the Midwest. A nuclear family, the 1950s. To tell you the truth, I’m forgetting that when my late brother and I were even younger, we were about 4, 5 or 6 years old, we would tell stories at night with our teddy bears. We would take trips to other planets in some kind of rocket ship.

I remember one of the concepts that we invented at the time was
“The Big Book of Knowledge” in which one could find the answer to any question. This concept was our version of Google in the early 1950s or late 1940s.

TA: So your imagination was all over the universe and all over human knowledge.

AM: Well, certainly we fantasized learning about different things and going off to different planets. On one of these planets, we even watched how little baby teddy bears would be born. We were figuring out our own versions of childhood birth theories.

Little teddy bears would grow like fruit on the trees. Their heads would emerge, and when they were ripe, the full bodies would fall off, and the little teddy bears would walk around on the ground.

TA: Okay, you just won the prize for most combined, disturbing-yet-cute imagery in a UX Pioneers interview.

Tell me a little bit about your experience in school, and what you focused on when you were in elementary school and in high school.

AM: Thank you very much for asking. I really appreciate the chance to remember.

I was always interested in science and art. I didn’t actually know very much about the word design because design was not then — and still is in many ways, not taught well, at least within the U.S. school system.

So I loved drawing and painting. I learned later as a college student about photography and calligraphy.

In grade school I was introduced to color theory. I still have the sheets of colored paper that I pasted up to demonstrate color phenomena I can’t remember which year-class it was. But because of this experience, I not only enjoyed science and mathematics classes, but I also enjoyed any creative classes of arts, crafts, and design.

In fact, I’m just remembering now my Speedball pen set, with which I made letters and posters and practiced calligraphy in grade school, and certainly in high school.

TA: I assume you’re talking about primary and secondary and tertiary colors and how that works.

AM: Color circles and so on. Complementary colors, harmonious colors, triadic colors, etc.

TA: And then you said in the same breath that you were very interested in physics and math. I don’t think a lot of people would put those two in the same breath.

AM: Well, I think that’s probably true. It was both my blessing and my curse.

I think I was a certified nerd early on. When I was six or seven or eight, my uncle was teaching me algebra. I thought that was kind of cool.
One of my childhood accomplishments of which I was proud, was that I had memorized the value of pi to 50 decimal places. I don’t know how many kids would want to do that, but that’s what I enjoyed doing.

I didn’t play that much baseball, and participate in other kinds of activities that more socially oriented kids would do.

TA: You did sort of concentrate on art and math, on art and technology.

AM: Art and math, art and science, yes.

TA: They were wrapped up in your head together as things that were interesting and related.

AM: Yep. I didn’t know how to put it all together. Not for decades, at least.

In fact, just to jump ahead, it took me about 10 or 15 years after I got my professional degrees to figure out how it all fit together.
I remember in 1979, when it became clear to me. I knew I had programmed computer graphics. I was interested in graphic design, visual communication. I was teaching at University of California at Berkeley at the time. I received an invitation to address federal graphic designers at the National Gallery of Art in Washington D.C., through colleagues of mine at the National Endowment for the Arts. They said a famous graphic designer had become ill; would I like to step in and give the presentation for this occasion? I was frightened to death, but I said, “Sure.”

Then had to figure out what on earth I was going to talk about.
I went into the ozone worrying about this pending event and thinking about it for about three days. It was somewhere around November 22, 1979.
Out of that turmoil came the realization that I could put together all of my background and interest of computers and technology; science and mathematics; philosophy and visual communication; graphic design, art, everything.

Out of that challenge came the understanding that I could apply the principles of visual communication to the emerging world of computer graphics technology. Once that pattern became clear, suddenly a lot of clouds disappeared from my thinking, and I saw the pathway forward.
But I did not see that so clearly for 10 or 15 years after finishing graduate school.

TA: That’s fascinating. It’s actually a great opportunity for me to say we’ve gone from the 50s when you were a child, fascinated with comic books and science and through a little bit of your primary education, which takes us a little bit through the 60s, and then we fast forward it to 1979, an exact date which I’m impressed that you remember. I’m terrible at that.

That’s when you realized that this sort of winding path was potentially making this wonderful straight line?

AM: It wasn’t maybe a straight line, but it was a tree of many possible branches, and the tree was clear. Before, I had been wandering around with many possible trees, or even many possible paths in the forest. I didn’t understand so well how it all fit together, and how I was somewhat destined to do this work.

When I got to art school and design school as a master’s student, I already owned all the necessary tools. I just didn’t know what they were for. But I’d been collecting them and using them all my life. But I didn’t understand why.

TA: Let’s backtrack for just one second.

AM: Sure.

TA: Where did you go to college? How did you decide where to go to college and what to major in?

AM: (chuckles) Sorry for laughing.

TA: Why?

AM: I got accepted to MIT and Cal Tech, if I remember correctly, but I decided to go to Princeton University because of the influence of a friend of mine with whom I’ve had contact for the last 50 or 60 years.
I thought it might be better if I had a liberal arts education and did not immediately have a “slide-rule education”, as I called it. I hope you know what I mean by a slide rules [The old analogue computing tool made of several pieces of wood or metal with number scales that could be used for simple computing.]

TA: Absolutely.

AM: I was glad, and fortunate, to have access to fine teachers … I specialized in physics; I had lots of math classes and philosophy, but I also took history, art history, painting classes, etc., as an undergraduate, and felt that that would be better for my general education.
It was with some concern that I felt I would not go on to graduate school in physics and study laser technology, which is what I was drifting towards, because I was really more interested in general theory of gravitation and other topics.

TA: So you went to Princeton. What year did you graduate?

AM: 1965.

TA: ’65; and this was with a liberal arts degree?

AM: It was a B.A. Not a B.S.

TA: Those are amazing schools. Princeton is now known just as much, I think, for its technical degrees as for its liberal arts degree.

AM: Yes; it had a strong engineering department at the time.
I subsequently applied to several art and design schools like Cooper Union and the Art Institute in Chicago and the Rhode Island School of Design. I got into all those schools but only one school was a graduate school, one school had a graduate program, and that was Yale.

I thought well, I don’t want to go through an undergraduate education all over again. I think it’d be best if I moved forward at the graduate level. They had a special one-year program in which I was supposed to learn everything that my peers had learned in four years!

TA: Okay, hold on a second. I know there’s more detail here. What is totally normal in your life history for you, sounds wacky to everybody else.

You went to Princeton, where you were thinking that you were going to move forward into lasers and physics, and did go ahead with a wonderful undergraduate liberal arts degree as well, and all of the sudden you’re talking about getting into amazing undergrad and graduate level art/design programs.

That’s not something that’s familiar to a lot of people; being fluent in highly technical, mathematical, scientific fields and being able to get into Yale Art School.

Tell me a little bit more about that.

AM: Because I had built up a portfolio, all on my own and with the help of a mentor at Princeton, who was a Yale graduate –

TA: What? Were you drawing? What were you painting in your spare time?

AM: I had learned photography and had taken hundreds and hundreds if not thousands of photos. I developed all of them on my own in the little labs that we had for photography.

I took printing and typography classes in my spare time in Princeton’s Visual Arts Program. I learned Italian chancery cursive handwriting, because I thought it was beautiful and interesting, and I practiced it.
I sat in on painting classes with figure painting and still-lifes. So I had some things to show.

TA: Was this just because you had to? Do you have that artist’s drive where you can’t not?

AM: Yes; I had to do that. I could not not do it.

TA: So are all of your notes from college and university covered with drawings?

AM: Yes, as a matter of fact. And I still have my diaries from the past 50 years or more, sitting about ten feet from me. There are about five feet of running length of them. Many of them, certainly the earlier ones, are filled with many drawings, sketches, and cartoons,

TA: We’ll have to find some of the less personal pages so we can share a few with the people who are reading this interview.

AM: I never thought to do that, but sure. I forgot to tell you that in my undergraduate career I was the art editor for the humor magazine and drew many cartoons.

TA: So I think the people who are reading this, if they’re anything like me, will just be gobsmacked at how you managed to find the time and mental energy for all of those things, pursuing such a difficult set of degrees.

AM: It wasn’t easy. And, I should point out, that when I got to graduate school – I tell this to people – that my graduate school years were some of the most fun and happy times of my life, because everything was new.

When I first arrived in graduate art and design school and was taking design courses, I had no idea what anyone was talking about. I didn’t know what the words meant. If someone said “Oh, that works,” I would say, “What works? What is it that’s “working? What are you talking about?”
It took about a semester, half the first year, year, before I got it; I grokked it. I realized that what they were talking about was something similar to what I had done in physics labs where I tried experiments.
A lot of the work that I was doing as a graduate design student was trying things and thinking about principles that might lead one from here to there, and where to go next, and taking note of what happens

And it began to make sense to me.

TA: I love this. I love this for so many reasons. One is I can personally relate, because when I went to graduate school – this was in ’94 – I went in to get a degree in Technical Communication, because at the time there weren’t programs that were in human/computer interaction that I qualified for. Later, I was one chapter ahead of my students as I taught. I didn’t know what technical communication was.

So to contextualize this for everyone reading, imagine going to grad school as if you came in with the understanding of your field that you would on the first day as an undergraduate. That’s amazing that you went in so fluent in the stuff that you had studied as an undergrad, and yet so non-native in the field that you were doing advanced work in.

AM: Well, realize, I was sitting in color classes cutting out little pieces of colored paper and gluing them to a page. I wondered to myself: am I perhaps dreaming, and actually in an insane asylum and just doing therapy? Because I could swear the last semester I was doing relativistic quantum mechanics and dealing with Schrödinger’s equations, and now I’m cutting out these little pieces of paper.

As I say, it all seemed a little surreal to me. But … eventually things settled down and I was learning so fast that essentially I didn’t sleep for about three years.

All during those three years, to get the first baby graphic design degree in the first year and then move onto the master’s degree for two additional years, was so exciting. Everything was so new and fresh. Whereas for my classmates, a lot of it was just more of the same with some refinement.
I can understand that. But for me, I was on another planet. It was so interesting and so much fun that it was too boring to sleep.

TA: Do you think that you were unusual in the amount to which you observed yourself having these experiences? I’m hearing you saying that you were very tuned into your own newness. You were watching yourself have this experience, of being new and learning.

AM: Well, I would write things down in my diary and draw people and things that I saw and take photographs of them. So I would say that I always felt that I was an outsider observing and taking notes.

That’s part of the challenge I have faced personally in my career because I was never fully accepted by the design community, nor fully accepted by the computer technology community because I was always slightly different.

I considered myself a Janus figure – a doorway-figure sitting between disciplines. I could look into each of them. I could talk with all of the people. I had learned how to be them. But I was not just one of them.

TA: Right. I often tell people who don’t understand what I do, that I’m a translator between the worlds of engineering and business and design. Does that resonate for you?

AM: Yes, yes … very much. It took me a long time to be able to explain to people, and to my parents who asked: what is it that you do?

It was quite frustrating at times.

I want to say, also: you said “did you consider yourself unique” in doing something. In my early years, as young people are perhaps prone to do, I did think of myself as unique in some ways but as I’ve grown older, I’ve just found myself so typical in many ways.

Growing older, having a family, having the pressures and the challenges that one has in life, I am just startled by how average and typical and ordinary my concerns and interests and directions are. So that I realize if I have what I used to think of as brilliant, new idea – I think: “Oh, there are probably nine or 900 people having the same idea right now around the earth, thanks, in part, to our means of instantaneous communication.
That realization has fueled my interest in cross-cultural communication and some of the other topics in which I’ve been interested in, like mobile persuasion design, in the last decade.

TA: So as you hop ahead, it’s a great reason for me to hop back.
Let’s hop back to that moment in 1979. You had been finished with your undergrad degree, you were finished with both your undergrad and grad degree in graphic design. When did you finish with your degrees?

AM: ’68. And it was only in art and design school that I actually learned computer programming because a classmate said, “Hey, you have this ability in science and math that none of the rest of us do. You should go to this interview from AT&T Bell Labs because they’re hiring people. Maybe they’ll want you.”

TA: So that’s where you went in 1968? You hadn’t learned enough– now you learn computer programming?

AM: Yes; 1967 I decided I wanted to learn about computers because I’d never done it in the physics realm. It was too early in the history of computers and computer graphics.

I taught myself at the computer center FORTRAN programming, a popular programming language for science and engineering. I went to this interview from AT&T Bell Labs which took place in….it must have been May of 1967. I did the sort of thing that you would tell someone never to do

TA: What did you do?

AM: I came in to meet the two interviewers, who were dressed like Men in Black people. They didn’t have the sunglasses but they had dark suits and they looked faintly sinister.

I said, “I don’t know why you would want me.”

TA: Oh, no.

AM: …But I have this background in physics and math, and I’m now at art/design school, and I’ve learned computer programming. Is that of any use to you?”

The two men turned their heads, looked at each other, smiled gently, looked back at me, and one of them said, “Well, actually we’re looking for someone exactly like you.”

TA: Wow.

AM: It was pretty eerie and creepy – and wonderful.

I went on to work with those two guys as a summer intern at AT&T Bell Labs. I was suddenly plunked down in the most advanced computer technology on the planet.

TA: Why were they looking for you?

AM: They were looking for people in the art and design realm who could tolerate and work with computer programming, computer graphics technology at the time, which was extremely cumbersome and clumsy.

TA: And why did they want to do that? This was in 1969?

AM: Ah, because in 1964 they had demonstrated the AT&T Picturephone ™.

TA: Wow.

AM: At the World’s Fair in New York City in ’64. In ’69 I became a consultant at AT&T Bell Labs to program a desktop publishing system for the Picturephone, about five to ten years ahead of commercial systems.

TA: Did that little kid, that little rocket ship designer in you, just plotz?
Were you just over the moon inside?

AM: I was; I plotzed. I was extremely – you know – thankful and I couldn’t believe my good fortune. At that time, AT&T Bell Labs in ’67, ’68, ’69, ’70, had some leaders (like Billy Klüver) who were very interested in bringing designers and artists together with computer people – technology people in general.

There was an exhibit in New York – the Experiments in Art and Technology – which published a conference proceedings in about 1970 or ’72. It gave me at Bell Labs an opportunity to meet advanced filmmakers; Nam June Paik, one of the first and major video artists, from South Korea who was considered a national treasure of his country; Stan Brakhage, Virgil Finlay, if I remember correctly; and others.

Peter Denes was my superboss, and Michael Noll was my immediate supervisor. At that time, Michael was creating the first 3D virtual realities. Ken Knowlton was creating the first computer animations. Max Matthews, invented computer music and computer speech was around – you know the “Daisy, Daisy” song from the film 2001?

(Listen starting at 40 seconds to hear the ‘Daisy, Daisy’ song recorded as part of a 1963 Synthesized Computer Speech demonstration by Bell Laboratories. )

 

TA: Oh, yeah.

AM: Well, I have an old plastic tape reel from that time, because Max was around the corner perfecting speech. It was his song, “Daisy, Daisy” that Stanley Kubrick and others learned about, and incorporated into the movie.

TA: That just gave me chills.

AM: As HAL breaks down.

TA: That is amazing.

AM: That is a recording that I had already heard, and of which I have a copy, going back to the 1960s.
When I started working there as a consultant in ’69-71, it was the first time that television sets, that is, raster scan displays, had been attached to computers. It was black and white television screens, so to speak. Not cathode ray oscilloscope or CRT [cathode ray tube] screens with vectors.

TA: And you were trying to code – people were moving from coding on punch cards to –

AM: Actually most work then was on cards. Yes; I still have my punch paper decks somewhere in my archives….And I still have some of my first microfilm displays of computer art that I programed in 67, 8 and 9, ’70 and ’71.

So I finished that desktop publishing project and published a paper about my work, as I tried to figure out what was the future for design, in particular graphic design.

[See, Marcus, Aaron, “A Prototype Interactive Page-Design System,” Visible Language, Vol. 5, No. 3, Summer 1971, pp. 197-220.]

In fact, that was one of the first occasions in which I designed a computer-based user interface for the software that would be running on the Picturephone.

 

TA: How far along did the Picturephone get?

AM: Oh, well there were brief attempts to commercialize this in the ‘60s but it never went too far.

TA: It’s amazing because we started our conversation today, just for context – what is it – it’s June 2014 – and we are sitting here talking over Skype where we just turned off the video, and we’re recording this call. It’s just astounding, in a way, how early it started and how long it’s taken to get to this point.

AM: It’s been a long, winding road.

TA: Did you have moments where you remember feeling like they’re going to realize I’m an imposter any second and throw me out of here? You must have as a graphic designer.

AM: I had many such moments. Including when I began teaching at Princeton. I went back there to teach for nine years. I taught one of the first courses in computer graphics and design for designers and liberal arts students and computer technology people, all to get together and work together on projects. I actually published a paper about the course:

“A Computer-Graphics Course for the Two Cultures,” Third International Conference on Computing in the Humanities, State University of New York (SUNY), Albany, NY, August 1977.

When I first started teaching, I was only – well – one or two years older than the students I was teaching. I thought, “Oh my goodness; they’re going to see through me, that I know nothing much more than they do, about what I’m supposed to pontificate.”

TA: And yet you did because nobody knew what you were pontificating about.

AM: Fortunately I had a few things to say that were of interest, but it was a dark and stormy time. I was so nervous. I wrote out all of my lectures in those days, of course, on typewriters. I read them. It must have been excruciating.

TA: Oh, no. There was no funny hat at that point, huh?

AM: No; I didn’t start that until 1982, 3, 4 5, I think. I used to wear Mickey Mouse hats and other things. Eventually, I stopped writing out my lectures and moved to completely visual imagery that would just remind me of things to say.

TA: We’ve gone from when you got your grad degree in graphic design, to this strange Men in Black interview, where they let you in to the most technical world of worlds at AT&T Labs working on the Picturephone project, which is so James Bond, I can’t even stand it.
You met all these amazing people. That takes us up to what; 1972, ’73, ’74?

AM: Well, I continued teaching at Princeton, decided to move to Israel in 1977, because I wanted to start a computer graphics and design center at Bezalel Academy of Art and Design in Jerusalem. That happened not to work out.

TA: Wait a second. When did you leave Bell Labs?

AM: I stopped working with them in ’71. I continued teaching at Princeton to ’77. That didn’t work out in the way that I wanted it to so I needed to come back into the U.S. teaching system, and I didn’t know how to do that.
Suddenly Sheila de Bretteville, a very famous graphic designer and now Chair of the Graphic Design Department of Yale, which is completely coincidental, wrote me and asked me if I would like to go to Hawaii for a half a year as a research fellow. I said, of course, “Oh, yes. Why not?”
She didn’t want to or couldn’t do that particular activity. So I moved my family from Jerusalem to Honolulu and became a Research Fellow at the East-West Center, on the University of Hawaii campus, in order to visualize global energy interdependencies.

TA: That’s totally random.

AM: Yes. It sounded interesting. It sounded like the combination of technical communication and visual communication that might be challenging. I led a team of visual communicators to create a non-verbal storytelling of charts and diagrams, signs and symbols, to tell the story of how we need to get together to solve our energy problems around the world…. or we’re all going to die.

The East-West Center had been producing reports 600 pages long about all of these topics but few were reading them. So they decided to create a little project to see if they could create informational storytelling. That’s what we worked on for six months. [See the pdf of the publication that resulted.]

It also was an incredible, wonderful experience. I represented the entire United States. I had a friend representing all of Japan. A woman geographer representing China. An Indian audio/visual person representing all of India, and the translator of Marshall McLuhan into Farsi, from Iran.

TA: Ok. So that’s amazing. And, this is the first – well – no it’s not because the Picturephone, too; but this is clearly a huge user-interface design challenge.
If you’re trying to get together the replacement of a 600-page book into something in the format that people from all over the world will not only understand but be moved by, to affect change –

AM: That is correct.

TA: That’s a huge … is this … I don’t even know how to word this question because of the history you have. But is this one of the first times you’re thinking about “the user,” or did that come up a lot before this?

AM: I was already thinking about users. I didn’t have the terminology. I didn’t even necessarily know the word “user interface” and certainly not “user experience”. I don’t know when that term was first used. But I was already thinking about that when I was programming displays in 1969-71 for the Picturephone. But what this “visualizing global energy interdependence” project did was move me towards information-oriented visual communication.

I should have said earlier, I’ve been interested since 1976 in semiotics, the science of signs, because I happened to meet Umberto Eco at Princeton, who is one of the great people of the field.

I realized in this project that we were trying to create visual storytelling, a sort of visual/technical communication that summarized hundreds and hundreds of pages of complex documents in presenting policies, world views, new technology, etc.
We were trying to communicate to world leaders, econometricians, and the person on the street.

I even later published a paper about the semiotic rhteorial tropes used in our presentation:

TA: Do you think that your “ah-ha” moment, when you had to give that presentation putting together graphics and technology in 1979, impacted how you approached this project, and how you approached your work, in a new way?

AM: It’s the Hawaii project which impacted my realization in ’79, because I was in Hawaii in ’78. And they were related. I sensed my destiny or my path not to be purely a technology person or a technocrat or a technical researcher…. nor a corporate designer, graphic designer or cartoonist or something like that, but to be something in between that was combining all of this informational communication; of complex structures and processes.

In fact the slogan for our company has been, “We help people make smarter decisions faster.” That’s what we were trying to do in ’78 in this audio-visual presentation. I had wanted to do it through computer graphics but the facilities at the University of Hawaii were just not sufficiently capable at the time, so it was done in a hand-crafted form from previous decades.
But we published the article about our work. It moved me towards a realization of what it was that I could do, and that led to my second ridiculous job interview at the Lawrence Berkeley Laboratory’s Computer Science and Mathematics Department.

I somehow convinced the head of the department, Carl Huang, to let me come in for a job interview…and I began with my classical words: “I don’t know why you would want to hire me, but I have this background, and I think I can help you in some ways to communicate the computer graphics systems you are developing.”

Luckily, he said ‘Okay; let’s give it a try.” Well, I realized I had about $4 million of computer equipment around me, not too far from Silicon Valley, and that they were building for the Department of Energy a multiple-database, computer graphics display-equipment independent software system that could munch on 60 different data bases at 60 different levels of aggregation and spit out charts, maps, and diagrams of any information obtained by the U.S. Government.

One of the first things they were doing was to try to produce for the public, U.S. Census data on demand. So it was a very democratic, in the ‘little-d’ sense, form of visual communication.

I wrote one of the first user interface design guidelines in 1979 – 80.

TA: What was it called, that document?

AM: The system they were creating was called SEEDIS. It stands for Socio-Economic, Environmental, Demographic Information System. Ha ha.

TA: Ah, friendly.

AM: Yes; very friendly. I wrote a “User Interface Guideline for SEEDIS.”I think is what I called it. I was attempting to help the researchers there and their clients in the U.S. Department of Labor and U.S. Department of Energy produce a better way for people to use that data, and to create templates or affect how information would be displayed.

So, one of the tasks at that time was to design “a one-million page book.”

TA: Oh, my gosh.

AM: It isn’t exactly that, but it was the first time that people around the country could call into the U.S. Government and produce laser-printed documents that were designed to show whatever information they wanted, for whatever city or state or county or standard metropolitan statistical area they wanted….and receive back in the mail their document.

TA: So this was publishing on demand, in the early ‘80s.

AM: It was publishing on demand of U.S. public data, for the first time, being available to the public.

TA: What did you have to design?

AM: The way all the text, charts, maps, diagrams, and page layouts would look for any possible combination of contents.

TA: Wow. Let me also do a pause right here, and just mention – you’ve just said something that is a rolling theme in these interviews, which is a sort of a moment or series of moments in a career where you realize two things; a) I’m in between. I’m an in-betweener; I’m in between these different disciplines.

AM: Yes.

TA: And even more importantly, b) This is where I want to be and this is a valid place. Because I think that the field of user-experience design or user-interface design, or whatever you’re calling it this week, is more of a blossoming of an in-between than it is the development of something brand new.

You were one of those people who sat in between computer graphics and coding.

AM: Mm-hm ….

TA: I think right about the time you’re talking about in the early ‘80s, this is when other little, if we take the plant metaphor, little buds were showing up in other parts of the world as well.

AM: Affirmative.

TA: Did you begin to become aware – and did you suddenly have the language – if you call this document a “user interface guideline” that suddenly those words had appeared in the zeitgeist.

AM: Yes, yes.

That was partly through, say, a seminal publication by Jim Foley who wrote a great article in about 1976 about the fundamental concepts and terms and actions of user interfaces. I had contact with him all through the 1970s and 80s as his career progressed at Georgetown University and later at the Georgia Institute of Technology.

He along with others, like Andy Van Dam, another friend from early on in the ‘70s, created hypertext, created terms and concepts that helped people give names to these fuzzy phenomena that we all were encountering but may not have named in quite that way. Prof. Van Dam and I were in touch a number of times in the 1980s, and I wrote an article with him about the future of user interfaces in 1993.

Marcus, Aaron, and Van Dam, Andries. (1991, September). User-Interface Developments for the Nineties. ACM Communications, 24(9), 49-57.

In fact, one of the key things I’ve said for decades emerged in the 1980s: that user interfaces are essentially combinations of metaphors, meta-models, navigation, interaction and appearance characteristics.
I’ve been interested for a long time in metaphor design and metaphor concepts. It’s the search for metaphors that people stumble around and eventually find something that works that spreads rapidly, becomes a meme, becomes a paradigm, maybe even eventually a stereotype. But it’s that struggle to encapsulate essential characteristics in something that makes sense; that people can remember and learn easily, and that is appealing. Like the desktop.

TA: Right. There must have been something tremendously validating to know that there were other people out there that in some ways were like you.

Do you recall when you started to become aware of, or to help start developing a community of people in what we now call user interface design?

AM: Yes, yes, yes.

I remember going to the first conference before the CHI conferences came into being in 1982; SigCHi. SigCHI began in Gaithersburg, MD in 1982, and I attended that one.

In 1981 there was a pre-cursor conference in Ann Arbor, MI, of social scientists and others, who were thinking of trying to create an organization that would provide a home place, a home base, for people who were moving in these directions that were not considered “authentic enough” by computer-science departments or electrical-engineering departments or social-science departments or whatever you might mention. And certainly design schools and art departments were not ready for them.

For many years, for decades, SigCHI meetings were like being at summer camp. To be together with people who all were trying to understand how technology could be changed to be more humane; how various disciplines could contribute unique expertise and experience, and could come together to make cross-disciplinary contributions. Like what happens when we now take for granted all of the people who create a movie. All those credits that roll at the end. We don’t see that for user interfaces.

By the way, I also attended SigGRAPH conferences regularly too, which were also a kind of summer camp for the computer graphics people. However, a lot of artists, designers, and user-interface people “hung out” at that summer camp, also all through the 1980s and 90s.

TA: No; in fact the opposite is kind of true, where if we do our jobs well as user-interface designers, we should become somewhat invisible behind this obvious solution that we create.

AM: Well, that is a theme that was presented to me when I first came to design school; that is, good design would be somewhat invisible.
Now, the fact is that some design does call attention to itself and that has value also: for being amusing, for making us self-aware of the artifact. But in many cases, certainly, if things are working well, we don’t even notice them.

TA: Do you remember a next big moment in your awareness of your career as a user-interface/user-experience person? You founded Aaron Marcus and Associates –

AM: …In 1982, based on our getting a DARPA grant. It was quite unusual for a graphic designer to become a co-principal investigator for a research project, especially one funded by DARPA. My friend and colleague, Prof. Ron Becker, for many years at the University of Toronto’s Computer Science Department, but then head of his own software company, and I succeeded in getting that grant, mostly through Ron’s expertise and experience. I had to learn how to start a small business. I had been a faculty member for ten or more years. I didn’t know very much about being in business. That was certainly kind of an “ah-ha” or “oy vey” moment.

TA: Exactly.

AM: Kind of a combination; I don’t know what the expression would be. I had to learn how to explain what we did quickly. We had to learn how to create projects and create deliverables and remember to ask for payment and make sure we got paid, etc.

TA: It was another language in which you had to become fluent. It’s interesting your bringing that up.

AM: Absolutely.

TA: Yeah; it’s something that I think many of us face over our careers: figuring out the business of being in our field.

AM: Yes. One of the books I started but never finished, unfortunately, is a book still tentatively titled, So You Want to Be a Rock Star Usability Analyst. I co-ran panels on this topic for some years , together with Jeff Johnson and Austin Henderson, among other people, two leading user-interface researchers/analysts. At the CHI conferences and elsewhere, our panels, birds-of-a-feather, or special-interest group-sessions sometimes had hundreds of people attending to discuss what was it like to be in business? How did one survive? What did one need to do or know? Because there were many publications of our disciplines and work being done but not much discussion about –not how to be a professional — but how to be a business person exercising these professional skills.

TA: It’s something I talk a lot about now when I talk to new professionals in our field: that we have to make ourselves usable; that we have to consider ourselves to be a product, and that our customers, our clients, are our users.

AM: Absolutely. We ourselves must be usable, useful and appealing….:-)…

TA: And to do that, the product that you created in Aaron Marcus and Associates was really Aaron Marcus. You are the one long-standing employee.

AM: That is certainly true. I knew that I could not be taken away from our name, our skill set, and our expertise and experience. So, it was a practical matter to need to utilize that persona or that brand. What do you think about the issue of scaling the user interface business, of creating agencies? Well, I remember when Bill Moggridge, may he rest in peace, one of the founders of IDEO, came to my tutorial in 1984.

TA: We have an interview with him, thankfully.

AM: And I wondered why was he coming to my tutorial on user interface design? He’s a product designer – industrial designer. But I found out soon enough.

IDEO was beginning to move into what I might have erroneously called “my turf” of user interface design. They were very successful. They had the marketing engine. They had the connections to corporate contacts. They had an established track record in product design and in industrial design, and they succeeded as did Hartmut Esslinger, the founder of Frog (59.32), and others, to create very large organizations.

Some of them didn’t survive. You may recall March First; do you recall that company?

TA: I actually don’t, no.

AM: It was proclaimed with great fanfare in name because it started on March 1, 2001, and grew to be one of the largest Web-oriented design agencies in the United States and perhaps the world. And one year later it was bankrupt and out of business. It had grown too fast. The economy had moved into the recession of 2002 and 2003. Our business, too, almost disappeared. That was one of the giants that disappeared just like giant legal firms in 2008 and 2009 disappeared, that you would have thought might be indestructible.

Some of the large agencies have survived. We’re drifting into business territory here, but we survived because we were a “guerrilla group;” small enough to be flexible and to change our activities. We did a lot of work in the 1980s for R&D departments of NCR and HP and other corporate companies that eventually did far less of that R&D and outsourced it to start-up companies, which they could then conveniently buy.

We had to shift to more commercial projects, after our first 10 or 15 years. We still do a mix of corporate R&D, as well as work for start-ups and commercial product development. We also gravitated to a new component in addition to our tutorials — which had always been there but which is more evident – namely, creating executive marketing presentations for new technology.

TA: Which is still user interface design; the product in this case is –

AM: Yes; it’s storytelling.

TA: Yes.

AM: Story-selling, to be specific.

The larger agencies were wise enough and capitalized well enough to continue growing. And they have now succeeded perhaps beyond their wildest dreams, because from my experience, a lot of corporate groups are so burned and frightened and understaffed that they only want to deal with very large agencies; that is, the larger corporations: fewer contacts and less paper work. It’s different for start-ups who can’t necessarily afford the larger design, research, and usability services.

So we have been excluded quite a lot because we’re not big enough, in the last three years, let’s say.

TA: Does that bother you?

AM: It concerns me. It does bother me. It’s frustrating. But it’s just caused other changes in our strategy for how to survive.

Basically large corporations wanted to outsource all the administration of multiple contracts to people and just have one large agency to deal with. But in fact what they’ve also created are substitute or virtual agencies that simply hire all of those smaller contractors –

TA: Right.

AM: And they take over the paperwork. So they just outsourced the paperwork.

TA: Yes.

AM: Now the problem is, it’s very difficult to get to know these third parties and to even present credentials to them. You wouldn’t know about them because they kind of operate in a stealth mode.

TA: As we end up close to an hour, I’d love to sum up with a couple of questions if that’s okay with you.

I know we’re missing a lot of the more recent work. Perhaps we should do another conversation.

AM: We’ve completely glossed over all of our Machine projects for the past five years that combine persuasion theory with information design to change peoples’ behavior.

TA: I think we should do another conversation about that, if that’s OK with you.

AM: It would be my pleasure.

TA: Okay. Let me ask you this, to sum up this particular conversation.

As you know, the title of this series of interviews is UX Pioneers and it really is talking to folks who have been absolutely instrumental in creating what we now call our field.

Is there anything else that you want to point out about your experience as one of the creators of this new field? Was there a particular time that you saw it really blossom?

AM: It was certainly a lovely, exciting and creative time in the early ‘80s when we started. A lot of companies were filled with barefoot friendship. I remember when the first version of the Macromedia software came out (multimedia editing software called Macromind), and people could actually access and share each other’s code or scripts.

And then people discovered, “Oh, wait a minute; that person’s stealing what I did.” A lot of protective barriers came up around things. But prior to that time, there was a kind of innocence.

It was charming. Perhaps short-lived. But for not only being in business but just staying in business, business attitudes had to prevail.

I would say that it’s been my experience that every five to seven years, there’s a new technology, a new thrill, a new bunch of newbies in the fields, and we kind of re-invent things again, often falling into some of the same patterns or mistakes, oversights, or tunnel vision that we’ve had before.
People then gain some maturity. The particular technology evolves and becomes more sophisticated, more humane, better designed.

People have forgotten, I’m sure, for the most part, the half decade of Video text and Teletext, which came out in the early ‘80s, I believe. Maybe late ‘70s but certainly early ‘80s … The displays were wild and crazy, and unsophisticated, and confusing, and terrible, because there were only eight colors that could be used, and everyone wanted to use all eight colors on every screen.

It takes a while for maturity to evolve.

TA: Do you think that has to do with it being such a youth-oriented culture; the culture of computing, and I guess it seems you’re saying that every generation feels that they’re inventing everything.

AM: Yes; I have to admit it’s a wise newbie or youth who actually looks to some older people and says, “Oh, they might have something there; let me go ask.” A more typical opinion is, “I think I know better. This is all new. Nothing like this has ever happened before and I have unique insight into the situation.” That may be so. But I think we can learn from our elders.
Especially now that I am an elder, I naturally favor that opinion. …:-)…

TA: Of course.

AM: But I think, even younger, I did have respect for some of my teachers, my elders, who at some various times I could view as bizarre, whacko individuals. But still I realized I could learn from them.

TA: So speaking of youth; let’s think about that kid making rocket ships in your basement.

If you could go back and tell that kid about something that you were going to do in the future in your career, what do you think would be the most exciting thing you could tell that kid about that you’ve done?

AM: That I’ve done? I thought you were going to ask for a “word to the wise is sufficient.”

TA: No, actually, if he could have some preview of the work that was ahead of him, what do you think would have excited him the most that you would end up doing?

AM: I guess I might have thought to tell him I was an advisor to the U.S. Space Station.

TA: Oh, my gosh. You were?

AM: Well, my wife and I were invited in a unique circumstance to provide some color guidance for the interior for the U.S. Space Station.

TA: No way; that’s so cool.

AM: My wife, Leslie Becker, who is a noted graphic designer and teacher at the California College of Art in San Francisco, even flew down to the Johnson Space Center and continued to provide some guidance. She has even more experience than I do as a graphic designer also and knows much more about interior design than I do.

TA: You helped to design the interior of the Space Station?

AM: Well, we just provided some thoughts. Gentle and tiny tidbits of consulting.

TA: That’s still pretty stinking cool.
So finally, my last question, is what fascinates you today?

AM: 3-D printers and the Maker society, I find intriguing. Wearables are intriguing. A younger colleague of mine wants to be the Queen of Beauty Technology.

TA: Beauty technology?

AM: Yeah; lipsticks that can change color depending on your mood or face decorations that are actually electrically conductive or that sense your skin condition.

This particular person, now in Brazil, together with a colleague in Hong Kong, designed a headpiece that magnifies one’s eye blinking. So if you want to send a really strong signal to the other person, you can blink your eyes and your headset glows with LEDs that amplify the signal.

There are all kinds of things being invented that have to do with the body and outside the body that are intriguing.

As I‘ve mentioned, I’m also interested in all the work we’ve done with combining persuasion theory with information design in mobile technology to change peoples’ behavior; through these “Machine” concept-projects we are trying to figure out how a mobile device of some kind could help improve our bonding with other people, our general state of happiness, how we learn, how we drive, how we manage our finances, and how we tell stories.

Those have been the subject matter that we’ve been involved with on 10 different Machine projects. I’m trying to write that up into a book now called Mobile Persuasion Design, to be published next year by Springer.

TA: Sign me up. To be the first person on the list to buy that. Hopefully we’ll follow this up with another conversation on your work – your amazing work and extensive work – since we left off at 1984.For now I want to thank you so, so much for sharing these wonderful stories with us and for the work that you’ve done for the field. Thank you.

AM: You’re welcome. Thank you for the pleasure of telling stories.