Tooling up

The road from design to production is often hard-won, filled with well-intended design detours, steep learning curves, and the occasional road block. Here’s a common scenario: A team of designers begins building wireframes for a new product. Several weeks, revisions, and pin-ups later, they emerge with a “final,” “shippable” file, which they hand off to the development team. The engineers poke holes in the design, deeming several elements “unbuildable,” and begin rendering out some semblance of the design in code. Several weeks later, they deliver a staging site. The designers are thrilled—this is the first time they’ve seen their design IRL—but are quickly deflated when they realize how much of the design has been lost in translation. The typography is wonky. The animations and interactive elements need to be re-tuned. A volley of annotated emails with screenshots ensues, until finally, the designer and developer sit side-by-side, and visually adjust the design directly in code.

More often than not, this entire process is repeated several times.

“In my experience, 60% of design happens during implementation, and a lot of the original design winds up not working,” says Owen Otto, a UX designer at Google. Otto participated in a field study in which he interviewed numerous designers working across the digital space, he found the pain points to be almost universal. When it comes to creating rich, interactive experiences, where narratives are nonlinear and so much is determined by the user’s input, designers’ tools are woefully limited. “They’re using hacks to communicate motion,” says Otto. “Sometimes they just draw an arrow, saying, ‘This slides up.’" But a new generation of technologists are heeding the call. I sat down with Paul Colton of Pixate, Max Weisel of Relative Wave—both innovators in the field of digitally native prototyping who’ve recently joined the Material Design team at Google—and Matias Duarte, VP Design at Google, to discuss the challenges and opportunities in creating the next wave of design tools, and the ongoing quest to realize the WYSIWYG.

Amber: The problems plaguing designers in the digital space seem to be pretty complex. Did a lot of the impetus for building these tools come out of your own personal experiences and frustrations?

Max Weisel, Form: Totally. Relative Wave started out as a contracting studio, creating apps for musicians like Bjork and Lady Gaga. We found that when it came time to design and build a lot of these apps, most of our time was spent doing tedious things. Our iteration process was slow because our designer would take a first pass at something, and it would take us a few weeks to implement it to see if it was actually good or not. Form was borne out of us wanting a tool that not only allowed us to iterate more quickly and efficiently, but also would allow our designers to test different ideas and use different technologies without having to ask the engineers if certain things were possible or not.

Matias Duarte, VP Design at Google: The scenario Max just described—I have seen it so many times at different parts of the industry, with different sized teams and experience levels. You get people together to try to talk about an idea, and the designer comes in with an artifact that is so limited—just a couple of static images. There’s so much that happens in between those images, like how the flow of those images connects, that is only clear in the designer’s mind. As a designer, you’ll have meetings where you’re sure everybody is talking about the same thing, and then, usually right before time runs out, you realize, ’Oh my gosh. They totally didn’t get it.’ There’s just this huge gap in understanding how things are supposed to move, or how one state connects to another state. That’s a real problem. It’s really hard to communicate what’s in your head. It’s much better to actually show it.

Paul Colton, Pixate: Pixate’s mission starting out was: How do we make it easier for designers to do the things they do well, with the flexibility they’re used to, and enable them to do it easily for native mobile apps? We also asked companies, what are the fundamental pain points you have building and designing your mobile apps? Across the board, there was a consistent theme: It’s difficult to take ideas and drawings and sketches and bring them into reality. What we really wanted to figure out was, how do we get these ideas into something that someone can hold and use and touch in the native way?

Amber: These tools are not only helpful to designers for testing their ideas, they must also be useful for selling ideas to clients and collaborators

Max: We definitely used Form in that way, but I think for us it was more just a tool to help with our own thinking. A big mistake we made early on was sharing unpolished work. Clients would be like, "It’s going to be better than this, right?" It got to the point where we didn’t show anything unless it was either clearly a sketch or a nearly finished prototype. We would even take nice prototypes and make them look more like a sketch in order to show our clients. Form makes it easier to share your imagination.

Matias: It is kind of true. It’s kind of like Michelangelo coming up to the Pope and being like, "See this big rock? It’s going to be this beautiful statue! Trust me." He’s going to be like "Okay... Maybe you could ... I dunno ... draw me a sketch of what this statue is going to look like?" Maybe I can imagine it, but I’m not sure what’s in my imagination is the same as what’s in your imagination.

Max: It becomes even more difficult when you start getting into engineering. Our burden was if the designers wanted to show some sort of complex motion, then we had better allow them to show it exactly the way they’d imagined it.

Paul: The other frustration is how can designers “round trip” their ideas? In other words, how does the designer push forward with his or her design concept while internalizing engineering needs and constraints? Our goal going forward is, how do we continue to close that gap?

Amber: I like the term “round tripping” because it addresses the converse side of the problem—making designers better at creating experiences that are plausible in code.

Paul: One of the reasons for us to be native was exactly that. When a designer can draw anything, let’s say, in Sketch or Photoshop, it’s so easy for the developers on the other end to say, "It can’t be done” or "it’s too hard." We wanted to enable the designer to take their idea so far that, with quite a bit of ammunition they could say, "It’s not necessarily easy, but it can be done, and here is the proof running on a device, natively." That’s a huge step for empowering design.

Amber: Do you feel like essential elements of the design process are sacrificed when you try to shoehorn print-based design tools into building interactive products?

Max: I think it depends a lot on the problem you’re trying to solve. If you’re trying to solve a design problem in an area that’s very unexplored, you almost can’t expect to have a proper tool. A good example is designing for VR. Nobody knows how you’re supposed to really interact with it. A lot of the iteration and seemingly tedious parts in the design process are just a fact of life. You’re going to have to bend an existing tool to work towards what you want.

On the other side of the argument is mobile wear. A lot of the interaction patterns have been figured out already. Instead of wrestling the tool, you can now spend time prototyping higher-level ideas or maybe a few forms of very unique interaction that you would need in your app.

Amber: How do you ensure that design innovation isn’t limited by the tool?

Max: You need a tool that has common patterns for the people who are not trying to create something new, so they can work quickly and solve the problem. But you should also have broad tech that allows the smaller percentage of designers who are trying to create new forms of interaction, who are trying to solve very complex mobile design problems, to create new forms from scratch.

Matias: This isn’t a totally foreign problem in the toolspace, right? Things like Photoshop and many other creative tools have taken a position about this in the sense that they are modular and extensible. If that set of things is intrinsically extensible, then you can have the best of both worlds like Max is describing.

Paul: With Photoshop, most of the palette items haven’t changed in 20 years. Even the icons have barely changed. If you just focus on the basic building blocks, you can always use them to build something more sophisticated—don’t try to inform what that pencil tool can do, just make it a really good pencil tool. It’s very important for us that not everything made on Pixate looks like it was built on Pixate.

Amber: What is the relationship between building this system of tools and the material design guidelines?

Matias: They’re all trying to solve the same problem, this problem of getting a better software experience out there. Things like having a coherent framework and guidelines for gaining efficiency and learning and establishing expected patterns, just like Max was saying. That’s one step we can do. Getting designers and developers to really understand basic motion principles and invest in them, that’s also something we can do and evangelize through guidelines.

But the rubber meets the road, when it comes time to actually implement those things. It’s very hard for designers to communicate those motions and interactions, or to develop them, so creating new tools is a natural extension. It’s almost like if you pick up the material guidelines and it’s the instruction book for how to fish, and somebody’s like "Okay, great, but I need a fishing rod." It’s like, okay, great, let’s go make some fishing rods! It’s the next logical step. I think across all of those efforts, we have the same philosophy. We don’t want to make a tool for one kind of experience, one kind of application. We don’t want to make a tool that is only useful for material design. We believe very strongly in this idea of making the system resilient, and organic, and composed out of lots of pieces that work really well together. If you don’t want to take them all, you’re still going to get a lot of value out of any of the pieces that you do take—we still consider that a big success.

Amber: Paul, you brought up the idea that many of our design tools haven’t really changed in the last 20 years. How will you all ensure that these tools address new problems but are still intuitive enough for users—that the onboarding isn’t a deterrent to adoption?

Paul: We just try to err towards less is more. As an example, we only have a single primitive. It’s called a layer. People keep asking, give me this, give me that, and we probably will at some point, but the interesting part is at first we only did one thing because that’s all we had time to build. But what we found—almost without exception—is we haven’t held anyone back. They have been able to do almost everything they need to do with this single primitive. That speaks to the point that, if we make it simple but powerful, it makes the onboarding easier. There’s not 50 things to learn. There’s just one core concept that you have to learn well. There is education that has to happen here because some of this content is brand new— like bringing interaction and motion as a core part of an app.

Max: The way I look at it is, if your tool doesn’t make any strong assumptions about what you’re going to create, it’s not going to date itself when what you create changes. When I look at the set of tools that I have in my garage for fixing stuff around the house versus the ones that my dad had when he was growing up, they’re almost the same. The things that I do with them now are different, but they’re tools for a very specific purpose that have very broad application. I need to look through the tools in Photoshop again, but I think a lot of the tools don’t make assumptions about what the user is going to create, and they can be used for so many things. They haven’t changed because they haven’t needed to change.

Amber: Do you feel that if the tool is working properly, it’s going to be able to allow designers to do what they’re best at, as opposed to being overwhelmed by learning new technologies?

Matias: This is an interesting challenge, and it comes from the fact that this problem of motion and interaction is newly critical to our discipline and to our industry. I think actually it’s putting a strain and crisis on many people. If you went to school and you picked up the fundamentals of graphic design and layout and had those mental concepts in your head, then you come to a tool—say Photoshop, Illustrator, or Dreamweaver, whatever it was back in the day—and you’re immediately in the right head space. You’re just learning the tool itself. To use Max’s analogy of tools in the garage—all you’re worried about is learning the difference between using a manual screwdriver and an electric drill. You’re focused on the details of the tool, but you have an orientation and a framework for the problem you’re trying to solve. Once you start dealing with some of these questions of motion, and particularly interactions, you’re faced with this double whammy of both trying to learn the tools that will allow you to express them, and also the fundamental concepts. That’s hard because so many of the people who are working in this field right now barely have time to orient themselves to a new tool, much less familiarize themselves with some of the more underlying concepts.

Paul: After Effects compared to Photoshop looks so scary—even to me—because I’m most familiar with static design. It’s very, very daunting. What’s really sweet about, Pixate’s approach is, we actually went beyond the timeline. Right when people thought, okay I get it, still images over time. We said, no there is no time anymore, it’s all about user input. That is another big jump for users. We said, what if things animate based on gestures, and you can control the gesture? It may go forward and back, not necessarily in time, but definitely in position. That introduces yet another layer—your users are the timeline. Their fingers on your UI, are what dictates what happens next. That is another leap in the design space.

Matias: It’s not only just that the user is the timeline—it’s like a four-dimensional timeline. They’re providing other input that could be going anywhere on the screen. Or they could be providing input with multiple inputs at once. Their finger is moving one way on the screen, but they’re also rotating the device. That’s different than just simply designing a single piece of motion.

If somebody’s coming from a world where they’re just thinking about one static layout at a time, even if they’re given a tool that maybe has this capability to do something richer, their instinct will be to just be like, “Okay, I’m going to wire this one state to this other state”. Maybe it’ll have an animation, but it’s not going to be the kind of rich interaction that’s possible today. We need to do both. We need to provide an on ramp, but also we need to recognize that no matter what we’re doing in terms of the tools, that we are drawing designers into a new space that that we’re all kind of discovering together. This is new for all of us.

Amber: The narrative a designer has to think through and design around is infinitely more complex and variable—how does designing with real data help solve for edge cases and user flows?

Paul: They’re not even edge cases anymore! I mean, did you really think that when you designed that high DPI, very large image on your 30-inch display, something would have to run on a 1-inch screen on somebody’s wrist? It’s not just about designing the interaction. It’s about how this interaction will play out across different form factors and sizes we don’t even know about yet, or may not be able to control. You’ve got so many new inputs that get dropped on someone who is used to working in a static, two-dimensional world.

Matias: This is another good point. Many teams, particularly here at Google, really rely on real data. For example, our search team can do some static studies and interactive motion studies, and then go and build an interactive prototype. At this point, there’s hardcore engineering using data to help make design decisions. That’s the thing that gets the product validated before it goes into production engineering. I mean, it’s a super robust process, but it’s extremely expensive. That’s why it’s so cool to see things like what’s in the Form SDK—to have that same capability that the search team has, to create these plugins, which can pipe real data from your systems or from any open API’s and get that to be part of your exploration—that’s so exciting to me.

Max: For every design problem there’s a different way to test all of those edge cases and understand whether your design is going to work in practice. A big focus for Form recently has been designing in context, so users can test their design on a device in the wild. A lot of the tools that we see today don’t have the support for any of that. We have had designs that used lorem ipsum all over the place. We had built a commenting system for a client app and when somebody comes in and writes a one-word comment, the design actually kind of looked pretty terrible.

Amber: Max, Paul, what excites you most about bringing Form and Pixate to Google?

Max: I think the big excitement for me is that the problem we’re trying to solve is so huge. We’re trying to make it easier for everyone who designs for mobile. That’s a problem you need to be addressing at a company that has the resources, and the thinking, and the talent to be able to work on that scale. When it comes time to test our theories on large volumes of data or users, like even just dogfooding internally, it allows us to really test a lot of our theories before they even see the light of day.

Google is operating at the same scale in which this problem exists and has all of the ability to solve it well. Based on what we’ve seen in previous years with material design, they’re already headed down that track and feel very strongly about it, and they are executing pretty brilliantly. We want to be a part of that.

Paul: I enjoy being as disruptive as possible. I think it would be a lot harder to do that at the scale we were at versus the scale we’re at now joining Google. Change requires a lot of effort. Some of this is brand new thinking, and we want to be disruptive to the status quo of how things have been done, and also disruptive in how things will be done in the future. It was our goal even as a very small startup, and now hopefully we will continue doing that. I think that’s the opportunity. How do we make it better for everyone to work in this area and frankly to short circuit much of the nasty processes that are out there.

Amber: Matias, how do you envision these products in the larger context of the future of design at Google?

Matias: It’s all a part of this mission to make it so there can be wonderful digital interactive experiences out there. That’s the whole reason we’re doing this. We’re doing this to raise all boats. We’re going to leave no stone unturned in our pursuit of that effort. We put out guidelines, we host workshops, we host conferences. We’re going to give you tools. We want to get out there and we want to have impact at a massive scale using our unique leverage because of who we are and what our touch points are because we really just want to make design better. At the end of the day, it’s all about what the designer is going to create, what the developer is going to create—what’s going to actually get out there and touch users lives—and we’re passionate about making all of that better.

Amber: In that aim, is there any anxiety about keeping design up to pace with the rapid speed of technological advancement?

Matias: It’s about the entire creation process. You know, it’s funny, design is one of those words that very few people agree on—it’s a loaded word. To me, design is really anything that is a process of imagining and understanding a problem, conceptualizing a goal, and then trying something to satisfy that goal, and doing that figuratively. That is the practice of design—it is a considered creation. There are plenty of examples of things that are created out in the animal kingdom, but considered creation is unique to humans. I think that’s the thing that is worthy of being called design.

When you deliver a product, whether it’s via the web or via an app—or via whatever comes in the future—the larger endeavor is a design process. Inside that process, you have people who are designing and implementing more technical things, and people who are designing and implementing aspects of the usability, and other people who are really focusing on the emotional experience or the comprehension. There’s so many different people contributing in their own way to the design, to the ultimate solution. That process is really hard in a number of ways. Part of the challenge is how different parts of the teams work together. How do you keep the things from falling between the cracks? Part of it is how do people explore and ideate how things like state or interactive motion are going to play out, if they don’t have good tools to do the interaction (visualization, sketching, experimentation). All that is in service of this larger problem of—we just want to make things. We want to make good things. That’s where I think the investment and the problem is. I feel like if you look at other industries, people have tools and processes that are refined and much more efficient and you’re not so frustrated about things falling between the cracks or people misunderstanding each other. Looking again at print design, which is largely done with digital tools today, the process and the craft has been really well honed over hundreds of years. In contrast, these digital interactive experiences, these stateful experiences, they’ve only been around for a couple of decades. It’s not a surprise that we’re still creating these first tools. We’re at the era of having a printing press, but nobody’s actually got it figured out that you need to run a proof first.

Amber: That’s a nice analogy. Do you feel like these tools are going to are going to start to leveling the playing field in terms of design specialization?

Matias: If anything, this space is going to make it worse. I remember back in the early days of the web.com boom [in the early 2000’s], when people were worried about the fact that interaction design or information architecture was a thing that nobody had dealt with before. You had people who had been on print and information and communication graphics for ages, and the idea that you might have a nonlinear experience of text and images was completely foreign to them, and they needed to rationalize and understand how that fit into that world. Now, the idea that you might have experts in motion or interactive motion or interaction through states, makes it even more complex. I think it’s going to throw people for a loop. It’s like any rich discipline. There are some kinds of roles that, in order to be effective, need to be really broad. If you’re going to be a good movie director, for example, you need to understand all the elements that go into making your movie. A really great director—even a very hands-on director who has a really distinctive cinegraphic style, or who is a very strong editor, still hires a proper editor. There are still specialists who go deep into these topics because there’s a lot of work to be done. The debate is completely wrong. It’s not an either or question. Our craft is maturing and all of these roles are maturing, and you’re going to see people who become and can succeed at being proficient across some of these boundaries, and other people who succeed at being really specialized.

Max: I think, a lot of it, at least from what I’ve seen, kind of comes down to team size, too. I have a friend who is a graphic designer in New York, and he has worked at design firms ranging in size from five people to hundreds. I asked him, well what’s the difference? Do you like one more than the other? He said the biggest difference is, on the smaller team, you had to be good at everything. On a larger team, there are people who specialize. Ideally, better tools mean you get both better generalists and better specialists.

Paul: We’ve talked to a lot of designers who’ve told me, “Hey I’m going to have to go learn X code so I can do what I need to do.” That’d be like me saying, I’m just going to learn real quick how to draw really well. One, you’re trivializing the effort, and two, there are better ways to use your time. You can empower a designer with better tools and facilities, versus them having to jump completely to the other side of the process, which is what a lot of designers are having to do today. In a way, yes, we’re going to level the field. I want to see the engineering side peek on over and say "Can I use that?" Seriously, that’s the goal, right? Everybody wins.

Amber: As gestures become almost canonical with a product’s brand identity— the one that everyone always references is Tinder’s “swipe right” to say yes—I imagine these tools are going to become even more essential to the design process.

Max: You could argue that the swipe right and swipe left gestures weren’t necessary — it’s not even necessarily a better solution to that problem—but it is interesting how it completely defined the brand and had such a large impact on the culture in general. I crack jokes about swiping left and right all the time! There’s definitely something interesting in having unique forms of interaction, even if they’re not functionally better.

Matias: This is something that we found in our research early on that underscores the point of why you don’t just want to be hitting things on the screen. There’s actually a psychological and neurophysical difference in the kind of engagement that you have when you’re hitting something, batting it and waiting for it versus when you’re engaged with something and you’re moving and you’re sliding things around. It’s much more positive, and engaging, and delightful, and stimulating.

Amber: As you evolve these tools to have new features, how do you determine what will be valuable to designers—what will be a lasting feature and what might just be a fad?

Max: Everyone on our team works closely or hangs out with other designers and engineers. You can kind of get a good feeling of where their frustrations are and where they actually get stuck over a few beers after work. Sometimes what people find frustrating is something they kind of enjoy doing, but it takes them too long, so if it took like 10 minutes, that’s huge. You may never know unless you look at what people are actually doing and where they spend most of their time.

Amber: People can be unreliable narrators, too.

Max: Yeah, exactly. Sometimes, you might do something tedious and it takes you thirty seconds, and you complain about it all day. However, you spend fifty five minutes doing some large design and it’s probably fun. But the tool could be improved so you could do it much more quickly.

Paul: For us, there are two kinds of features. The ones that everyone knows they want, and then there are the processes you discover. The harder ones are the features they don’t yet know they need that are maybe game changing, or just different. It’s so easy to do all the things that people ask for, whatever priority you’ve given them. The real game changing features are the ones you’ve just got to bet on. You’ve got to say, I know designers don’t know they need this yet, but I can see how it’s going to change their lives in a meaningful way.

Matias: That’s why we’re doing this. Because we see a need that is unmet, and we really want to help out.