What led you into AR design?

I think it all kind of fell into place with all the things I grew up with and was interested in. I was lucky enough to grow up with access to a computer and played a lot of games, specifically flash games, Neopets, and mmorpgs. I made websites on Geocities, Xanga, I posted art on Deviantart.

Then in school I went to a more traditional art school and studied print graphic design, I struggled to find a focus I was kind of a mess actually — I never officially minored in anything but I dabbled in illustration, animation, packaging and actually ended up weirdly focusing on screen-printing and printmaking and interestingly those were really tactile physical objects — that’s one aspect. The other aspect is that at this time this was also the era of Tumblr and LiveJournal and I loved making themes and tweaking CSS. I realized I still loved making things move digitally. So you can kinda see where I’m getting at here.

So fast forward to now, I had a few design jobs before Google but 5 years ago joined the Lens team with the goal of getting back into what I really loved doing, making virtual things come alive. I think a combination of my background in designing physical objects and my personal interest in making things move digitally led me to an intersection of where I am today.

I think it’s exciting to be a designer at Google right now, particularly working on Lens. We’re super focused on making it as natural and intuitive as possible for people to explore information about the world around them, which opens up so many new design possibilities and opportunities.

Google Lens allows anyone to point a phone at text, and hear that text spoken out loud.

What are some pieces of work you’re proud of?

At I/O last year, Google announced an upcoming feature called scene exploration that ties into the effort of making it more intuitive to Search the physical world around you.

Here’s how it works: say you’re trying to buy a new snack at the grocery store, all you have to do is pan your camera in front of a grocery shelf and right away, you can get helpful insights about what you’re seeing so it’s easier to make a decision and find what you’re looking for.

This project definitely was super fun to work on. It gave me the opportunity to help bridge the physical and digital worlds, to contribute to making powerful breakthroughs in our devices’ ability to understand the world, and to make all of this experience as natural and intuitive as possible, so that everyone can use it. I’m really proud of my team for helping make all of this come to life.

The second piece of work I’m really really proud of and is super special to me, is building a camera-based tool designed to help people who face reading and other language-based challenges. There are more than 800 million people in the world who struggle to read and this was one of the first Lens features I got to work on to help solve this problem.

Imagine moving to a new country and not speaking the language, trying to fill out a form in a different language, and worrying if you wrote something incorrectly could be a terrifying experience. And then there’s a second problem, some people may not be able to type words into Google Translate because they may not know how to type in a new or different language. Then there’s a third problem, even if text is translated into English or a language someone understands, they still may not know how to read it if there are literacy challenges.

With this feature, all you have to do is take a picture of the form with this camera on your phone, and the image will be translated. All the text will be transformed in its place, and the phone will read everything back out loud to you with audio, kind of like karaoke. Lens basically combines computer vision, machine learning, Google’s Knowledge Graph, and an extremely simple and delightful visual-forward UI so that people who might struggle with reading or a second language would understand how to use this camera. All of this had to also work seamlessly on a low-end phone that costs $20.

At I/O 2019 we showed a video of a real Lens user and I think it did a really good job at telling the story of the impact this project has made in someone’s life. I remember tearing up after watching this video. I’m grateful to have worked on a project that made such an impactful difference in people’s lives around the world.

What design challenges do you face at your company?

I think as a designer in an innovative space, since there’s not a lot of existing research or products out there you can reference, a lot of it relies on Google’s history of technological innovation, how we’re able to learn and understand how humans interact with technology, and leveraging that to create radically new and helpful experiences for people around the world.

So an important aspect is research. This is still such a new space where we’re still improving our understanding of how people interact with spatial experiences. There are still questions around social expectations around using your camera in public, or safety when people are walking and using their phones simultaneously, and even behind all of these learnings how do we make it really easy for the people behind-the-scenes like designers and researchers to test their hypotheses without so much overhead.

Comparing it to 2D UI design, the industry didn’t really blow up until Sketch came along, there was Photoshop and Illustrator but those tools weren’t totally meant for UI design at that time, people kinda just hacked things together to make it work, but when you send it to a teammate everything breaks. Then more scalable collaborative tools like Figma came, where you didn’t need an expensive Macbook or pay for expensive and intensive software to create. Everyone was able to move together faster.

So when you’re working on spatial design at a large organization, you have to collaborate with your teammates, your design needs to be iterated on, shared with researchers, shared with other designers, engineers, product managers… Everyone’s busy and not everyone’s gonna have the bandwidth to go through all the setup required. So I think we’re kind of at that little inflection point where there’s a need for the Figma equivalent for spatial design. There are existing tools and attempts in creating such niche tools, ultimately I think the industry is still missing a collaborative spatial design tool used at scale that allows designers to quickly iterate, collaborate with researchers and engineers, and seamlessly share their work wherever they go on whatever device they’re on.

Any advice for ambitious designers who want to get into spatial computing design?

I think the advice I have applies to many other things too, but I’d say to develop and trust your intuition. There are endless resources on the internet and there are so many books out there you can read, but personally I think, as a designer in an ambiguous space where you’re exploring new innovative tech and trying to build it at scale, you can’t rely on what’s existing, cuz you’re the one trying to pioneer something, you really have to depend on your intuition.

And to start building your intuition whether it be design or just being a human, like think about how do humans use things and interact with the world? You’d have to experience it yourself, struggle with it, build and intuition on how something might be better. Try out how other people built their experiences, and then maybe try to reverse engineer how other people built certain experiences, there’s a lot of resources to learn how to use these tools, like Bezel! I think the best way to learn is to understand something, experience it, deeply understand humans, build an intuition and then try building something yourself.

What’s the most exciting thing about Bezel for you?

Similar to what I said earlier, I think the familiarity of Bezel to other existing design tools makes the learning curve easier, and because it’s web-based you can use any device to create, the crazy magical ability to easily preview in the spatial environment you intend it to be in so that YOU can immerse yourself in that experience, and share it with someone simply with a URL!

And they can open it on any device, making it so easy to iterate, test, have people interact with your design, give feedback, it makes the whole design process faster to validate. I think instant preview in XR simply with a URL is so exciting. I feel like Bezel is at a similar stage as Figma was right before it became a widely used tool in the industry.

***

Bezel is a collaborative 3D design & prototyping tool for spatial computing (AR/VR/XR) on your browser.

👉 Free access to Bezel (beta): bezel.it
🐦 Follow us on Twitter: @Bezel3D
👾 Join our Discord community: bezel.it/community