UX design

Adobe

Advocating for the responsibility and accountability of equitable, ethical, and accessible generative AI in creative workflows, particularly for creatives with cognitive or intellectual disabilities.

Some mockups of Connie's work at Adobe on Firefly.

Role

Experience Designer

Product Strategy

Duration

May 2023 - Aug 2023

Team

Timothy Bardlavens

Es Braziel

Annika Muehlbrandt

Lexi Hradisky

Tools

Figma

Adobe XD

Overview

Resculpting creativity in the age of Generative AI

During my time at Adobe, I worked as the first Experience Design intern on the Product Equity team. I was tasked with examining exactly how well Adobe Firefly helps artists with their creative processes, especially through the lens of equity and accessibility.

My full designs are under NDA, but read on for a summary of my process and findings!
A jaguar being circled and identified in Adobe Firefly.

Background

Generative AI is making creativity easier... or is it?

Firefly is Adobe's generative AI web application, committed to be being a responsible approach to the emerging generative AI boom. It can be used for the creation of images, text effects, color palettes, and aims to support and improve the creative process of all creators.

However, is Firefly really meeting every creative where they are, and truly making the creative process easier for everyone?

The words 'Prompt Engineering' in large lettering, surrounded by other related terms.

Community Feedback in Beta Testing

Screenshots of conversations with Firefly users where the users are confused on how to write good prompts.

Users in community forums and surveys expressed challenges with writing prompts in Adobe Firefly, with difficulty ranging from the learning curve of prompt engineering to existing learning disabilities.

A list of things users have to know before being able to effectively use generative image technology.

Plethora of skills needed to best utilize generative image technology.

Problem definition

Generative technology remains inaccessible due to its learning curve

Through feedback collected from Firefly's beta users and social media forums about generative creative tools on the market at large, we identified two main problems with the existing mode of prompt building in Generative AI tools:

Pain point 1

📚

Requirement of written literacy

Through feedback collected from Firefly's beta users, social media forums about generative creative tools on the market at large, we identified two main problems with the existing mode of prompt building in Generative AI tools.

Pain point 2

📈

Learning curve of prompt engineering

The way prompt engineering is designed today requires deep knowledge of language models and how they will interpret any given written prompt, making it inaccessible to far more people than just those with cognitive and intellectual disabilities.

How might we enable all users to craft effective prompts for generative AI without (prompt) literacy?

Solution

1. Enabling multi-modality in prompt building

It was important to incorporate multiple assistive tools into existing product interfaces. Offer more methods of entry to build prompts, including voice and image.

A mockup of a workflow where a user is able to submit a reference image for their query in Firefly.

Image reference

A mockup for a voice-to-text prompt generator.

Voice to text

Disclaimer:

Pictured visualizations are not the final designs, which are protected under NDA.

2. Deconstructing existing paradigm around learning prompt engineering

Reexamining the way creatives are expected to learn prompt literacy, we propose adopting an ability-based design framework, focusing on what ours users can already do, not what they can't. As designers and developers, we can meet creatives where they are by accommodating their abilities and shifting the burden of prompt literacy onto the system instead.

What does it mean to teach someone prompt literacy? How can we eliminate that step for artists altogether?

Considerations

Who bears the burden of prompt literacy?

How can we meet artists where they are, instead of making them adapt to environments that aren't suited to them?

A diagram showing the spectrum of where burden lies from the user to the system.

MY DESIGN NORTH STAR (and awesome team)

Product Equity, and what we stand for.

Adobe's Product Equity team a new and nimble cross-disciplinary team that focuses on committed to changing the way products are built within the company and, ideally and over time, across the industry. We examine product research, design, and development processes throughout the company, and partner with teams throughout product development lifecycles.

The Product Equity team helps make decisions that consider the full spectrum of gender, race, age, ethnicity, ability, culture, and all other human variabilities.

Read more about the Product Equity team's work here.
An abstract illustration of various concepts related to product equity, such as an eye with a cross through it.

Product Equity Illustration by Abby Kwon

Design process + deliverables

How might we make it easier for people with cognitive and intellectual disabilities to use generative tech in their creative practice?

And in turn, make prompt building easier for everyone?

Phase 01

Understanding Adobe's creatives

Weeks of primary and secondary research to understand what assistive tools creatives with cognitive and intellectual disabilities utilize in their creative practice, and where generative tech can assist.

Phase 02

Introducing assistive features

In close collaboration with researchers and stakeholders, iterating on prototypes derived from collected research findings to incorporate assistive tools.

Phase 03

Structural product changes

Further iteration and dismantling the entire model of prompt building to propose a new framework of approaching accessibility. Showcase to entire company design org.

Reflection

Equitable design is better design

Good design is design that is able to apply to all. Focusing and solving the needs of a specific group can improve the experience of everyone, and no audience is too auxiliary. The work that the Product Equity team is doing at Adobe is transformative and pioneering in ways that I hope will grow and set the status quo across the entire industry.

Various photos of Connie with her team.

Autonomy and advocating for myself

As the first design intern on my team, I had both the opportunity and challenge of carving out my own role and position within the company and taking agency over my work. I learned how to establish strong support networks for myself and find mentorship whenever I needed it, and in turn met so many amazing people and learned a lot of self efficacy. I've since been overwhelmed by the amount of kindness and support shown to me by so many people I've met.