Claire Ferrari, PhD
All work

05 · Google Creative Lab, with NYU Ability Project · 2018–2019

Creative tools for people of all abilities

I led user research with 25 people with disabilities for a set of accessible creative tools that won the 2019 Cannes Lions Design Grand Prix.

Org and Year
Google Creative Lab, with NYU Ability Project, 2018–2019
Role and Team
Lead researcher, accessibility specialist and creative technologistTeam of 5
Methods
User research with 25 participants with disabilities, multimodal prototype testing
Creatability, shown over a darkened photo of a hand on a laptop keyboard. On screen is a Creatability experiment with a Keyboard settings panel, body tracking options and large colored keys labeled C and D.

  • Video, 8 seconds, no sound: a person faces a webcam and draws looping black lines across the frame by moving their nose, tracked as a blue dot.

  • A person in a striped sweater stands in an office while yellow zigzag lines from body tracking trace their head, shoulders and arms.
  • Colored note columns labeled C, D, E, G, A and C over a webcam view of a person. A blue line from their nose points into the D column, which is highlighted orange.
Creatability – a series of accessible creative tools.

How do gesture, sight and sound work as inputs for making art and music, for people with diverse physical, sensory and cognitive abilities?

Context

Creatability is an open-source set of experiments that use machine learning to make creative tools more accessible, built by Google Creative Lab with NYU's Ability Project.

Approach

I led user research with 25 people with a mix of disabilities: people with cerebral palsy, people with fine and gross motor disabilities, members of the Deaf community, people who are blind, and neurodivergent teens. We tested new interaction modes as they were built, working across research, design and engineering on a team of five.

Example: Sound Canvas

With Josh Miele of the Smith-Kettlewell Eye Research Institute, I co-designed Sound Canvas, an experiment that lets people draw through sight, sound and gesture. Blind and visually impaired design and usability experts were involved at every stage. Key design decisions:

  • Several ways to draw: a mouse or trackpad, the arrow keys, or a point on the body such as the nose, so people with a range of abilities can take part
  • Keyboard commands for every important control, including a Go-To command that jumps straight to a position on the canvas
  • Sound that maps the canvas: stereo panning for horizontal position, pitch for vertical position, and a Where Am I command that announces the cursor position
  • Two ways to explore by ear: moving across plays a whole column at once for an overall sense of the drawing, and moving up or down plays single points for exact placement

Read more in Drawing With Sound, the article Josh Miele and I wrote about the design.

What changed

Research findings guided the design of the tools. The project went on to win the 2019 Cannes Lions Design Grand Prix.