Danielle Bragg
About Research Honors Publications

Danielle Bragg

Senior Researcher
Microsoft Research
danielle.bragg [at] microsoft.com


ABOUT ME


I am a Senior Researcher at Microsoft Research, in the New England and New York labs.

My research focuses on developing systems that expand acces to information, in particular for people with disabilities. My work is interdisciplinary, combining Human-Computer Interaction, Applied Machine Learning, and Accessibility. I completed my PhD in Computer Science at the University of Washington advised by Richard Ladner, and hold an AB in Applied Mathematics from Harvard. My work has been featured in the Microsoft Research Podcast and in The Economist.

EXPERIENCE

Current Senior Researcher, Microsoft Research
2018-2020 Postdoc, Microsoft Research
2013-2018 PhD Student, University of Washington
2015, 2016 Intern, Microsoft Research
2014 Intern, Microsoft Bing
2011-2012 PhD Student, Princeton
2010-2011 Research Assistant, George Washington University
2010 AB, Applied Mathematics, cum laude, Harvard

RESEARCH


I build systems that improve access to information by leveraging modern computing capabilities in innovative ways. My work combines human-computer interaction (HCI) with applied machine learning, in collaboration with interdisciplinary colleagues from computer vision, machine learning, graphics, linguistics, typography, and the social sciences. My work employs both quantitative and qualitative methods, using data collected through crowdsourcing platforms to iterate on designs, explore solution spaces, and solve data scarcity problems. I partner with target users to incorporate input and build community trust, and am committed to publicly releasing projects so those users can benefit. Below is a list of some projects that I have led.

Screen shot of ASL Wiki interface, showing a signer at the left and an article text on the right

ASL STEM Wiki -

To help advance sign language modeling, we created ASL STEM Wiki - the first continuous signing dataset focused on Science, Technology, Engineering, and Math (STEM). The corpus contains 254 Wikipedia articles on STEM topics in English, interpreted into 300 hours of American Sign Language (ASL). In addition to its size and topic, unlike many prior datasets, it contains videos of professional signers (including many Deaf interpreters), was collected with consent under IRB approval, and Deaf team members were involved throughout.

More: project page, wiki, paper below
Screen shot of ASL Sea Battle app, showing a signer at the top and a grid of tiles below

ASL Citizen -

Lack of data is the biggest barrier to developing intelligent sign language systems. To help advance research, we release ASL Citizen, the first crowdsourced isolated sign language dataset. ASL Citizen is about 4x larger than prior single-sign datasets and samples everyday signers and environments. It is the only dataset of its scale to be collected with Deaf community involvement, consent, and compensation. Alongside the data, we release a searchable dictionary view of the dataset and baseline models with code. Our dataset brings state-of-the-art single-sign recognition from about 30% accuracy in prior work to 63% accuracy, under more challenging test conditions.

More: project page, dictionary, paper below
Screen shot of ASL Sea Battle app, showing a signer at the top and a grid of tiles below

ASL SEA BATTLE -

Lack of data is the biggest barrier to developing intelligent sign language systems. To collect larger, more representative AI datasets, we propose a sign language game. Our game is a variant of the traditional battleships game. In our ASL version, grid squares are labelled with signs, and players guess by recording a sign, thereby contributing to the dataset. The opponent views the video and taps the matching square to reveal a response, thereby providing a label. The game also provides a fun and educational sign language resource.

More: recorded demo, presentation, paper below
Screen shot of a signer with a tiger head avatar replacing their head

SIGN LANGUAGE ML PRIVACY -

Sign language data is intrinsically personal -- typically consisting of videos of people signing, which capture the face, body, and surroundings. We propose applying filters to these videos to help address privacy concerns, boost dataset participation, and thereby potentially boost ML model performance as well. We experimented with various filters applied to the signer and surroundings, and ran both user studies and computer vision experiments to validate this idea.

More: presentation, paper below
Diagram of a wearable device, which consists of a baseball hat with an attached 3D-printed piece hosting a speaker/microphone, smart glasses, and camera with fisheye lens

CHAT IN THE HAT -

Virtual remote interpreting (VRI) makes sign language interpretations available through a video call. VRI requires the signer to hold a smartphone (constraining signing and multitasking), or to place the video call device in a fixed location (constraining mobility). We prototyped a wearable device to free the signer's hands and enable mobility while using VRI. The device consists of a baseball hat with a speaker/mic (English input/output), mounted fisheye camera (ASL input), and smart glasses displaying the interpreter (ASL output).

More: presentation, paper below
The word livefonts in several scripts, including an animated script

SMARTFONTS -

By redesigning English letterforms, smartfonts and livefonts challenge our assumption that we should read traditional letterforms on modern screens. Personal devices enable this challenge, by allowing users to adopt new letterforms without language reform or mass adoption. While smartfonts leverage color, shape, and spacing, livefonts add animation to the design space. Potential benefits include increased legibility, privacy, aesthetics, and fun. Try it out by downloading a smartfont and uploading it to Chrome's Font Changer extension, to render all your web content in a smartfont!

More: interactive demo, smartfonts presentation, livefonts presentation, smartfonts paper below, livefonts paper below
an animated ASL character for the sign UNDERSTAND
ANIMATED SI5S -
While ASL character/writing systems offer many potential benefits, none have been widely adopted. Time and effort to learn them is one obstacle to adoption. All past ASL scripts have been stationary (non-animated), which makes depicting movement difficult. We capitalize on modern animation capabilities, proposing the first animated ASL character system prototype. By representing movement iconically, our system reduces the need to memorize symbolic representations of complex movements.

More: interactive demo, paper below
Workflow of ASL-Search

ASL-SEARCH -

ASL-Search is an ASL dictionary that lets students look up the English meanings of signs. Looking up the meaning of a sign is difficult because a sign is a 3D movement not easily described with written words. Our dictionary lets users enter a set of features including hand shape, location, and movement, to describe a sign and look it up. The dictionary learns from the features that previous users enter to improve results for future users.

More: https://asltoenglish.org (beta release), paper below
Workflow of ASL-Search

ASL-FLASH -

ASL-Flash is a site that both helps people learn American Sign Language and provides featural descriptions of signs. The site provides "flashcards" of signs, showing visitors videos of signs and quizzing them on the English meanings and compositional features (e.g., handshape and hand location). The data that users provide helps us build the ASL-Search dictionary. Check it out at www.aslflash.org and learn some signs!

More: https://aslflash.org, paper below
Screen shot of the study, with the words 'Press play when you are ready' and a large play button

LISTENING RATES -

We provide the first inclusive, large-scale studies on human listening rates. As conversational agents and digital assistants become increasingly pervasive, understanding their synthetic speech becomes increasingly important. Our studies presented volunteer participants with a series of challenging listening questions in order to learn more about their own listening rates. Our results inform synthetic speech rate optimization and future inclusive crowdsourced studies.

More: initial study paper below, expanded study paper below

HONORS


SELECT PUBLICATIONS (Google Scholar)


CONTACT DANIELLE

Click to email me.