Portfolio by Kevin Yang






Making images on the web accessible.




Problem

“I want to understand the images on the web just as well as my sighted friends and family"


Alt Text Demo

alt

This image has no alt text.

two Samsung phones. Text: International brands, shop now link.
alt="two Samsung phones. Text: International brands, shop now link."

This image has alt text.



Data

Alt Text %

graph titled: percentage of images without alt text. To read more visit the Medium article we wrote!

Users' Grievances

@_Red_Long

I’m a blind [internet] user. There are a lot of us out there. Increase your ability to reach us and help us interact with your pictures, it’s really simple and makes a huge difference to our internet experiance allowing us to see your images our way.
Source: https://twitter.com/_Red_Long/

@damonrose

... any chance you could put that text in something other than a graphic? Blind users can't read pictures but can read regular text with screenreaders. Can it be added in an alt text field
Source: https://twitter.com/damonrose/

Facebook

... blind people are interested in visual content on social media, but they often feel frustrated and even excluded or isolated because they can't fully participate in conversations centered on photos and videos.
Source: https://code.facebook.com



Solution

Iteration 1

sample email to describe@include.ai including link
sample email response from describe@include.ai
sample email response with labeled images from describe@include.ai

Iteration 2



chrome plugin information about include.ai's chrome plugin



screenshot of chrome plugin getting started page


screenshot showing the chrome menu bar


Labeling Images



screenshot of label.include.ai, where volunteers can help label images

Findings

  • Having a detailed description of the image is not always important. Often times an alt tag contains a brief description of the image containing unique characteristics such as person, place, or thing being depicted, an action they might be doing, and/or text included. With most websites the most frustrating images or graphics are the ones used in place of an interactive element i.e. button, link, or toggle. For someone using a screen reader we hear, "G1024.jpg" rather than "Submit".
  • Generally I’d think all images being described is always better. That way future users can access the same page, but different images if necessary
  • If the picture is of Jane or John wearing a red shirt smiling outdoors, we really do not care who Jane or John is, only that it is of a male wearing a red shirt outdoors smiling. For this high level description computer vision is sufficient and can be extracted and added to an alt tag if one does not exist.
  • Facebook has added some CV to images posted on profiles. It will tell you how many faces and if they are smiling. This is some what useful but really does not provide any beneficial information to non-visual users. A better way to do this would be to utilize the tagging feature facebook already has for images on facebook to add a name to the faces smiling in the picture. For example if it is going to automatically tag John in Janes picture when it recognizes John from pictures included in his profile then why cant it add that label to the image in an accessible format? The picture could then read, "John and Jane smiling". Or John on the left, Jane on the right smiling".


Branding

include.ai logo with a sophisticated font
sample include.ai logo with a coding font
current include.ai logo

Extra Demo

Detectron

alt

This image has no alt text.

small white dog walking on red brick. note: this image is interactive
alt="small white dog walking on red brick"

Alt text and interactive image labels generated by include.ai, with support from detectron.

Dog Bush Road Background

Relevent Links:
include.ai
label.include.ai
medium.com/include-ai







HarmonAI



Motivation

"Interactive and fun musical Artificial Intelligence demo + simple automatic harmony generation."

Solution

Interactive Visual


screenshot of instructions on how to use HarmonAI




Interactive Vocal


cartoon keyboard connecting to cartoon laptop with midi cable
amazon alexa
HelpIntent  
  
 help
StartIntent 
  
 start recording
StartIntent start a new recording
StopIntent  
  
 stop recording
NameIntent  
  
 name the recording {happy birthday|RecordingName}
ListIntent  list recordings
ListIntent  list my recordings
ListIntent  list all my recordings
ListIntent  
  
 generate a harmony for the recording {happy birthday|RecordingName}
PlayIntent  
  
 play the recording {happy birthday|RecordingName}
PlayIntent  
  
 play the harmonized recording {happy birthday|RecordingName}




Relevent Links:
harmonai.kyang.co



fun + words =

WordArithmetic

Motivation

"Interactive and fun Artificial Intelligence demo with words."

Solution

Interactive Visual


screen shot of the wordarithmetic.com homepage

screenshot of the wordarithmetic.com homepage on a mobile device


Relevent Links:
wordarithmetic.com
medium.com/@kevin_yang medium.com/@kevin_yang/serverless

Thank You