All Articles

Week4 Project Deaf Tone

Collaboration with Sara Ro. See her post here: Link

DEMO

We made an ASL instrument this week. Sign language Images are classified by a model trained @teachablemachine. They are then mapped to 8 notes covering an octave from C4 to C5. Since we want this project to be accessible by deaf people as well, we added several visualizations including a waveform and notes falling from the sky.

const labels = {
  'ASL-C': [262, 'C'],
  'ASL-D': [294, 'D'],
  'ASL-E': [330, 'E'],
  'ASL-F': [350, 'F'],
  'ASL-G': [392, 'G'],
  'ASL-A': [440, 'A'],
  'ASL-B': [494, 'B'],
  'ASL-I': [523, 'C5'],
  'background': [0, 'Deaf Tone'],
};

Try it here

Reference