Sonification and data representation design

colinbdclark at gmail.com colinbdclark at gmail.com
Thu Apr 9 15:48:34 EDT 2015


Hi everyone,

As part of the Floe Project's efforts to create personalized, accessible
user interfaces that can be used across a variety of Open Educational
Resources, we've been working on a design framework and JavaScript toolkit
for authoring multimodal charts, graphs, and other data "visualizations."
One of the central goals of this effort is to make it easier for teachers,
students, and content authors to represent data in "layers" consisting of
different modalities--graphics, text, and audio.

To start, we've been focusing a lot on sonification, the process of
representing data using sounds. We're in the midst of a very early
brainstorming, sketching and idea generation process. Our work is
documented in the wiki:

http://wiki.fluidproject.org/display/fluid/Floe+Sonification+Framework
http://wiki.fluidproject.org/display/fluid/Exploration+and+Early+Sketches
http://wiki.fluidproject.org/display/fluid/Sonification+Sketches
http://wiki.fluidproject.org/display/fluid/Use+Cases

In order to start exploring the potential of data sonification in a way
that allows us to experiment with different approaches and to iterate from
mockups to working implementations reasonably quickly, we've constrained
our current design sketches to tools that will help authors produce
multimodal "pie charts." The goal of this tool is to enable authors to
produce layered representations of fairly simple data, and to give
end-users the ability to explore, remap, and share their own personalized
sonifications and visualizations.

As part of this process, we've been exploring some new methods for how we
design sonifications strategies and evaluate their effectiveness. We've
started working with a small, informal group of people in a co-design
context, and will also be sharing our in-progress work here on the list.
What we've done so far is to prototype several different types of
sonifications using low-tech tools and then shared them with people using a
process of "progressive explanation." We start by having them listen to the
sonification with no additional cues or explanation, asking them to
describe their impressions (including how they imagine the sounds map to
some underlying data set). From there, we progressively explain more about
the intentions behind sonification (such as describing the sound mapping
using an "audio legend"), and continue to gather impressions and ideas from
our listeners. We've found this to be a very helpful process for exploring
how much textual or explanatory supporting material to provide with a given
sonification approach. Sepideh has posted some great examples and
prototypes in the wiki.

Over the coming months, we'll expand this design effort to encompass more
complex data and to more interactive situations such as simulations, games,
and performances.

We'll continue to share ideas, sketches, and works in progress here on the
mailing list, in the #fluid-design IRC channel, and in the wiki.
Constructive feedback and creative ideas are always appreciated during this
early stage in the process, as well as the understanding that we're still
experimenting and exploring the design space. Failures and half-baked ideas
are as useful at this stage in the design process as successes.

Colin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idrc.ocad.ca/pipermail/fluid-work/attachments/20150409/24251577/attachment.html>


More information about the fluid-work mailing list