Lightbox User Testing Protocol

Daphne Ogle daphne at
Wed Oct 17 17:36:39 UTC 2007

A couple thoughts about the protocol scenario...

We are asking users to pretend they are an Art History Professor and  
reorganize this images based on images we are providing.  I think  
there are two challenges here:

- Since they aren't familiar with the images, I have some concern  
that users will get caught up in trying to understand conceptually  
how the image should be ordered rather than the process of moving the  
images around.  We could overcome this by asking adding tasks  
requesting they move image A in between image e & d or move image G  
to the beginning of the collection for instance.

-  Eileen, the primary persona for the Image Gallery and therefore  
the lightbox makes sense of her images as she reorganizes them.  The  
process of moving them around and seeing them in different states  
helps her create her course content and sometimes even define the  
topic for a particular class session.  If we choose images for users  
in the test, there is no meaning in the images themselves to that  
particular user.  Needing to see the images in a certain way likely  
requires seeing a certain number of images at once, elegant scrolling  
as they move images down the page below the fold, moving groups of  
images together as their model comes together, etc.  For testees that  
are in image heavy disciplines we could ask them to provide a  
collection of images that have meaning.  Thoughts?

More on using personas for the test...   In my experience personas  
are great as a design tool.  They give us a clear picture all the  
users of the system but rather they identify the archetypes that  
allow us to focus design and still build a system that works for  
everyone.  So by designing for Eileen, we think the system will work  
for her unique goals and needs but also work for a variety of users.   
 From About Face 2.0 (the author Cooper is the inventor of personas)  
"The key to choosing the right individuals to design for, ones whose  
needs represent the needs of the larger set of key constituents, and  
knowing how to prioritize design elements to address the needs of the  
most important users without significanlty inconveniencing secondary  
users."  In the case of user testing I think we want to test a  
diverse number of users that represent our constituents.  So although  
we designed for Eileen, she won't be the only kind of user.  Make sense?


On Oct 17, 2007, at 8:17 AM, Barbara Glover wrote:

> Hi Allison
> Thanks for sending the link to the chart.  That will help out.   
> Originally I wanted to do a more comprehensive study of users  
> representing both our faculty persona as well as student personas.   
> Daphne and I had a conversation about that this week and with the  
> shortened time-line we thought to focus this round on faculty with  
> the addition of a few key-board only users.
> Since our planning meeting yesterday though it appears we may have  
> more time to complete further testing.  In light of this, I would  
> like to propose we start the testing with faculty matching our  
> primary persona for Sakai and then, branch out to include the  
> students.
> As far as extrapolating to the Re-orderer in general, that was not  
> the intent of these studies.  However it is a good suggestion and  
> one as we do more planning around user testing in general that we  
> should consider.
> cheers
> Barbara
> On 16-Oct-07, at 8:46 PM, Allison Bloodworth wrote:
>> Hi there,
>> It seems that one addition we should make to the testing protocol  
>> is identifying the attributes of the users we want to test, and  
>> then keeping track of which user types we've been able to find.  
>> See  
>> for an example of an (admittedly complicated--we probably don't  
>> need nearly this much detail) matrix Daphne, Judy and I came up  
>> with for the Content Management Research. Are we only planning to  
>> target folks who are similar to the two Lightbox personas (http:// 
>>, both of  
>> which are faculty members? It sounds like we at a minimum need to  
>> add "keyboard-only user" to the list of target characteristics.  
>> Are there others?
>> A question that may help answer that is to ask whether we trying  
>> to extrapolate testing for the Re-orderer from testing of the  
>> Lightbox? If so, I am wondering if we should consider also testing  
>> students and people who are using Sakai for research or project  
>> management as well (e.g. 
>> fluid/Sakai+Personas), since they may be using a version of the re- 
>> orderer in another Sakai tool (e.g. Page Order Helper). It  
>> wouldn't be testing in context, but at least it would be testing  
>> user types who may encounter the Re-orderer. There would also be  
>> the users of other Fluid applications to consider, but I'm  
>> guessing they may be covered by some of the Lightbox or Sakai  
>> personas, and am not sure it would make sense to define our target  
>> testing subjects in an incredibly granular fashion (because we are  
>> going to be limited as to who we can access).
>> Allison
>> On Oct 16, 2007, at 2:54 PM, Colin Clark wrote:
>>> Hi Barbara and all,
>>> This is a really good testing protocol for the Lightbox. Nice work!
>>> The UX team talked about user testing briefly on our planning  
>>> call today. For those of you who weren't there, here's a quick  
>>> summary.
>>> The 0.1 release is unusual in that we haven't had the opportunity  
>>> to do a lot of user testing on the Lightbox yet. Now that we've  
>>> got a solid test plan, it will serve as the basis for identifying  
>>> and fixing usability issues with the Lightbox and Reorderer over  
>>> the next few releases. User testing will become an ongoing  
>>> activity for us from here on.
>>> We'll fix as many issues as we can before 0.1, and then continue  
>>> with user testing and refactoring into 0.2, refining the test  
>>> plan as we go.
>>> Barbara mentioned during the meeting that she's planning to run  
>>> this plan with Justin Obara early next week here in Toronto. Is  
>>> anyone else able to do some testing with users at their institution?
>>> Colin
>>> Barbara Glover wrote:
>>>> Hi Daphne/Mike/Allison/Shaw-Han
>>>> We're trying to get some quick user testing done for the release  
>>>> of  the Lightbox at the end of October.  Several of you have  
>>>> offered to  help with this testing especially the accessibility  
>>>> sections at your  universities.
>>>> I've put together a protocol that we can use with the  
>>>> instructor  user.  Please review and make changes at the  
>>>> folllowing wiki page.
>>>> +Protocol
>>>> Let's try and determine if we can get any testing completed in  
>>>> the  next week before the release is done.
>>>> cheers
>>>> Barbara
>>>> _______________________________________________
>>>> fluid-work mailing list
>>>> fluid-work at
>>> -- 
>>> Colin Clark
>>> Technical Lead, Fluid Project
>>> Adaptive Technology Resource Centre, University of Toronto
>> Allison Bloodworth
>> Senior User Interaction Designer
>> Educational Technology Services
>> University of California, Berkeley
>> (415) 377-8243
>> abloodworth at

Daphne Ogle
Senior Interaction Designer
University of California, Berkeley
Educational Technology Services
daphne at
cell (510)847-0308

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the fluid-work mailing list