Lightbox User Testing Protocol

Barbara Glover barbara.glover at
Wed Oct 17 17:50:01 UTC 2007

Ok it looks like there are some major changes needed to the current  
protocol so I'm thinking we probably cannot start testing until later  
next week.

I'll try and set up a meeting.

On 17-Oct-07, at 1:36 PM, Daphne Ogle wrote:

> A couple thoughts about the protocol scenario...
> We are asking users to pretend they are an Art History Professor  
> and reorganize this images based on images we are providing.  I  
> think there are two challenges here:
> - Since they aren't familiar with the images, I have some concern  
> that users will get caught up in trying to understand conceptually  
> how the image should be ordered rather than the process of moving  
> the images around.  We could overcome this by asking adding tasks  
> requesting they move image A in between image e & d or move image G  
> to the beginning of the collection for instance.
> -  Eileen, the primary persona for the Image Gallery and therefore  
> the lightbox makes sense of her images as she reorganizes them.   
> The process of moving them around and seeing them in different  
> states helps her create her course content and sometimes even  
> define the topic for a particular class session.  If we choose  
> images for users in the test, there is no meaning in the images  
> themselves to that particular user.  Needing to see the images in a  
> certain way likely requires seeing a certain number of images at  
> once, elegant scrolling as they move images down the page below the  
> fold, moving groups of images together as their model comes  
> together, etc.  For testees that are in image heavy disciplines we  
> could ask them to provide a collection of images that have  
> meaning.  Thoughts?
> More on using personas for the test...   In my experience personas  
> are great as a design tool.  They give us a clear picture all the  
> users of the system but rather they identify the archetypes that  
> allow us to focus design and still build a system that works for  
> everyone.  So by designing for Eileen, we think the system will  
> work for her unique goals and needs but also work for a variety of  
> users.  From About Face 2.0 (the author Cooper is the inventor of  
> personas) "The key to choosing the right individuals to design for,  
> ones whose needs represent the needs of the larger set of key  
> constituents, and knowing how to prioritize design elements to  
> address the needs of the most important users without significanlty  
> inconveniencing secondary users."  In the case of user testing I  
> think we want to test a diverse number of users that represent our  
> constituents.  So although we designed for Eileen, she won't be the  
> only kind of user.  Make sense?
> -Daphne
> On Oct 17, 2007, at 8:17 AM, Barbara Glover wrote:
>> Hi Allison
>> Thanks for sending the link to the chart.  That will help out.   
>> Originally I wanted to do a more comprehensive study of users  
>> representing both our faculty persona as well as student  
>> personas.  Daphne and I had a conversation about that this week  
>> and with the shortened time-line we thought to focus this round on  
>> faculty with the addition of a few key-board only users.
>> Since our planning meeting yesterday though it appears we may have  
>> more time to complete further testing.  In light of this, I would  
>> like to propose we start the testing with faculty matching our  
>> primary persona for Sakai and then, branch out to include the  
>> students.
>> As far as extrapolating to the Re-orderer in general, that was not  
>> the intent of these studies.  However it is a good suggestion and  
>> one as we do more planning around user testing in general that we  
>> should consider.
>> cheers
>> Barbara
>> On 16-Oct-07, at 8:46 PM, Allison Bloodworth wrote:
>>> Hi there,
>>> It seems that one addition we should make to the testing protocol  
>>> is identifying the attributes of the users we want to test, and  
>>> then keeping track of which user types we've been able to find.  
>>> See  
>>> for an example of an (admittedly complicated--we probably don't  
>>> need nearly this much detail) matrix Daphne, Judy and I came up  
>>> with for the Content Management Research. Are we only planning to  
>>> target folks who are similar to the two Lightbox personas (http:// 
>>>, both of  
>>> which are faculty members? It sounds like we at a minimum need to  
>>> add "keyboard-only user" to the list of target characteristics.  
>>> Are there others?
>>> A question that may help answer that is to ask whether we trying  
>>> to extrapolate testing for the Re-orderer from testing of the  
>>> Lightbox? If so, I am wondering if we should consider also  
>>> testing students and people who are using Sakai for research or  
>>> project management as well (e.g. 
>>> display/fluid/Sakai+Personas), since they may be using a version  
>>> of the re-orderer in another Sakai tool (e.g. Page Order Helper).  
>>> It wouldn't be testing in context, but at least it would be  
>>> testing user types who may encounter the Re-orderer. There would  
>>> also be the users of other Fluid applications to consider, but  
>>> I'm guessing they may be covered by some of the Lightbox or Sakai  
>>> personas, and am not sure it would make sense to define our  
>>> target testing subjects in an incredibly granular fashion  
>>> (because we are going to be limited as to who we can access).
>>> Allison
>>> On Oct 16, 2007, at 2:54 PM, Colin Clark wrote:
>>>> Hi Barbara and all,
>>>> This is a really good testing protocol for the Lightbox. Nice work!
>>>> The UX team talked about user testing briefly on our planning  
>>>> call today. For those of you who weren't there, here's a quick  
>>>> summary.
>>>> The 0.1 release is unusual in that we haven't had the  
>>>> opportunity to do a lot of user testing on the Lightbox yet. Now  
>>>> that we've got a solid test plan, it will serve as the basis for  
>>>> identifying and fixing usability issues with the Lightbox and  
>>>> Reorderer over the next few releases. User testing will become  
>>>> an ongoing activity for us from here on.
>>>> We'll fix as many issues as we can before 0.1, and then continue  
>>>> with user testing and refactoring into 0.2, refining the test  
>>>> plan as we go.
>>>> Barbara mentioned during the meeting that she's planning to run  
>>>> this plan with Justin Obara early next week here in Toronto. Is  
>>>> anyone else able to do some testing with users at their  
>>>> institution?
>>>> Colin
>>>> Barbara Glover wrote:
>>>>> Hi Daphne/Mike/Allison/Shaw-Han
>>>>> We're trying to get some quick user testing done for the  
>>>>> release of  the Lightbox at the end of October.  Several of you  
>>>>> have offered to  help with this testing especially the  
>>>>> accessibility sections at your  universities.
>>>>> I've put together a protocol that we can use with the  
>>>>> instructor  user.  Please review and make changes at the  
>>>>> folllowing wiki page.
>>>>> +Protocol
>>>>> Let's try and determine if we can get any testing completed in  
>>>>> the  next week before the release is done.
>>>>> cheers
>>>>> Barbara
>>>>> _______________________________________________
>>>>> fluid-work mailing list
>>>>> fluid-work at
>>>> -- 
>>>> Colin Clark
>>>> Technical Lead, Fluid Project
>>>> Adaptive Technology Resource Centre, University of Toronto
>>> Allison Bloodworth
>>> Senior User Interaction Designer
>>> Educational Technology Services
>>> University of California, Berkeley
>>> (415) 377-8243
>>> abloodworth at
> Daphne Ogle
> Senior Interaction Designer
> University of California, Berkeley
> Educational Technology Services
> daphne at
> cell (510)847-0308

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the fluid-work mailing list