Proposal for Engage services layer technologies

[Colin Clark]

[John Norman responded to me off-list with this question, and I'm  
happy to respond to it openly. I've retained his post unedited, and  
responded inline.]

Hi John,

On 23-Jul-09, at 5:04 AM, John Norman wrote:

> This post is framed in the context of a decision for Fluid Engage, a  
> project to implement a specific set of functionality.
>
> However, I read into it a statement of strategic direction for the  
> whole Fluid endeavour as far as Javascript on the Server goes. I was  
> uncertain whether to reply on-list because I don't know if it is  
> your intention to open that can of worms or simple to try something  
> out with a single project and see how it goes before considering the  
> strategic implications.

You're right, this proposal is definitely geared specifically to the  
goals and concerns of Fluid Engage. As far as a larger strategy, I  
don't think we've fully fleshed out how this approach might affect  
things beyond what we ship for Engage. We're taking it one step at a  
time.

That said, I settled on this approach because I wanted to leverage  
Infusion and ensure that it continues to grow and be used in  
interesting new ways. I do think there's a lot of potential in  
JavaScript on the server in general, and our use of Kettle in Engage  
might be a first step in this direction. We'll have to see how it goes.

> If it is a question about broad direction for a range of Fluid  
> activities that go beyond Engage, and Engage is just the currently  
> funded project that enables you to get started, then my question  
> would be simple. Can I run Kettle in front of Sling, and in  
> particular Sling/K2 JSON services? The emphasis on JSON suggests  
> this should be possible and that we should expect similar  
> flexibility over migration that you are projecting for Infusion  
> components, but because the proposal is couched in terms of a full  
> stack for a single project, it is not clear how you expect the  
> technologies to be used with other projects (if at all).

At the moment, we don't have any formal plans to support Kettle as a  
standalone product, distinct from the rest of the Engage deliverables.  
However, it's a major goal to ensure that the dependencies between  
Infusion, Kettle, and Engage are clear and separable, though. If we  
find it's maturing well and people are interested in using it, we'll  
definitely consider supporting it as a full product. In the meantime,  
the code is there for people to see and work with.

In terms of you using it with Sling, I think that's probably quite  
possible. Indeed, we're assuming that some of our museums are already  
going to have existing systems and databases that they'll want to use  
instead of the default CouchDB feed that will ship with Engage. The  
McCord Museum, for example, has an excellent LAMP-based CMS. Instead  
of importing all their data into Couch, they'll simply create a data  
feed that is compatible with the views we've setup in Couch as data  
feeds.

In short, there will be nothing hard-baked about the use of CouchDB in  
Kettle. It's just a lightweight Web container for building  
applications in JavaScript on the server. So it should be viable to  
use it within the context of Sakai K2 and Sling. Our existing proof-of- 
concept Kettle implementation is probably still a bit raw to try this  
out, but it will be growing quickly in the coming months, especially  
as we push towards a simple Engage 0.1 release at the end of September.

> What would be particularly valuable to me is a similar PoC demo for  
> Kettle plus Sakai K2/Sling.
>
> I can repost to the list if you intended this sort of issue to be  
> raised and wish me to carry on in an open forum.
>
> John

These are really good questions, and hope this response has been  
helpful. Don't hesitate to continue the thread on fluid-work.

Colin

---
Colin Clark
Technical Lead, Fluid Project
Adaptive Technology Resource Centre, University of Toronto
http://fluidproject.org