Engage restructing proposal

[Colin Clark]

Hi Antranig,

On 9-Oct-09, at 9:24 PM, Antranig Basman wrote:
> i) I assume that engage-client is also nested inside src/webapp as  
> well...
> otherwise I guess we would not easily be able to composite it inside
> Maven.

We certainly can build engage-client with Maven, but at the moment we  
don't, simply for expediency. Except for Kettle itself, I have to  
wonder about the value of building with Maven, since it can't do  
everything we require during the build phase by itself. We can  
certainly look at Maven plugins for handling minification,  
concatenation, and other requisite JS packaging, but in the end, we'll  
probably also need Ant in the mix.

> ii) I object to the somewhat arbitrary distinction between  
> "components"
> and "services" which obscures the generally harmonious bent to the  
> framework :P
> I guess there will in time be more to what I hope we will not call a  
> "service" than a single .js file, since these will have their own  
> configuration/manifest supplied in some JSON structure.

I think at the moment, there is a sensible distinction between  
components and so-called "services": services depend on server- 
specific APIs provided by Kettle. This is, fortunately, pretty minimal  
and I expect it always will be. Beyond the files that initialize and  
manage the services on the server, I think we've got an interesting  
and exciting situation where almost all of our code is free to run  
either on the client or the server.

I don't particularly like the name services either, but I was at a  
loss for something else. After 0.1, we're going to have to do a Kettle- 
wide terminology blitz.

> iii) whilst "mapping" doesn't actually contain any mapping, it does  
> contain a lot of useful JSON to XML benchmarking - I am guessing by  
> "remove from SVN altogether" you just mean "do not promote from  
> incubator along with the rest of Engage"?

Yes, exactly. Mapping needs to stay in the incubator but not move to  
core with the rest of Kettle and Engage. For 0.3, we're planning to  
evolve this code into a full-fledged import service with a UI that  
museums can use to easily bring their data into Engage.

> iv) concerned about having engage client-specific framework, I've  
> looked over it a bit, but don't see anything particular about most  
> of it that insists it lives on the client, although it does touch  
> some tags. But this does touch a quite interesting issue that we  
> should ponder, even if we don't do anything about it -

I guess the key thing to note is that engage-client isn't necessarily  
client-specific. It just isn't server-specific. If we were to describe  
it by what it is not, we'd call it "engage-non-server," or something  
equally funny.

There's essentially nothing in the engage-client/framework directory  
that is client-side specific, and indeed it actually does run on the  
server at the moment, though it can quite freely be moved to the  
client side. I'm still not sure the code that is in there comprises a  
"framework" yet, but I think Justin and Yura chose this name to convey  
the fact that it contains code that is used by a number of services  
and/or components in the system. Fair enough.

> Right now - all of the code we have on the server, has visibility of  
> all of the code we write on the client - even that large part of it  
> relating to DOM manipulation, DOM events, etc. that is useless.  
> Whilst in some sense this is not correct, I guess we justify this by  
> noting that the costs of loading superfluous code on the server  
> (once per server startup) are much lower than loading it on the  
> client (once per page serve). All the same, it is not great, and we  
> should give at least some thought to the concept of a "core  
> framework" which consists just of those definitions which make sense  
> both on the client and the server. In terms of infusion, this is not  
> too hard, since it consists largely of Fluid.js and the Renderer,  
> plus or minus a few bits and pieces. So, I think we should make  
> efforts to ensure that as we "go forward", engage code is designed  
> with this in mind too - that is, parts of the code which are "model- 
> directed" and interact with markup only through the renderer, and  
> parts of code which get involved with grubby details of DOM  
> manipulation.

Given the browser-compatibility environment that Env.js provides us, I  
think it's quite sensible that we can have a full suite of Infusion  
tools available to use within Kettle. Looking down the road a ways,  
I'd like to see us offer a super-simple client-side mode where  
developers can produce very simple services and data feed using pretty  
much only the techniques they're already familiar with on the client.

That said, I don't think we'll take advantage of this approach  
ourselves for Engage, since we're interested in writing more  
sophisticated and performant code on the server.

All that said, I completely agree with the idea of further  
modularizing Infusion. As we move to deployment on mobile devices, the  
only really sensible solution is to ensure that our code is very  
carefully modularized so we can create custom device-specific builds  
that contain only the essentials for the particular context. We  
started down this path before Infusion 1.0, and the new Infusion  
Builder helps us a lot. Let's consider continuing to break out the  
core framework into smaller and more self-contained packages.

Great discussion!

Colin

---
Colin Clark
Technical Lead, Fluid Project
http://fluidproject.org