Engage restructing proposal

[Antranig Basman]

Hi there - Colin and I have had a further chat about naming which I will 
summarise.
The current proposal looks pretty good, so long as we do it in the 
spirit that the layout is still fairly "provisional" - the Kettle 
framework itself is still evolving rapidly together with our 
understanding of how to organise client-server cooperation. That said, 
we want to do something now that gets as as close as possible to a 
reasonable layout that will minimise work later.

It's worth reiterating Colin's point that there is nothing in 
"engage-client" that could not physically run on the server, assuming an 
env.js implementation of sufficient quality - even though that is not 
actually the approach we are intending to recommend (for DOM-twiddling 
code). All the same, this implies that the role of "engage-client" is 
not quite morally that of a "client" but really is that part of engage 
which is not dependent on a particular server-side implementation (which 
in our case will be kettle). So we propose renaming "engage-client" to 
something like "engage-core" to minimise the cognitive dissonance when 
we make reference to its code from the server-side.

Correspondingly, a proposal for "engage-server" is to name it to 
"engage-kettle" to emphasise the fact that this is just *one* solution 
for a suitable server-side to interact with the Kettle "client-side" 
artefacts. Just as with the Infusion/Kettle split, we want to continue 
to emphasise that whilst Fluid is now starting to provide server-side 
infrastructure, we are not prescriptive about this and are committed to 
maintaining a "pay-as-you-go" system with client-side definitions 
prepared to work with any suitable server-side.

Cheers,
Antranig.


Colin Clark wrote:
> Hi Antranig,
> 
> On 9-Oct-09, at 9:24 PM, Antranig Basman wrote:
>> i) I assume that engage-client is also nested inside src/webapp as 
>> well...
>> otherwise I guess we would not easily be able to composite it inside
>> Maven.
> 
> We certainly can build engage-client with Maven, but at the moment we 
> don't, simply for expediency. Except for Kettle itself, I have to wonder 
> about the value of building with Maven, since it can't do everything we 
> require during the build phase by itself. We can certainly look at Maven 
> plugins for handling minification, concatenation, and other requisite JS 
> packaging, but in the end, we'll probably also need Ant in the mix.
> 
>> ii) I object to the somewhat arbitrary distinction between "components"
>> and "services" which obscures the generally harmonious bent to the 
>> framework :P
>> I guess there will in time be more to what I hope we will not call a 
>> "service" than a single .js file, since these will have their own 
>> configuration/manifest supplied in some JSON structure.
> 
> I think at the moment, there is a sensible distinction between 
> components and so-called "services": services depend on server-specific 
> APIs provided by Kettle. This is, fortunately, pretty minimal and I 
> expect it always will be. Beyond the files that initialize and manage 
> the services on the server, I think we've got an interesting and 
> exciting situation where almost all of our code is free to run either on 
> the client or the server.
> 
> I don't particularly like the name services either, but I was at a loss 
> for something else. After 0.1, we're going to have to do a Kettle-wide 
> terminology blitz.
> 
>> iii) whilst "mapping" doesn't actually contain any mapping, it does 
>> contain a lot of useful JSON to XML benchmarking - I am guessing by 
>> "remove from SVN altogether" you just mean "do not promote from 
>> incubator along with the rest of Engage"?
> 
> Yes, exactly. Mapping needs to stay in the incubator but not move to 
> core with the rest of Kettle and Engage. For 0.3, we're planning to 
> evolve this code into a full-fledged import service with a UI that 
> museums can use to easily bring their data into Engage.
> 
>> iv) concerned about having engage client-specific framework, I've 
>> looked over it a bit, but don't see anything particular about most of 
>> it that insists it lives on the client, although it does touch some 
>> tags. But this does touch a quite interesting issue that we should 
>> ponder, even if we don't do anything about it -
> 
> I guess the key thing to note is that engage-client isn't necessarily 
> client-specific. It just isn't server-specific. If we were to describe 
> it by what it is not, we'd call it "engage-non-server," or something 
> equally funny.
> 
> There's essentially nothing in the engage-client/framework directory 
> that is client-side specific, and indeed it actually does run on the 
> server at the moment, though it can quite freely be moved to the client 
> side. I'm still not sure the code that is in there comprises a 
> "framework" yet, but I think Justin and Yura chose this name to convey 
> the fact that it contains code that is used by a number of services 
> and/or components in the system. Fair enough.
> 
>> Right now - all of the code we have on the server, has visibility of 
>> all of the code we write on the client - even that large part of it 
>> relating to DOM manipulation, DOM events, etc. that is useless. Whilst 
>> in some sense this is not correct, I guess we justify this by noting 
>> that the costs of loading superfluous code on the server (once per 
>> server startup) are much lower than loading it on the client (once per 
>> page serve). All the same, it is not great, and we should give at 
>> least some thought to the concept of a "core framework" which consists 
>> just of those definitions which make sense both on the client and the 
>> server. In terms of infusion, this is not too hard, since it consists 
>> largely of Fluid.js and the Renderer, plus or minus a few bits and 
>> pieces. So, I think we should make efforts to ensure that as we "go 
>> forward", engage code is designed with this in mind too - that is, 
>> parts of the code which are "model-directed" and interact with markup 
>> only through the renderer, and parts of code which get involved with 
>> grubby details of DOM manipulation.
> 
> Given the browser-compatibility environment that Env.js provides us, I 
> think it's quite sensible that we can have a full suite of Infusion 
> tools available to use within Kettle. Looking down the road a ways, I'd 
> like to see us offer a super-simple client-side mode where developers 
> can produce very simple services and data feed using pretty much only 
> the techniques they're already familiar with on the client.
> 
> That said, I don't think we'll take advantage of this approach ourselves 
> for Engage, since we're interested in writing more sophisticated and 
> performant code on the server.
> 
> All that said, I completely agree with the idea of further modularizing 
> Infusion. As we move to deployment on mobile devices, the only really 
> sensible solution is to ensure that our code is very carefully 
> modularized so we can create custom device-specific builds that contain 
> only the essentials for the particular context. We started down this 
> path before Infusion 1.0, and the new Infusion Builder helps us a lot. 
> Let's consider continuing to break out the core framework into smaller 
> and more self-contained packages.
> 
> Great discussion!
> 
> Colin
> 
> ---
> Colin Clark
> Technical Lead, Fluid Project
> http://fluidproject.org
>