Services layer redundancy

Colin Clark colinbdclark at
Sat Jan 30 22:50:17 UTC 2010

Hey Yura,

On 2010-01-25, at 5:45 PM, Yura wrote:
> I attempted to sketch a set of service utilities to eliminate some of the redundancy in the services layer feeds. Here the reshaper functions weren't really modified but simply used by the utilities in their original state to demonstrate the flow of events. The patch can be found here: . Please send your thoughts and feedback.

I know Antranig gave you some one-on-one review of this code, and has added some additional comments on the JIRA ticket. Nonetheless I thought I'd respond with a couple of points that occurred to me while reading over your code.

I like where you're going with this: having a declarative way to define feeds and services in Kettle makes a lot of sense to me. If you look at the current service code, there are consistent patterns throughout all of them:

 1. Mount the necessary handlers at specific paths within the overall URL space
 2. When responding to requests:
   * Parse query parameters
   * Fetch raw data from Couch
   * Transform/reshape/map the model from a museum-specific format to a general view-oriented model
   * Handle errors
   * Provide components with suitable options in the markup feeds

I think Antranig pointed out the problem with how you're handling errors when fetching data in this patch. Indeed, we haven't been doing a very good job of handling this case throughout the code base. His new fluid.kettle.getData() method should help quite a bit, but error handling will be a big issue to tackle in 0.5.

The other key architectural point with your approach is that it feels "upside down" to me. You've created an API that developers can use within the code they write to define their own service. But I think the appropriate solution would flip this around--the service declaration you've got in engageConfig should be all a developer needs to write in order to have the framework create the service for them. Assuming it's got the right extension points, a developer's work shifts from the low-level mechanics of coding a service to defining the overall business of their services.

Another point that I think is really interesting about your patch: you've tried to start simplifying our data URLs. At the moment, these tend to be pretty repetitive. To take what you've sketched out a bit further, I think we want to provide a single point for defining the base URL for all database calls. This will simplify museum configuration by providing a single place for them to define the location of their own database. Data spec URLs should specified relative to this base URL. With this approach, I think we may not need to parameterize the Couch view URL fragment in the way you've done here.

I really like the start you've got here. I think we've probably got a fair bit more thinking to do before we can implement something like this in a way that will satisfy all of the service use cases we've got. Given how focused we are on end user-facing functionality for Engage 0.3, let's pick this issue up again for Engage 0.5 and see what we can do. This is something you can pair up with Antranig or me to work on then.

Glad you got the conversation started,


Colin Clark
Technical Lead, Fluid Project

More information about the fluid-work mailing list