Engage / ArtifactView / Comments / Storing documents in couchdb and other things
Colin Clark
colinbdclark at gmail.com
Mon Dec 21 19:10:05 UTC 2009
Hey all,
[Joan, I know you're currently away from work due to an injury, so
don't worry about the details of this. My main goal in responding to
your questions (sorry for the delay!) is so that the whole community
has clarity on our next steps in terms of data access.]
I like the concept of the database.js implementation. Since it was
written, we uncovered two JavaScript APIs that ship directly with
CouchDB:
http://titan.atrc.utoronto.ca:5984/_utils/script/couch.js
http://titan.atrc.utoronto.ca:5984/_utils/script/jquery.couch.js
One is a plain JavaScript API, and the other is designed as a jQuery
plugin. They offer overlapping functionality, but in general a
preference is emerging for the jQuery plugin--it's properly
namespaced, and a bit better structured. Here's a thread on the fluid-
work list from last week where Antranig and I discuss the issues:
http://old.nabble.com/Fluid-Engage-nightly-build-broken-td26785515.html#a26832718
I think we're always better to contribute back to existing projects
rather than rolling our own where it's appropriate, so I'd like to
suggest that instead of working on database.js, we contribute some
patches back to Couch for their jQuery plugin. One area where we'll
need to contribute back, in addition to the ones mentioned by
Antranig, is support for Couch Lucene views, which we use frequently.
Antranig also has some code in his FLUID-213 patch that illustrates
some slightly higher-level ways of making Couch requests within Kettle:
http://issues.fluidproject.org/secure/attachment/10833/ENGAGE-213-demonstration.patch
This covers the low-level issues of data access. I think our second
goal is to provide nice abstractions so that every Engage service
developer doesn't have to write the same data access boiler plate
goal. In a recent development meeting, we talked a little bit about
the notion of "data source specs," which would provide all of the
information needed to retrieve and save data to/from Couch in a
declarative way.
This data source spec would include the URL in Couch that identifies
the document or set of documents being used by a particular data feed
and component. It would also include the requisite model mapping
information required to correctly reshape the model into a form the
component can suitably use. You can see a rough sketch of our current
model mapping code in this file:
http://source.fluidproject.org/svn/fluid/engage/fluid-engage-core/trunk/framework/js/Engage.js
Take a look at the fluid.engage.collections data structure for an
example of the model maps we're currently using. This code needs quite
a bit of work to be correctly parameterized and generalized, but the
idea is to be able to have a document in Couch that represents the
necessary transformations required on the model for a particular view
context. The framework will include a number of "canned reshapers" for
common mapping tasks, and museums could extend this with their own
specific code if needed.
This second layer of data access will take a bit longer for us to
implement, so let's focus first on the task--contributing back to
Couch's jquery.couch.js file based on our own needs.
I hope this helps clarify the issue of data access in Engage a bit.
This is all still very much in the concept stage, so no doubt it also
opens up some new questions. Everyone, please don't hesitate to
respond to this thread with thoughts, comments, and questions.
Colin
On 1-Dec-09, at 9:31 AM, Joan Garcia Vila wrote:
> Hi all.
>
> Recently I've posted an alpha implementation of the Database
> component (which has to become, I suppose, a subcomponent of
> ArtifactComments [server side]).
> Needs a conceptual review.
>
> - Talking about storing data on couchdb, this has been my first
> approach:
>
> // Artifact Comment example.
> var artifactComment = {};
> artifactComment.engage_type = "artifact-comment";
> artifactComment.artifact_id =
> "00311f0d3f5456c91934e472b880e9c5";
> artifactComment.insertDate = "2009-11-30";
> artifactComment.name = "Juan";
> artifactComment.location = "Barcelona, Catalonia, Spain";
> artifactComment.abuse = false;
> artifactComment.info = "Barcelona is a beautiful
> mediterranian city.";
>
> There are few essential key points to check (understand / validate /
> talk about):
>
> 1.- This document has a field that identifies its type
> ("engage_type").
> If we store different object types inside the same database this
> will help and give higher scores in lucene fulltext search if we add
> the document type in the query
> (Please see/run the example).
>
> 2 .- This document has a reference to another document. See field
> "artifact_id". Important to have good views with high selectivity.
> This will help us to create "in-server" high speed paged queries.
> Views for each document type if we need this functionality.
> (Please see/run the example).
>
> 3.- The object also gets its _id and _rev attributes after it has
> been stored.
> This is coherent with couchdb (object oriented database) and the key
> is an uuid.
> (Please see/run the example).
>
> 4.- Data attributes is stored as a fixed text with pattern "YYYY-MM-
> DD".
> This will help us to have date sorted field views.
>
> 5.- One important thing is that having the documents (2 diferent
> types) inside the same database allows us to create fast views (see
> Database.js).
> We can also create "pseudo joins" [complex views using map/reduce
> _design functions]
>
> - Other things that come to my mind, while developing the component,
> are:
> 1. - How do we know who we are?, I haven't see a login page in the
> wireframes.
> But there is behaviour that seems to depend on knowing who we are.
> Like report abuse or delete, for an example.
> makes sense?
>
> 2.- How do we know that the user is the moderator?
> makes sense?
>
> Cheers,
> joan
>
> ps: you can find (Database. [js/html] on scratchpad/bug-parade.
> _______________________________________________________
> fluid-work mailing list - fluid-work at fluidproject.org
> To unsubscribe, change settings or access archives,
> see http://fluidproject.org/mailman/listinfo/fluid-work
---
Colin Clark
Technical Lead, Fluid Project
http://fluidproject.org
More information about the fluid-work
mailing list