CouchDB - managing design documents?

Tony Atkins tony at
Wed Aug 30 16:45:29 UTC 2017

Hi, All.

Monday the 11th looks like the only day that works for everyone.  I am
waiting to chat with Colin on his return before picking the time slot.
Please hold 15:00-17:30 CET on that date for now, I will get back with the
final time on the 4th.



On Tue, Aug 29, 2017 at 11:14 AM, Tony Atkins <tony at>

> Hi, Simon:
> That's a fair point, at a minimum we should be aware of the need and see
> what if any ground work we can take care of in working on managing design
> documents.
> As far as I know we do not currently have a general solution to managing
> data migrations.  In the rare cases when I've needed to restructure, I've
> used a bulk read, transform, and bulk update, but obviously that's only a
> very temporary solution.
> At the risk of entangling this conversation with another fairly big topic,
> one idea that comes to mind is using versioned JSON schemas and some
> convention of indicating what schema is used for a particular record (for
> example, a URI pointing to the schema).  That would let us:
>    1. Detect existing corrupt or incomplete records.
>    2. Limit a migration to a particular type and/or version of a record.
>    3. Verify that the results of the migration are themselves valid.
>    4. Determine which records have already been migrated, and which have
>    not.
> Anyway, we should bring up the topic in the architecture meeting and set
> up an initial meeting in the next week or so.  With that in mind, I have
> created a Doodle Poll:
> Cheers,
> Tony
> On Mon, Aug 28, 2017 at 5:31 PM, Bates, Simon <sbates at> wrote:
>> To broaden this topic slightly, I'm thinking that some form of data
>> migration automation would also be important to have, in addition to
>> updating design documents. For example, if a document structure is
>> modified, we would want to have an automated way of updating existing data
>> to the new structure.
>> Do we have anything currently existing to manage data migration and
>> document structure evolution? If not, I'm thinking it would be worth
>> including the topic in discussions on managing design documents.
>> Simon
>> ________________________________________
>> From: fluid-work <fluid-work-bounces at> on behalf of
>> Harnum, Alan <aharnum at>
>> Sent: August 28, 2017 11:19:59 AM
>> To: Tony Atkins
>> Cc: Fluid Work
>> Subject: Re: CouchDB - managing design documents?
>> Hi Tony,
>> Thanks for the replies – we’ve also been looking at the various Couchapp
>> implementations, but have been hesitant both for what you cite below (large
>> additional stack of dependencies) and this part of the current CouchDB
>> docs:, which discourages the
>> previously-promoted “CouchApps” paradigm that the various CouchApp tools
>> are designed to use. My sense is that the various Couchapp projects are not
>> under terribly active development in light of this change in CouchDB’s
>> latest documentation.
>> I’d also be an advocate of a Fluid component for capturing views and
>> perhaps other configuration aspects of an Infusion application that intends
>> to make use of CouchDB - I actually did some experimentation in this
>> direction last Friday, which I’ve now cleaned up a bit and added some
>> documentation:
>> In my ideal world I think I’d be able to write a CouchDB configuration
>> for an application in pure Infusion rather than pushing JSON around with
>> curl or other tools.
>> From: Tony Atkins <tony at>
>> Date: Monday, August 28, 2017 at 7:32 AM
>> To: "Harnum, Alan" <aharnum at>
>> Cc: Fluid Work <fluid-work at>
>> Subject: Re: CouchDB - managing design documents?
>> Hi, Alan:
>> Thanks again for starting yet another interesting and important
>> discussion.  I've had to do a lot of this in my own work, and have
>> alternated between:
>>   1.  Storing the design documents as raw JSON (mainly for testing).
>> This is similar to the approach used in the example you shared.
>>   2.  Writing design documents using couchapp<
>> uchapp/couchapp>.
>> The first approach seems easy enough, but is a bit tedious on a few
>> fronts:
>>   1.  It's difficult to write even moderately complex functions as
>> escaped strings.
>>   2.  Replacing an existing design document is a bit tedious, as it
>> involves either looking up the current document's _rev or DELETEing and
>> then POSTing the replacement.
>>   3.  Although it's possible to require libraries from a design document<
>> this is pretty hard to manage using raw JSON files.
>> Couchapp makes each of these easier:
>>   1.  You just save your functions to a file, which you can run through
>> our standard linting and more easily check for syntax errors.  You can even
>> write unit tests against the functions you plan to use in your design
>> documents.  This is especially helpful when developing reduce functions,
>> which can be a bear to troubleshoot with CouchDB or PouchDB.
>>   2.  Although it's trivial, couchapp makes it very easy to update design
>> docs repeatedly.
>>   3.  Couchapp makes it easier to bundle dependencies as part of your
>> design document (basically by deploying them to a "lib" or similar
>> directory within a design document's directory).
>> That being said, couchapp is something I have only relied on
>> reluctantly.  I don't use it in tests, for example, as it seems a bit much
>> to me to introduce a whole stack of dependencies (python, required
>> libraries) just to solve what's a fairly simple use case.  Antranig and I
>> have previously talked about building a Fluid component to assist with
>> creating design documents, and it seems like a good time to broaden and
>> refresh that discussion.
>> Regardless of whomever else is interested, I would be very happy to work
>> together with you (and I'm assuming Antranig) on a ticket outlining the use
>> cases in more detail.  Depending on the scope of what we agree is needed, I
>> might end up simply building it the next time I need to work on a batch of
>> design documents.  I'd also be very happy to help review something someone
>> else builds and test that in my own work.
>> Cheers,
>> Tony
>> On Thu, Aug 24, 2017 at 4:16 PM, Harnum, Alan <aharnum at<mailto:
>> aharnum at>> wrote:
>> Hi all,
>> We (Greg Moss & myself) are curious to know the opinions of others who've
>> worked with it (especially in the context of Infusion / Kettle) about how
>> to best manage design documents when using CouchDB as part of an
>> application.
>> Specifically, what approaches have worked for externalizing design
>> document code, ensuring the database has the latest design document code,
>> etc?
>> I know one example of doing this is in gpii-oauth2-datastore, which is a
>> JSON externalization of the views that is imported to the DB as part of
>> application provisioning, described at:
>> sal/tree/master/gpii/node_modules/gpii-oauth2/gpii-oauth
>> 2-datastore/dbViews
>> I am wondering if people have seen or considered other approaches.
>> My own feeling is that it would be useful to be able to model an expected
>> CouchDB database design document as an Infusion component, with the
>> component taking responsibility for ensuring a set of view functions are
>> present in the specified DB, but I don't feel a great certainty about this.
>> Curious as to the thoughts of others!
>> Alan
>> E aharnum at<mailto://>
>> 100 McCaul Street, Toronto, Canada, M5T 1W1
>> _______________________________________________________
>> fluid-work mailing list - fluid-work at<mailto:
>> fluid-work at>
>> To unsubscribe, change settings or access archives,
>> see
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the fluid-work mailing list