Proposal: better intermediary means of managing external front-end Javascript dependencies in Infusion-based projects using nom + grunt

[Antranig Basman]

Cheers, Alan - this is a really important and strategic issue and I wanted to thank you for organising the 
discussion in such a clear way.

I think the basic plan here is highly desirable - to move towards npm infrastructure and package.json as our 
central solution for managing JS dependencies, and to use browserify when we need to build single-file 
distribution bundles.

I am wondering whether we can simplify the steps in the proposal yet further - that is, whether we can cut 
out the grunt tasks and ignore structure, and as far as possibly simply use the node modules in place where 
they are fetched using npm.

You may be aware that every Infusion-aware npm module now participates in its "module registration scheme" 
kicked off from fluid.module.register - 
http://docs.fluidproject.org/infusion/development/NodeAPI.html#fluid-module-register-name-basedir-modulerequire- 
- this makes it easy to resolve paths into foreign modules by simple expressions like "%flocking/src/stuff" 
etc.

Kettle's "static middleware" could host these directories using these symbolic references directly - or even 
automatically, if we add extra directives to package.json or elsewhere -
https://github.com/amb26/kettle/tree/KETTLE-32#built-in-standard-middleware-bundled-with-kettle

Other points in our future roadmap might include BRFS or something like it to bridge the gap between the 
view of static assets from the point of view of node and the browser - https://github.com/substack/brfs - or 
little things like "staticify" https://github.com/petehunt/staticify - of which there seem to be a dozen.

Does this make sense/fit in with the use cases and road map you imagine? I'm keen to have something that 
involves us in as little build faff and boilerplate as possible, based on the npm infrastructure but with as 
many of the affordances of bower as possible - what do you think? Are there some "hard core use cases" that 
we can't easily meet with the files in their "native position" inside node_modules directories?

Cheers,

Antranig

On 01/03/2016 18:33, Harnum, Alan wrote:
> *Background*
> *
> *
> The Chart Authoring Tool (https://github.com/fluid-project/chartAuthoring) is built with Infusion, the D3
> data visualization library, and the Flocking audio synthesis framework (created by the Fluid community's own
> Colin Clark).
>
> At the moment the non-Infusion dependencies are directly included as files in the project repository. This
> is simple, but bad in various other ways in terms of keeping things up to date, keeping the repo clean, etc.
>
> There has been an outstanding issue since last August to improve this situation
> (https://issues.fluidproject.org/browse/FLOE-405?jql=text%20~%20%22bower%22
> <https://issues.fluidproject.org/browse/FLOE-405?jql=text ~ "bower">).
>
> Originally the thought was to use Bower, but Bower seems to be facing some challenges as a project
> (https://www.reddit.com/r/javascript/comments/3th4v6/bower_vs_npm_3_frontend_package_management/) and the
> general direction of many other projects seems to be towards using straight npm (sometimes with a tool like
> browserify) to manage front-end dependencies.
>
> Infusion itself is moving in this direction as NPM becomes a more viable for front-end management, as
> evidenced by this issue: https://issues.fluidproject.org/browse/FLUID-5745?jql=text%20~%20%22bower%22
> <https://issues.fluidproject.org/browse/FLUID-5745?jql=text ~ "bower">
>
> *My Immediate Goal / Challenge*
>
> I'm starting work on front-end Infusion-based components for the GPII Quality Infrastructure where I would
> like to be able to reuse some code from the Chart Authoring repo itself, so rather than replicate the
> problem again I’d like to have an approach that does the following:
>
> - works with the existing toolstack for builds (npm + grunt) without other tool dependencies such as
> browserify, webpack, etc
> - produces a suitable static “package" for serving publicly, as is used currently by
> http://build.fluidproject.org/
> - has minimal development overhead but improves the current situation
> - can be replaced easily enough in the future when we align on a better means of managing front-end dependencies
>
> *What I Propose*
> *
> *
> Examples here are in the context of the Chart Authoring Tool project in my fork at
> https://github.com/waharnum/chartAuthoring/blob/FLOE-405/ but I’d like the general shape of this to be
> applicable to other cases:
>
> - front-end dependencies should be managed using NPM’s /package.json/ format and installed via a standard
> ‘npm install’ (https://github.com/waharnum/chartAuthoring/blob/FLOE-405/package.json)
> - one or more grunt tasks should then be used to copy these dependencies into an appropriate structure for
> use by the front end project (https://github.com/waharnum/chartAuthoring/blob/FLOE-405/Gruntfile.js)
> - a .gitignore definition should be used to make sure this external dependency directory is not versioned
> (https://github.com/waharnum/chartAuthoring/blob/FLOE-405/.gitignore)
>
> Basically, a front-end project will pull in its dependencies via NPM and then distribute them as needed via
> grunt to create the necessary “product"; the specifics of this will depend on the particular project. We
> already use both npm and grunt in other contexts, they’re familiar to us, and they avoid the need (which we
> have, but longer term) to adopt a specific additional piece of tooling for front-end dependency management,
> which is a space that seems in heavy churn right now in the Javascript community.
>
> We talked about this approach briefly at the Floe planning meeting yesterday and people seemed receptive,
> but the broader community should weigh in and give their opinion.
>
> *Questions*
> *
> *
> 1) Do we feel OK with this approach to front-end dependency management in the short term?
> 2) If we don’t feel OK about it, what should we do instead?
>
> *ALAN HARNUM*
> SENIOR INCLUSIVE DEVELOPER
> INCLUSIVE DESIGN RESEARCH CENTRE, OCAD UNIVERSITY
>
> *E *aharnum at ocadu.ca <mailto://aharnum@ocadu.ca>
>
>
>
> _______________________________________________________
> fluid-work mailing list - fluid-work at lists.idrc.ocad.ca
> To unsubscribe, change settings or access archives,
> see http://lists.idrc.ocad.ca/mailman/listinfo/fluid-work
>