Non-public public functions: Conventions for identifying?

[Colin Clark]

Hey,

I'm reviving this thread now that we've finished up our demo sprint and have a bit more time to think these issues through.

So, I think the general consensus we came to last week was that we needed a clear distinction between the portions of our API that are:

1. in the process of becoming stable, core parts of Infusion
 vs.
2. not guaranteed to remain consistent from release to release but visible out of courtesy to developers

Code in category 1 fits our established definition of the Sneak Peek and Preview statuses. For category 2, I don't like the term "unstable," because it incorrectly implies that such code is buggy or prone to crashing. When we say unstable in this context, we actually mean that it's liable to change in the future due to new features and demands on the framework. It's code that wasn't designed to be relied on by the public, but is available in case we didn't anticipate a critical feature.

If it doesn't sound like too harsh a name, I'd suggest that code in category 2 be given a status of "unsupported," and clearly marked as such in the code with a comment, as well as in the API reference pages. 

The key for us as developers of Infusion is to ensure that only appropriate code gets into this category, rather than on track from sneak peek to production. If we do find that users actually need to use an unsupported function themselves, it suggests that we'll need to get that functionality into the stable public API.

Thoughts?

Colin

On 2010-11-10, at 3:12 AM, Antranig Basman wrote:

> Thanks for this reponse, Colin.
> The two of us had a chat after this thread on Monday which I wanted to summarise to the list. There are a few subtleties with this approach that we couldn't spend enough effort on trying to make clear :)
> 
> I started by mentioning that I didn't think the term "sneak peek" covered all of the cases for which we might make functions public that are not part of our stable, committed API. It carries the suggestion that everything we label in that way is something that we *do* have a definite intention to make stable and committed someday - however for example, giving the example of fluid.fetchResources.issueRequest that I mentioned, this is a function for which we have no such intention.
> 
> I then suggested that even though there were these two potentially distinct kinds of reason for supporting "public, unstable" functionality, it didn't seem important to distinguish to the users what the reason was - that all they might be interested in was that they were not stable and not to be relied on in a future release.
> 
> Colin then made the point that this is not necessarily true. There is an important issue of "dissemination and direction of interest" that goes beyond mere "use". For example, the API in our upcoming release "fluid.initDependents" whilst it is right now unstable, is something that we imminently intend to make stable, and is likely to become a highly important part of our infrastructure, used daily. A user who is interested in "getting acquainted with upcoming framework functions" would certainly, for example, be interested in investing time understanding this, "sneak peek" function. This is in contrast to, say, the example of "fluid.fetchResources.issueRequest" - this is not a fundamental feature that we have any definite intention about. A developer would only come to discover this function through a specific search in pursuit of delivering a particular kind of customisation to their users - it has been made public only as a "development courtesy".
> 
> So it seems clear that these two categories, broadly named "sneak peek" and "development courtesy" need to be conceived separately. This kind of clear focus seems necessary especially in regards of the dangers that can follow from being imprecise here - the "upside" of this approach is increased flexibility and the possibility of a more meaningful dialogue with our framework users about the kinds of APIs which will be successful. The "dark side" of this coin is that it creates an invitation for us to be sloppy about stabilising APIs and to continue being "exploratory" indefinitely - especially faced with the really high costs of maintaining API stability. Fluid is much more serious about this issue than most comparable frameworks (many of whom think nothing of making completely incompatible rewrites of their entire API from time to time) but we are now significantly feeling the pain of maintaining this stability under quite strong pressure to make API improvements.
> 
> One important technique that we plan to deploy for assuring API stability is the "options chewing framework". A piece of code that Colin wrote during the Fluid Engage period for "model transformation" is something that we plan to use to convert "old options to new" - that is, allowing implementations to evolve whilst exposing, on demand, interfaces conforming to historical versions. This sort of thing is only possible in light of Fluid's commitment to reducing all aspects of development and configuration to transparent and declarative forms.
> 
> We finished by observing that in the case of use of "development courtesy" APIs we would need to be very vigilant. I suggested that even a single attested use of one of these APIs by a framework client should strongly motivate us to stabilise that function as part of the public API.
> 
> On 08/11/2010 16:35, Colin Clark wrote:
>> Hey Antranig,
>> 
>> Wow, quite an essay! I think that you actually summarize your point quite well with this snippet:
>> 
>> On 2010-11-08, at 5:28 PM, Antranig Basman wrote:
>> 
>>> i) Fluid has a known and acknowledged audience of "expert and/or desperate users" who would prefer to knowingly make the choice of trading functionality for stability in the case they need to deliver new user features in a hurry - that is, their development cycles are significantly faster than Fluid can make a framework release.
>>> ii) Hiding functions in private scopes creates the risk of cut'n'paste code or even wholesale development of new approaches that can't be easily contributed back to the framework in future.
>>> iii) Writing core functions as public "utilities" forces us to think more clearly about their contract and also constrains their range of side-effects which improves the quality of the codebase in any case.
>> 
>> 
>> So, we're taking a different approach in Infusion, where we ensure that everything useful that we create is also available for our users to adapt and extend. This actually speaks to another issue touched on by Eli earlier in the thread--exposing functionality for testing. In reality, we've found that if a function needs to be made public in order to be unit tested, it's usually a sign that our users may also need that functionality in order to be able to fully customize or adapt the component. Our instinct to make something private is usually a sign of poor factoring, rather than a need to legitimately encapsulate it.
>> 
>> This approach puts a burden on us to think deeply about the contract and API of all our public functions, and means we have to do a better job of communicating which functions are in progress or subject to change in future versions.
>> 
>> I think we already have a tool for how to handle this sort of thing: our component status levels. In particular, the "Sneak Peek" status gives users an opportunity to work with new features early, yet clearly identifies APIs that are likely to change in the future. Perhaps we want to augment this by labelling certain functions as sneak peek and not providing detailed documentation for them at all. I'd suggest that developers can simply label such functions in the source code like this:
>> 
>> /**
>>  * This function has a status of sneak peek and is subject to change. As a result, it is currently undocumented.
>>  */
>> 
>> A comment like this should provide both our users and "the poor documentation sod," as Anastasia hilariously put it, with enough information to use bleeding-edge APIs wisely.
>> 
>> This brings us to Michelle's point: we need to compile a list of all the functions in Infusion 1.3 that are "sneak peek," as well as an accompanying list of all functions that we'd prefer to leave undocumented. Antranig, since you raised this issue originally, can you start us off by compiling a list of the functions you're aware of that are currently in this category?
>> 
>> Colin
>> 

---
Colin Clark
Technical Lead, Fluid Project
http://fluidproject.org