[Architecture] Headless mode coming for Chrome...

Tirloni, Giovanni gtirloni at ocadu.ca
Tue Jul 3 16:03:08 UTC 2018 

Hi Justin,

   Using vagrant and full browsers locally will be easier to troubleshoot and test small changes in a dev environment.

   Tony also raised a good point that, in order to test Windows browsers, we can't use Docker (because it runs on Linux without a desktop environment). I sometimes forget we're working with frontend components and need to test different environments :|

   Thinking more about this, the workflow I've proposed in the PR is actually mainly for backend components that will be deployed to Linux servers. It does seem like a good replacement for the *current* CI pipeline but it would hinder our ability to expand the test cases to other OSes.

   It's a complicated set of requirements to satisfy in a single way though:

   * Dev environment that's flexible for iterative programming
   * Tests that run locally and on the cloud, while minimizing run time and resource comsumption
   * Local environments that are as close as possible to production, to minimize surprises
   * UX is easy for end users but is powerful for more elaborated tests on the servers (complex scenarios that would be difficult to run on a laptop)

   All of this is easier with satisfy for backend microservices but that's not our reality. I think a hybrid approach might be doable for microservices (i.e. dev environment is Vagrant-based but tests run inside Docker containers... while on the CI servers they only run inside Docker containers without the virtualization layer). The improvement in test run time is probably not worth the complexity, considering the number of builds per week that we have.

   I'm working with Tony on testing out other tools and scenarios and will have more to share in the following weeks but would love to hear more about what people think in the meantime.

Cheers,
Giovanni

On 07/03/2018 10:33 AM, Justin Obara wrote:
> Hi Gio,
> 
> Would we still use vagrant and full browsers when running testem locally? Or would we switch that to the same setup as CI as well?
> 
> Thanks
> Justin
> 
> 
> On June 29, 2018 at 6:17:50 PM, Tirloni, Giovanni (gtirloni at ocadu.ca <mailto:gtirloni at ocadu.ca>) wrote:
> 
>> Closed that PR as it was getting too polluted with all the different things I was trying.
>>
>> Here's a new and simplified PR: https://github.com/fluid-project/infusion/pull/910
>>
>> Thanks!
>>
>> On 06/18/2018 02:09 PM, Tirloni, Giovanni wrote:
>> > Hello, 
>> > 
>> > I'm proposing we switch infusion's CI tests to use containers and headless browsers:
>> > 
>> > https://github.com/fluid-project/infusion/pull/909
>> > 
>> > I'd like to thank Tony for his work on enabling headless browsers in gpii-testem. It made everything easier while working on this.
>> > 
>> > All feedback is welcome! 
>> > 
>> > Cheers, 
>> > Giovanni 
>> > 
>> > On 05/30/2018 10:44 AM, Tirloni, Giovanni wrote: 
>> >> With Firefox 60 and Chrome 66, all tests pass in headless mode for fluid-project/infusion.
>> >> 
>> >> 
>> >> 
>> >> gpii-testem$ git diff src/js/testem-component.js 
>> >> 
>> >> diff --git a/src/js/testem-component.js b/src/js/testem-component.js 
>> >> index 60b0968..416836c 100644 
>> >> --- a/src/js/testem-component.js 
>> >> +++ b/src/js/testem-component.js 
>> >> @@ -429,10 +429,9 @@ fluid.defaults("gpii.testem.base", { 
>> >> ] 
>> >> }, 
>> >> "headlessBrowserArgs": { 
>> >> - // TODO: enable once a new enough version of Firefox is available in CI. 
>> >> - // "Firefox": [ 
>> >> - // "--headless" 
>> >> - // ], 
>> >> + "Firefox": [ 
>> >> + "--headless" 
>> >> + ], 
>> >> // See this ticket for details on the minimum options required to get "headless" Chrome working: https://github.com/testem/testem/issues/1106#issuecomment-298841383
>> >> "Chrome": [ 
>> >> "--disable-gpu", 
>> >> 
>> >> 
>> >> 
>> >> If this is all there's to it, we can start migrating CI to headless if people are confident in the new headless features in Chrome/Firefox.
>> >> 
>> >> 
>> >> On 06/06/2017 11:01 AM, Tirloni, Giovanni wrote: 
>> >>> On 07/11/2016 05:56 AM, Tony Atkins wrote: 
>> >>>> To continue our ongoing discussions around browser testing, I wanted to point out the upcoming release of a "headless" mode for Chromium.
>> >>>> https://bugs.chromium.org/p/chromium/issues/detail?id=546953
>> >>> 
>> >>> Hello, 
>> >>> 
>> >>> I tested Chrome 59 and Firefox 55 Nightly in headless mode [0][1]. Everything worked fine with Chrome (all tests pass).
>> >>> 
>> >>> Firefox is not quite there yet (multiple failures in Infusion and OAuth2 fail in GPII/universal).
>> >>> 
>> >>> This should greatly simplify our CI setup, once it's ready (no need for Xorg/Xfvb, the browser could be in a container image).
>> >>> 
>> >>> 
>> >>> 0 - https://bugzilla.mozilla.org/show_bug.cgi?id=1338004
>> >>> 1 - https://www.mozilla.org/en-US/firefox/channel/desktop/
>> >>> 
>> >>> Regards, 
>> >>> Giovanni 
>> >>> 
>> >>> 
>> >>> 
>> >>> #GPII/universal/tests/web/testem_qi.json: 
>> >>> 
>> >>> { 
>> >>> "test_page": "tests/web/html/all-tests.html", 
>> >>> "reporter": "tap", 
>> >>> "report_file": "report.tap", 
>> >>> "browser_args": { 
>> >>> "Chrome": [ "--headless", "--disable-gpu", "--remote-debugging-port=9222" ] 
>> >>> }, 
>> >>> "launchers": { 
>> >>> "Firefox Nightly": { 
>> >>> "command": "/home/gtirloni/Downloads/firefox-nightly/firefox/firefox --headless <url>",
>> >>> "protocol": "browser" 
>> >>> } 
>> >>> } 
>> >>> 
>> >>> } 
>> >>> 
>> >>> # fluid-project/infusion/tests/testem.json 
>> >>> 
>> >>> { 
>> >>> "test_page": "tests/all-tests.html", 
>> >>> "timeout": 300, 
>> >>> "reporter": "tap", 
>> >>> "report_file": "report.tap", 
>> >>> "browser_args": { 
>> >>> "Chrome": [ "--headless", "--disable-gpu", "--remote-debugging-port=9222" ] 
>> >>> }, 
>> >>> "launchers": { 
>> >>> "Node Module Basic Packaging Tests": { 
>> >>> "command": "node tests/node-tests/basic-node-tests.js --tap", 
>> >>> "protocol": "tap" 
>> >>> }, 
>> >>> "Firefox Nightly": { 
>> >>> "command": "/home/gtirloni/Downloads/firefox-nightly/firefox/firefox --headless <url>",
>> >>> "protocol": "browser" 
>> >>> } 
>> >>> } 
>> >>> } 
>> >>> 
>> >>> 
>> >>> 
>> >> _______________________________________________________ 
>> >> fluid-work mailing list - fluid-work at lists.idrc.ocad.ca <mailto:fluid-work at lists.idrc.ocad.ca>
>> >> To unsubscribe, change settings or access archives, 
>> >> see https://lists.idrc.ocad.ca/mailman/listinfo/fluid-work
>> >> 
>> > _______________________________________________________ 
>> > fluid-work mailing list - fluid-work at lists.idrc.ocad.ca <mailto:fluid-work at lists.idrc.ocad.ca>
>> > To unsubscribe, change settings or access archives, 
>> > see https://lists.idrc.ocad.ca/mailman/listinfo/fluid-work
>> > 
>> _______________________________________________________
>> fluid-work mailing list - fluid-work at lists.idrc.ocad.ca <mailto:fluid-work at lists.idrc.ocad.ca>
>> To unsubscribe, change settings or access archives,
>> see https://lists.idrc.ocad.ca/mailman/listinfo/fluid-work

More information about the fluid-work mailing list