GSoC'20 : Using a Game Controller as a Navigation Aid

zaid junaid zaidjunaid3 at
Sat Mar 14 18:30:42 UTC 2020

Hi  abhishek
I tried a similar thing and was given some input about the same, would like
to share it with you
Projects here tend to respect the full range of human diversity. the idea
you (and I) think of actually don't consider two major groups- people who
have mobile and visual impairment, so although this feature(navigating
through a pointer)  could act as an additional feature in the project, we
need to think of a better solution.
as for the frontend technology, we have our own Inclusion framework,
which we can use, you can have a look at it-

On Sat, Mar 14, 2020 at 11:38 PM Abhishek Garain <abhi211199 at>

> Hey all, My name is Abhishek Garain, currently a 3rd year BTech student of
> IIIT Gwalior, India interested in Application Development using Electron
> and React. I am also into Android Development. I am interested to work on
> the Gamepad Navigation Aid project during Inclusive Design Institute's
> GSoC'20 program.
> Many a times, I have faced a problem while playing games and all of a
> sudden, have to reach out for a mouse to attend a notification or to change
> the game. This becomes even more troublesome when the laptop is connected
> to a TV or Monitor. This is a problem, which is faced by the wider
> community of gamers and other users, which I would like to solve.
> My 30 minutes of experience of browsing the web without a mouse can be
> found here:
> Talking about the technical aspects of the project,
> 1. After analysing my experiences of browsing the web without a mouse, the
> main difficulty was to navigate to a particular link, To overcome this, I
> need to get hold of the pointer.
> Javascript can get pointer co-ordinates but can't control the pointer
> movements. Python on the other hand, can do so. As the axes provides analog
> inputs, the pointer can be shifted according to the scaling factor
> multiplied by the analog value.
> 2. Secondly, the keys which are regularly used, needs to mapped to the
> gamepad. Like opening a new tab with Ctrl+N may me mapped with a button.
> Swiping between tabs and scroll bar can also be mapped to the axes.
> 3.Thirdly, Speech to Text may be implemented for filling up forms or for
> giving voice commands.
> Based on my observations, I have made a prototype(implemented in Python)
> which can be found here:
> As per my resarch, the incorporation of Chrome functionalities can be
> implemented using a Chrome Extension(chrome.tabs API). But controlling the
> pointer needs Python in background. And for mapping the keys to a gamepad,
> we need an UI built on Electron or within the Extension itself. I am
> currently researching on various frontend technologies which goes hand in
> hand with a gamepad.
> I would like to get valuable feedback from mentors regarding my thoughts
> and prototype.
> Regards,
> Abhishek Garain
> _______________________________________________________
> fluid-work mailing list - fluid-work at
> To unsubscribe, change settings or access archives,
> see
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the fluid-work mailing list