Jarvis, at your sevice

Hello everyone! Its been a long time since I blogged. Today I will show you how to use 'Jarvis' which is an open source software which can be used to control your Linux system using your hand motion, gestures which was made as my Human Computer Interaction project. This is mainly an image processing based project developed using python.

Things u can do using Jarvis:

1. The first thing you can do using Jarvis is control your mouse. You just need a colored object (preferably has color different from its background). So you can do things like draw in air!
2. The second thing is that you can assign any gesture which is a combination of
   Left->Right, Right->Left, Top->Bottom, Bottom->Top to any command. So using this you can literally perform anything!!! The following are the things you can do by default:

• Maximize/Minimize/Close current window
• Go next and forward in PPT presentation
• Page up and Page down
• Switch window (Alt+tab)
• Take screenshot
• Shutdown/Suspend system
• Mute and unmute
• Open Calculator, File manager, Gedit

You can practically add anything else also. All you need to know is the command which does that and the equivalent of the gesture which you want to assign in the combination of Left->Right, Right->Left, Top->Bottom, Bottom->Top.

Getting dependencies:

Ubuntu:
$ sudo apt-get install python-opencv xdotool
Fedora:
$ sudo yum install python-opencv xdotool

Installation:

$ git clone https://github.com/alseambusher/jarvis.git
$ cd jarvis
$ ./install

Now you need to set the screen resolution of your screen. If your screen resolution is 1366x768 then skip this step.
$ gedit ~/.jarvis/config.py
change the value of variable RESOLUTION corresponding to your screen resolution.

Running:

Now simply open Jarvis from your Applications menu
OR
Do this:
$ cd ~/.jarvis
$ python main.py
Now click on Start Jarvis 

Add new gesture:

1. Go to settings from the File menu

2. Click on Add gesture from the File menu of settings

3. Suppose say that you want to add a gesture which opens terminal.
Say the gesture you wish to give is (Left to Right)->(Right to Left)->(Top to Bottom)->(Bottom to Top)
The command corresponding to open gnome terminal is 'gnome-terminal'

4. Fill the details and save it.

You are done!!!

Editing and deleting gestures are simple :P

How to use?

We need two different colored objects which are required to run. One the tracker and the other one is the flag!

1. If the flag is not exposed then the gesture is disabled and Jarvis works as a mouse controller.
2. When both tracker and flag are exposed the gesture begins. Perform the gesture using the tracker. Once the gesture is complete hide the flag. Jarvis then processes and analyses the gesture performed and checks for any matches from the existing database. If there is a match then it executes it!.

Customizing Tracker and Flag color:

By default the tracker is yellow color and flag is blue color.
You can change it by editing the config.py file

$ gedit ~/.jarvis/config.py

Change the min and max values of TRACKER_COLOR and GESTURE_COLOR corresponding to the HSV values of the color intended

By default these are the values:
TRACKER_COLOR={'MIN':[20,100,100],'MAX':[30,255,255]}
GESTURE_COLOR={'MIN': [108.0, 100, 10],'MAX': [118.0, 255, 255]}

Thank you. Dont forget to contribute to this open source project as there is a lot of scope for improvements. :)

Fork the project from here: https://github.com/alseambusher/jarvis