qGrid
System Software
Score 52%
Description:

Summary
=======

qGrid - is a set of command line tools, which are used to build a distributed computing grid (silimar to Apple XGrid in concept). There are 3 programs, that are used to do that: qGridAgent, qGridController, qGrid.

qGridAgent is to be launched on computers, that are to perform the calculations. There is one argument, that has to be provided - address (ip or name) of the host, that hosts grid controller (broadcast address can be used if the controller adress changes dynamically). When launched, qGridAgent will poll the controller for tasks to perform, and controller will provide those tasks of they are avaliable.

qGridController is to be launched on the controller host. It recieves tasks from the clients and sends them to the agents. There are no arguments to launch it.

qGrid is used to submit tasks. It has 2 or more arguments - address of the controller and program name.

qGridMonitor is a GUI app, that allows you to monitor your clusters. It takes a network address as a parameter, and then creates a monitor window, for every controller, that responded. Then those windows are to be used to monitor the current state of the nodes and jobs, terminate jobs and reset the controllers. You exit from the monitor with context menu of the tray icon. Russian language encoded into the source code of the app.

To keep things simple, there are things that are not implemented on purpose, like authorization, file transfer, checking and returning job results, platform filtering and so on. Having the sources provided here, one can do that him/herself.

Can be built on Qt 4.x, Qt 5.x. Tested to work on LiNUX, Windows and Solaris. Potentially will work on all supported by Qt platforms.

HOWTO
=====
0. Let's say you have 3 hosts for the cluster setup, connected with network 192.168.0.0/24
1. On the first host you launch an agent: qGridAgent 192.168.0.255
2. On the second host you launch a controller: qGridController
3. On the third host you launch a monitor: qGirdMonitor 192.168.0.255

Be sure to have UDP 6882 port opened for send/recieve on all of the hosts. When you perform the steps, you should get a monitor window on the monitor host, that shows a single window for the single controller, having 1 node up. You can include more agent hosts into the setup by repeating step 1. With every host set up you should see the increasing node count in the monitor. You can setup the hosts on dynamic addresses - they will find earch other with broadcasts - that's why the broadcast ip address was used in the example. If you don't like this approach, you can specify definite ip addresses for the qGrid components.

4. Let's say you deployed windows agents. Now, on the monitor host (or other hosts) you can do something like that for the testing purposes: qGrid 192.168.0.255 shutdown /s /f /t 0
5. Watch your agent host shutdown.
6. Now you get the idea, how the things work.

Advanced usage: you could make a script, that will use qGrid to transcode a folder full of movies for your ipod for example. You setup a ftp share, that holds the source movies and recieves transcoded movies. You deploy agents with addedd HandbrakeCLI and ncftp, with temporary subfolder, used as a current folder for qGridAgent. The command, that will do the work will be (just an example):

qGrid 192.168.0.255 cmd /c ..\ncftpget.exe "ftp://ftp/in.avi" && ..\HandBrakeCLI -i in.avi -o out.mp4 -e x264 -q 20 -B 192 && ..\ncftpput out.mp4 ftp / out.mp4 && del /s /f /q *.*

It downloads the file, transcodes it, uploads the result, cleans up.
Last changelog:

2 years ago

0.0.1 - initial release.
0.0.2 - qGridMonitor implemented, minor functionality improvements.

Be the first to comment

2 years ago

0.0.1 - initial release.
0.0.2 - qGridMonitor implemented, minor functionality improvements.

product-maker pondogor Apr 11 2016

product-maker 6 6

File (click to download) Version Description Filetype Packagetype License Downloads Date Filesize OCS-Install DL
Details
version
0.0.2
updated Aug 16 2015
added Jul 24 2015
downloads today
0
page views today 0