Posts

  • 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
  • Magic Cauldron: Harry Potter Themed Gender Reveal Party - #Aurdino

    Earlier this year, we had a very fun filled Harry Potter themed gender reveal party. For the reveal, I built a Magic Cauldron which would reveal the gender. Check it out for yourself! For this I needed: A Cauldron. WS2812B LED array. Aurdino UNO. Bread board and jumper wires. Dry ice. Kasa Smart bulbs I will go over in the following sections The Mist.

  • Kakashi: The Copycat Robot - #Aurdino #image processing

    In this post, I want to share about "Kakashi: The Copycat Robot"—a fun side project I built a few years ago. The name is inspired by the famous character from Naruto, Kakashi Hatake, also known as the Copycat Ninja.The goal of this robot was to mimic Kakashi's ability to copy movements—though, of course, in a more limited way. Check it out for yourself!Here are the things I used to build this:

  • Neural network inference pipeline for videos in Tensorflow - #Deeplearning #Tensorflow

    Just as we saw a huge influx of images in the past decade or so, we are now seeing a lot of videos being produced on social media. The need to understand and moderate videos using machine learning has never been greater. In this post, I will show you how to build an efficient pipeline to processes videos in Tensorflow.  For simplicity, let us consider a Resnet50 model pre-trained on

  • Finding Where's Waldo using Mask R-CNN - #Deeplearning #ML

    When I was a kid, I really loved solving Where's Waldo. There were few books (it used to be called Where's Wally) in our school library on which I spent hours finding Waldo. For people who do not know what it is, basically Waldo - a unique character is hidden among hundreds of other characters and you have to find him in all the chaos in the image. Now that I am too old to be solving it and

  • Higher level ops for building neural network layers with deeplearn.js - #Deeplearning #javascript #ML

    I have been meddling with google's deeplearn.js lately for fun. It is surprisingly good given how new the project is and it seems to have a sold roadmap. However it still lacks something like tf.layers and tf.contrib.layers which have many higher level functions that has made using tensorflow so easy. It looks like they will be added to Graphlayers in future but their priorities as of now is to

Saturday, 26 October 2013

Performance modelling of a linux system using fork bomb


Today I will be showing you a simple way to measure the memory allocation capacity of the system.

Unlike in case of httpperf my method doesn't rely on the network based requests to load the system

I will be loading the system with processes by running a shell script in each of the guest operating systems(If the system under test is a server virtualized system) The shell scripts can be controlled remotely using telnet.

I would like to measure active memory, free memory, swap memory, inactive memory while loading the system with processes.

The idea here is to get all the required performance metrics from /proc/meminfo while forking processes. For example at one particular instance on my system
/proc/meminfo:



MemTotal:        2945376 kB
MemFree:          765960 kB
Buffers:          144004 kB
Cached:           993792 kB
SwapCached:            0 kB
Active:           979460 kB
Inactive:         995496 kB
Active(anon):     838076 kB
Inactive(anon):   178496 kB
Active(file):     141384 kB
Inactive(file):   817000 kB
Unevictable:          84 kB
Mlocked:              84 kB
SwapTotal:       6261756 kB
SwapFree:        6261756 kB
Dirty:               800 kB
Writeback:             0 kB
AnonPages:        837144 kB
Mapped:           213612 kB
Shmem:            179416 kB
Slab:              91864 kB
SReclaimable:      60272 kB
SUnreclaim:        31592 kB
KernelStack:        4000 kB
PageTables:        39688 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     7734444 kB
Committed_AS:    4186924 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      556904 kB
VmallocChunk:   34359177692 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       63488 kB
DirectMap2M:     2942976 kB
=======

Also If we want to track any process, the folder /proc has details about each and every process running on the system.

Regarding loading the system:

There are two types of loads . Linear load  and Exponential load
In linear load, one process forks another process and waits for it to exit. The spawned process forks another process and waits for its child to exit. This keeps going on and on till the system eventually fails. To load the system with linear load we ran the following shell script:

#!/bin/bash
$0 &
wait

"$0 &" will run the same script in the background and 'wait' will make the current process to wait till the spawned process returns.

Similarly exponential load can be given by spawning 2 processes instead of one.
#!/bin/bash
$0 &
$0 &
wait

Collecting data:
The following code has to be embedded into the above script
# <epoch time> <PID> <Memory free> <swap free> <active memory> <inactive memory>

string=`date +%s`" "$$
i=`cat /proc/meminfo| grep "MemFree:"`
string=$string" "`expr "$i" : ".* \(.*\) kB"`
i=`cat /proc/meminfo| grep "SwapFree:"`
string=$string" "`expr "$i" : ".* \(.*\) kB"`
i=`cat /proc/meminfo| grep "Active:"`
string=$string" "`expr "$i" : ".* \(.*\) kB"`
i=`cat /proc/meminfo| grep "Inactive:"`
string=$string" "`expr "$i" : ".* \(.*\) kB"`
echo $string>>log
Sample output for linear load:
Sample output for exponential load:

Error removing file: Directory not empty

Something really weird happened today. I transferred some data to my external hard disk and for some reason I unplugged it from my comp without doing 'safely remove drive' (reason being I was lazy :P). Then after sometime I put it back on my laptop and tried removing the directory called Main_Game which I had created earlier. It said 'Error while deleting. Error removing file: Directory not empty'.

Then i tried doing that from terminal using 'rm -rf Main_Game'. Again it didn't work. It said rm: cannot remove 'Main_Game': Directory not empty


Then I thought there might be some process using that folder. So I restarted and tried again. It didn't work :(

Then I thought there might be some error in the filesystem. So I did force fsck.

$ sudo touch /forcefsck
$ sudo reboot

It successfully completed fsck. However the problem wasnt fixed

Then I did something which I should've tried long back and it worked!!

I simply deleted the folder (not force delete). The folder Main_Game was moved to recycle bin. I just deleted from there.

I dunno how this worked though :P