Life plus Linux: December 2017

Posts

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
Magic Cauldron: Harry Potter Themed Gender Reveal Party - #Aurdino
Earlier this year, we had a very fun filled Harry Potter themed gender reveal party. For the reveal, I built a Magic Cauldron which would reveal the gender. Check it out for yourself! For this I needed: A Cauldron. WS2812B LED array. Aurdino UNO. Bread board and jumper wires. Dry ice. Kasa Smart bulbs I will go over in the following sections The Mist.
Kakashi: The Copycat Robot - #Aurdino #image processing
In this post, I want to share about "Kakashi: The Copycat Robot"—a fun side project I built a few years ago. The name is inspired by the famous character from Naruto, Kakashi Hatake, also known as the Copycat Ninja.The goal of this robot was to mimic Kakashi's ability to copy movements—though, of course, in a more limited way. Check it out for yourself!Here are the things I used to build this:
Neural network inference pipeline for videos in Tensorflow - #Deeplearning #Tensorflow
Just as we saw a huge influx of images in the past decade or so, we are now seeing a lot of videos being produced on social media. The need to understand and moderate videos using machine learning has never been greater. In this post, I will show you how to build an efficient pipeline to processes videos in Tensorflow.  For simplicity, let us consider a Resnet50 model pre-trained on
Finding Where's Waldo using Mask R-CNN - #Deeplearning #ML
When I was a kid, I really loved solving Where's Waldo. There were few books (it used to be called Where's Wally) in our school library on which I spent hours finding Waldo. For people who do not know what it is, basically Waldo - a unique character is hidden among hundreds of other characters and you have to find him in all the chaos in the image. Now that I am too old to be solving it and
Higher level ops for building neural network layers with deeplearn.js - #Deeplearning #javascript #ML
I have been meddling with google's deeplearn.js lately for fun. It is surprisingly good given how new the project is and it seems to have a sold roadmap. However it still lacks something like tf.layers and tf.contrib.layers which have many higher level functions that has made using tensorflow so easy. It looks like they will be added to Graphlayers in future but their priorities as of now is to

Sunday, 17 December 2017

Tensorflow and AEM

It has been a while since google released Tensorflow support for java. Even though it is still in its infancy, I feel like it has everything we need. Build computation graphs - check, run session and compute stuff - check, GPU support - check. Now if you have all the time in the world to reinvent the wheel, you can pretty much build anything in java that we can build using python or c++.

So, I have been working on Adobe Experience Manager since I joined Adobe and recently, I started experimenting with several use cases where machine learning can help in content creation and discovery. As I have zero knowledge in building any deep learning models in java, I decided to build everything in java. How hard can it be? Right? Right? Sarcasm aside, as I mentioned earlier, Tensorflow for java has everything we need and as it internally uses JNI we can have interoperability with python and c++ (that's why I preferred this over deeplearning4j).

First off, I followed their official guide for the setup and had to face a lot of hurdles along the way. In this post I will show you how I managed to successfully setup Tensorflow on AEM (or any felix based systems).

Step 1

Add the dependency to your pom.xml file. Note that the scope set to compile.

Step 2

Add this configuration to your maven-bundle-plugin.

Step 3

Build and install to your AEM instance. Then, navigate to /system/console/bundles/ and look for the bundle which contains the dependency. See if the "Exported Packages" section has the following packages:

Step 4

Install JNI if necessary (this is mentioned in the link that I shared earlier).

Then place the library file in the appropriate place.

Testing

Lets write a simple sling servlet to check if everything is working as expected. Like I told earlier, Tensorflow for java is still in its infancy. So, I wrote a helper class a while back to manipulate the computation graph. Get GraphBuilder.java and place it where it is accessible to the sling servlet.

GraphBuilder.java

The following sling servlet includes things like:

Creating a computation graph
Creating placeholders, constants etc
Arithmetic operations, matrix multiplication.
Feeding data and computing values of placeholders.

When you go to /services/tftest you should get something like this:

4 -2 3 0 FLOAT tensor with shape [3, 3] 14 Testing done!

Now you can start building any deep learning model on AEM. Also, I will be writing about some of the real life applications of deep learning in content creation and content discovery. So stay tuned!