Tensorflow and AEM

It has been a while since google released Tensorflow support for java. Even though it is still in its infancy, I feel like it has everything we need. Build computation graphs - check, run session and compute stuff - check, GPU support - check. Now if you have all the time in the world to reinvent the wheel, you can pretty much build anything in java that we can build using python or c++.

So, I have been working on Adobe Experience Manager since I joined Adobe and recently, I started experimenting with several use cases where machine learning can help in content creation and discovery. As I have zero knowledge in building any deep learning models in java, I decided to build everything in java. How hard can it be? Right? Right? Sarcasm aside, as I mentioned earlier, Tensorflow for java has everything we need and as it internally uses JNI we can have interoperability with python and c++ (that's why I preferred this over deeplearning4j).

First off, I followed their official guide for the setup and had to face a lot of hurdles along the way. In this post I will show you how I managed to successfully setup Tensorflow on AEM (or any felix based systems).

Step 1

Add the dependency to your pom.xml file. Note that the scope set to compile.

Step 2

Add this configuration to your maven-bundle-plugin.

Step 3

Build and install to your AEM instance. Then, navigate to /system/console/bundles/ and look for the bundle which contains the dependency. See if the "Exported Packages" section has the following packages:

Step 4

Install JNI if necessary (this is mentioned in the link that I shared earlier).

Then place the library file in the appropriate place.

Testing 

Lets write a simple sling servlet to check if everything is working as expected. Like I told earlier, Tensorflow for java is still in its infancy. So, I wrote a helper class a while back to manipulate the computation graph. Get GraphBuilder.java and place it where it is accessible to the sling servlet.

GraphBuilder.java

The following sling servlet includes things like:

• Creating a computation graph
• Creating placeholders, constants etc
• Arithmetic operations, matrix multiplication.
• Feeding data and computing values of placeholders.

When you go to /services/tftest you should get something like this:

4 -2 3 0 FLOAT tensor with shape [3, 3] 14 Testing done!

Now you can start building any deep learning model on AEM. Also, I will be writing about some of the real life applications of deep learning in content creation and content discovery. So stay tuned!