Sunday 17 December 2017

    Tensorflow and AEM

    It has been a while since google released Tensorflow support for java. Even though it is still in its infancy, I feel like it has everything we need. Build computation graphs - check, run session and compute stuff - check, GPU support - check. Now if you have all the time in the world to reinvent the wheel, you can pretty much build anything in java that we can build using python or c++.

    So, I have been working on Adobe Experience Manager since I joined Adobe and recently, I started experimenting with several use cases where machine learning can help in content creation and discovery. As I have zero knowledge in building any deep learning models in java, I decided to build everything in java. How hard can it be? Right? Right? Sarcasm aside, as I mentioned earlier, Tensorflow for java has everything we need and as it internally uses JNI we can have interoperability with python and c++ (that's why I preferred this over deeplearning4j).

    First off, I followed their official guide for the setup and had to face a lot of hurdles along the way. In this post I will show you how I managed to successfully setup Tensorflow on AEM (or any felix based systems).

    Step 1

    Add the dependency to your pom.xml file. Note that the scope set to compile.

    Step 2

    Add this configuration to your maven-bundle-plugin.

    Step 3

    Build and install to your AEM instance. Then, navigate to /system/console/bundles/ and look for the bundle which contains the dependency. See if the "Exported Packages" section has the following packages:

    Step 4

    Install JNI if necessary (this is mentioned in the link that I shared earlier).

    Then place the library file in the appropriate place.


    Lets write a simple sling servlet to check if everything is working as expected. Like I told earlier, Tensorflow for java is still in its infancy. So, I wrote a helper class a while back to manipulate the computation graph. Get and place it where it is accessible to the sling servlet.

    The following sling servlet includes things like:
    • Creating a computation graph
    • Creating placeholders, constants etc
    • Arithmetic operations, matrix multiplication.
    • Feeding data and computing values of placeholders.

    When you go to /services/tftest you should get something like this:

    4 -2 3 0 FLOAT tensor with shape [3, 3] 14 Testing done!
    Now you can start building any deep learning model on AEM. Also, I will be writing about some of the real life applications of deep learning in content creation and content discovery. So stay tuned!