Tensorflow and AEM

It has been a while since google released Tensorflow support for java. Even though it is still in its infancy, I feel like it has everything we need. Build computation graphs - check, run session and compute stuff - check, GPU support - check. Now if you have all the time in the world to reinvent the wheel, you can pretty much build anything in java that we can build using python or c++.

So, I have been working on Adobe Experience Manager since I joined Adobe and recently, I started experimenting with several use cases where machine learning can help in content creation and discovery. As I have zero knowledge in building any deep learning models in java, I decided to build everything in java. How hard can it be? Right? Right? Sarcasm aside, as I mentioned earlier, Tensorflow for java has everything we need and as it internally uses JNI we can have interoperability with python and c++ (that's why I preferred this over deeplearning4j).

First off, I followed their official guide for the setup and had to face a lot of hurdles along the way. In this post I will show you how I managed to successfully setup Tensorflow on AEM (or any felix based systems).

Step 1

Add the dependency to your pom.xml file. Note that the scope set to compile.

Step 2

Add this configuration to your maven-bundle-plugin.

Step 3

Build and install to your AEM instance. Then, navigate to /system/console/bundles/ and look for the bundle which contains the dependency. See if the "Exported Packages" section has the following packages:

Step 4

Install JNI if necessary (this is mentioned in the link that I shared earlier).

Then place the library file in the appropriate place.

Testing 

Lets write a simple sling servlet to check if everything is working as expected. Like I told earlier, Tensorflow for java is still in its infancy. So, I wrote a helper class a while back to manipulate the computation graph. Get GraphBuilder.java and place it where it is accessible to the sling servlet.

GraphBuilder.java

The following sling servlet includes things like:

• Creating a computation graph
• Creating placeholders, constants etc
• Arithmetic operations, matrix multiplication.
• Feeding data and computing values of placeholders.

When you go to /services/tftest you should get something like this:

4 -2 3 0 FLOAT tensor with shape [3, 3] 14 Testing done!

Now you can start building any deep learning model on AEM. Also, I will be writing about some of the real life applications of deep learning in content creation and content discovery. So stay tuned!

Most original prize at The 2017 Deep Learning Hackathon

Although I have worked on several deep learning projects in the past, I still consider myself to be a newbie in deep learning because of all the new things that keep coming up and it is so hard to keep up with all that. So, I decided to take part in "The 2017 Deep Learning Hackathon" by Deepgram to work on something I have been wanting to do for a while now.

I built something called Medivh - prophet from Warcraft who has seen the future.  The idea was to build a tool for web developers to predict how users are going to see / use the site even before deploying. Basically, it generates heat maps on websites which show where the user might look at. Example:

I will write another post with all the technical details. Here is the sneak peak of how it was done.

Apart from building that, We got an opportunity to interact with people like Bryan Catanzaro - maker of CUDNN and VP at Nvidia,  Jiaji Huangform from Baidu, Jonathan Hseu from Google Brain etc.

We also got to interact with people from Deepgram and their caffe like framework called Kur which seems pretty good. I think I'll write a review about Kur after playing around with it for some more time.

Also this:

Shenanigans at the @DeepgramAI #DLhackathon !! #ai #hackathon pic.twitter.com/ympPZpPWFG

— BEAST Pets (@beastpets) March 26, 2017

This is me presenting before the results.

@DeepgramAI #deeplearning #hackathon @GPUComputing @awscloud @googlecloud class activation mapping web dev guidance @Adobe engineering #AI pic.twitter.com/sDD4vrIUAf

— Leo K Tam (@LeoKTam) March 26, 2017

For Medivh, I won the "Most original prize" -  Nvidia Titan X pascal. What a beauty!

Introducing mailing in crontab-ui

Now crontab-ui has option to send mails after execution of jobs along with output and errors attached as text files. This internally uses nodemailer and all the options available through nodemailer are available here.

Defaults

To change the default transporter and mail config you can modify config/mailconfig.js.

var transporterStr = 'smtps://user%40gmail.com:password@smtp.gmail.com';

var mailOptions = {
    from: '"Fred Foo 👥" <foo@blurdybloop.com>', // sender address
    to: 'bar@blurdybloop.com, baz@blurdybloop.com', // list of receivers
    subject: 'Job Test#21 Executed ✔', // Subject line
    text: 'Test#21 results attached 🐴', // plaintext body
    html: '<b>Test#21 🐴</b> results attached' // html body
};

Troubleshooting

Make sure that you have node at /usr/local/bin/node else you need to create a softlink like this

ln -s [location of node] /usr/local/bin/node

Setting up crontab-ui on raspberry pi

In this tutorial I will show you how to setup crontab-ui on raspberry pi.

Step 1

Find your architecture

uname -a
Linux raspberrypi 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux

Note that it is ARMv7. Download and extract latest node.

wget https://nodejs.org/dist/v7.7.2/node-v7.7.2-linux-armv7l.tar.xz
tar xz node-v7.7.2-linux-armv7l.tar.xz
sudo mv node-v7.7.2-linux-armv7l /opt/node

Step 2

Remove old nodejs if it is already installed and add the latest node to the $PATH

sudo apt-get purge nodejs
echo 'export PATH=$PATH:/opt/node/bin' > ~/.bashrc
source ~/.bashrc

Step 3

Install crontab-ui and pm2. And start crontab-ui.

npm install -g crontab-ui
npm install -g pm2
pm2 start crontab-ui

Now your crontab-ui must be running. Visit http://localhost:8000 on your browser to see if it is working.

Step 4 (Optional)

In order to be able access crontab-ui from outside, you have to forward the port 8000. Install nginx and configure.

sudo apt-get install nginx
sudo vi /etc/nginx/sites-available/default

Paste the following lines in the file:

server {
    listen 8001;

    server_name localhost;

    location / {
        proxy_pass http://localhost:8000;
    }
}

Restart nginx

sudo service nginx restart

Now, crontab-ui must be accessible from outside through port 8001. So, to access crontab-ui, go to

<ip address of pi>:8001

You can also setup http authentication by following this.
Thanks!
Fork me on Github

My solutions to cmdchallenge

I recently stumbled upon https://cmdchallenge.com which sort of tests your command line knowledge and comfortability. You have to basically solve all the challenges in a single line of bash. It is pretty simple and fun. You should give it a try before checking the solutions.

hello_world/

# Print "hello world".
# Hint: There are many ways to print text on
# the command line, one way is with the 'echo'
# command.
# 
# Try it below and good luck!
# 

Solution:
echo "hello world"

current_working_directory/

# Print the current working directory.
#

Solution:
pwd

list_files/

# List all of the files in the current
# directory, one file per line.
#

Solution:
ls -1

last_lines/

# Print the last 5 lines of "access.log".
# 

Solution:
tail -5 access.log

find_string_in_a_file/

# There is a file named "access.log" in the
# current working directory. Print all lines
# in this file that contains the string "GET".
#

Solution:
grep GET access.log

search_for_files_containing_string/

# Print all files, one per line that contain
# the string "500".
# 

Solution:
grep -rl * -e 500

search_for_files_by_extension/

# Print the relative file paths, one path
# per line for all files that start with
# "access.log" in the current directory.
# 

Solution:
find . -name "access.log*"

search_for_string_in_files_recursive/

# Print all matching lines (without the filename
# or the file path) in all files under the current
# directory that start with "access.log" that
# contain the string "500".
# 

Solution:
find . -name "access.log*" | xargs grep -h 500

extract_ip_addresses/

# Extract all IP addreses from files that
# that start with "access.log" printing one
# IP address per line.
# 

Solution:
find . -name "access.log*" | xargs grep -Eo '^[^ ]+'

delete_files/

# Delete all of the files in this challenge
# directory including all subdirectories and
# their contents.
# 

Solution:
find . -delete

count_files/

# Count the number of files in the current
# working directory. Print the number of
# files as a single integer.
# 

Solution:
ls | wc -l

simple_sort/

# Print the contents of access.log
# sorted.
# 

Solution:
sort access.log

count_string_in_line/

# Print the number of lines
# in access.log that contain the string
# "GET".
# 

Solution:
grep GET access.log | wc -l

split_on_a_char/

# The file split-me.txt contains a list of
# numbers separated by a ';' character.
# Split the numbers on the ';' character,
# one number per line.
# 

Solution:
cat split-me.txt | sed s/\;/\\n/g

print_number_sequence/

# Print the numbers 1 to 100 separated
# by spaces.
# 

Solution:
echo {1..100}

remove_files_with_extension/

# There are files in this challenge with
# different file extensions.
# Remove all files with the .doc extension
# recursively in the current working directory.
#

Solution:
find . -name "*.doc" -delete

replace_text_in_files/

# This challenge has text files that contain
# the phrase "challenges are difficult". Delete
# this phrase recursively from all text files.
# 

Solution:
find . -name "*.txt" -exec sed -i 's/challenges are difficult//g' {} +

sum_all_numbers/

# The file sum-me.txt have a list of numbers,
# one per line. Print the sum of these numbers.
#

Solution:
cat sum-me.txt | xargs | sed -e 's/\ /+/g' | bc

just_the_files/

# Print all files in the current directory
# recursively without the leading directory path.
# 

Solution:
find . -type f -printf "%f\n"

remove_extensions_from_files/

# Remove the extension from all files in
# the current directory recursively.
# 

Solution: (note you cant use find .)
find `pwd` -type f -exec bash -c 'mv "$1" "${1%.*}"' - '{}' \;

replace_spaces_in_filenames/

# The files in this challenge contain spaces.
# List all of the files in the current
# directory but replace all spaces with a '.'
# character.
# 

Solution:
find . -type f -printf "%f\n" | xargs -0 -I {} echo {} | tr ' ' '.'

files_starting_with_a_number/

# There are a mix of files in this directory
# that start with letters and numbers. Print
# the filenames (just the filenames) of all
# files that start with a number recursively
# in the current directory.
# 

Solution:
find . -name '[0-9]*' -type f -printf "%f\n"

print_nth_line/

# Print the 25th line of the file faces.txt
# 

Solution:
sed '25q;d' faces.txt

remove_duplicate_lines/

# Print the file faces.txt, but only print the first instance of each
# duplicate line, even if the duplicates don't appear next to each other.
# 

Solution:
awk '!seen[$0]++' faces.txt

corrupted_text/

# You have a new challenge!
# The following excerpt from War and Peace is saved to
# the file 'war_and_peace.txt':
# 
# She is betraying us! Russia alone must save Europe.
# Our gracious sovereign recognizes his high vocation
# and will be true to it. That is the one thing I have
# faith in! Our good and wonderful sovereign has to
# perform the noblest role on earth, and he is so virtuous
# and noble that God will not forsake him. He will fulfill
# his vocation and crush the hydra of revolution, which
# has become more terrible than ever in the person of this
# murderer and villain!
# 
# The file however has been corrupted, there are random '!'
# marks inserted throughout.  Print the original text.
# 

Solution: (Found this on hackernews)
< war_and_peace.txt tr -s '!' | sed 's/!\([a-z]\)/\1/g' | sed 's/!\( [a-z]\)/\1/g' | sed 's/!\.!/./g' | sed 's/ !/ /g'


Also, you can checkout the creator's solutions here.

Look before you paste from a website to terminal

Most of the time when we see a code snippet online to do something, we often blindly copy paste it to the terminal. Even the tech savy ones just see it on the website before copy pasting. Here is why you shouldn't do this. Try pasting the following line to your terminal (SFW)

ls ; clear; echo 'Haha! You gave me access to your computer with sudo!'; echo -ne 'h4cking ## (10%)\r'; sleep 0.3; echo -ne 'h4cking ### (20%)\r'; sleep 0.3; echo -ne 'h4cking ##### (33%)\r'; sleep 0.3; echo -ne 'h4cking ####### (40%)\r'; sleep 0.3; echo -ne 'h4cking ########## (50%)\r'; sleep 0.3; echo -ne 'h4cking ############# (66%)\r'; sleep 0.3; echo -ne 'h4cking ##################### (99%)\r'; sleep 0.3; echo -ne 'h4cking ####################### (100%)\r'; echo -ne '\n'; echo 'Hacking complete.'; echo 'Use GUI interface using visual basic to track my IP'
ls -lat

It should look something like this once it is pasted onto your terminal.

View post on imgur.com

You probably guessed it. There is some malicious code between ls and -lat that is hidden from the user

Malicious code's color is set to that of the background, it's font size is set to 0, it is moved away from rest of the code and it is made un-selectable (that blue color thing doesn't reveal it); to make sure that it works in all possible OSes, browsers and screen sizes.


This can be worse. If the code snippet had a command with sudo for instance, the malicious code will have sudo access too. Or, it can silently install a keylogger on your machine; possibilities are endless. So, the lesson here is, make sure that you paste code snippets from untrusted sources onto a text editor before executing it.

Thanks for reading!