In the previous post I explained how to create your AWS lambda environment using Docker, and how to package a python bundle and launch it on AWS Lambda.
In this post I’ll show how you can launch arbitrary executables from an AWS Lambda function.
To make this tutorial even more useful, the example of an arbitrary executable I’ll be using is LuaJIT – an incredibly fast Lua implementation created by Mike Pall. After this you should be able to write blazing fast Lua code and run it on AWS Lambda.
I assume you already have a Docker container that emulates AWS Lambda Linux – if not, check the previous post
So, first thing is to install LuaJIT on the Docker amazon lambda container. Start the container for the amazon linux (use: docker ps -a or docker container list to find the container and docker start -i <name> to connect to it.
Once in the container, make sure you have wget and unzip installed. If not then:
yum install wget yum install zip
Next, download the latest version of LuaJIT (in my case this was 2.0.5) from here
wget http://luajit.org/download/LuaJIT-2.0.5.zip # download latest source of LuaJIT unzip LuaJIT-2.0.5.zip # unzip it cd LuaJIT-2.0.5 # go to source directory make # build LuaJIT make install # install LuaJIT
To run an arbitrary binary from AWS Lambda, we’ll first include and any dependencies it might have in the zip package that we’ll upload to Lambda.
So let’s create the ingredients of this package. For starters we’ll create a directory to place all the relevant files:
mkdir lambdalua cd lamdalua mkdir lib # we'll place any luajit dependencies here
Since we compiled and installed luajit, let’s check where it was placed:
which luajit
in my case, the result is: /usr/local/bin/luajit
Now, we’ll copy luajit to the directory we are in so it will be part of the package
cp /usr/local/bin/luajit .
Next, let’s check whether there are any dynamic linked libraries that luajit depends on, as they’ll need to exist on AWS Lambda too in order for luajit to successfully run:
ldd /usr/local/bin/luajit # find the shared libraries required by luajit
The result:
linux-vdso.so.1 => (0x00007ffdb75a7000) liblua-5.1.so => /usr/lib64/liblua-5.1.so (0x00007f6deea61000) libreadline.so.6 => /lib64/libreadline.so.6 (0x00007f6dee81c000) libncurses.so.5 => /lib64/libncurses.so.5 (0x00007f6dee5f6000) libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007f6dee3d5000) libm.so.6 => /lib64/libm.so.6 (0x00007f6dee0d3000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f6dedecf000) libc.so.6 => /lib64/libc.so.6 (0x00007f6dedb0b000) /lib64/ld-linux-x86-64.so.2 (0x00007f6deec8d000)
Above, we can see that most of the shared libraries luajit depends on (those starting with /lib64) are part of linux (and hopefully they are the same version as those on AWS Lambda amazon linux).
However, one file is not part of lambda linux, and that is /usr/lib64/liblua-5.1.so (this was added as part of installing luajit).
We’ll need to make this file available to luajit on lambda so let’s copy it to the lib/ directory we created.
cp /usr/lib64/liblua-5.1.so lib/
create the following hello.lua file in the directory we’re in:
local str = "hello from LuaJIT - " for i=1,10 do str = str .. i .. " " end print(str)
Now we create the Python file that will launch the above Lua script using LuaJIT. We’ll name this file lambdalua.py. Note the explanations in the comments within the code:
import subprocess import sys import os def lambda_luajit_func(event, context): lpath = os.path.dirname(os.path.realpath(__file__)) # the path where this file resides llib = lpath + '/lib/' # the path for luajit shared library # Since we can't execute or modify execution attributes for luajit in the directory # we run on aws lambda, we'll copy luajit to the /tmp directory where we'll be able # to change it's attributes os.system("cp -n %s/luajit /tmp/luajit" % (lpath)) # copy luajit to /tmp os.system("chmod u+x /tmp/luajit") # and make it executable # Since we don't have permission to copy luajit's shared library to the path # where it looks for it (the one shown from the ldd command), we'll add the # path where the liblua-5.1.so is located to the LD_LIBRARY_PATH, which enables # Linux to search for the shared library elsewhere # add our lib/ path to the search path for shared libraries os.environ["LD_LIBRARY_PATH"] += (":%s" % (llib)) # prepare a subprocess to run luajit with the hello.lua script path as a parameter command = "/tmp/luajit %s/hello.lua" % (lpath) p = subprocess.Popen(command , shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) # launch the process and read the result in stdout stdout, stderr = p.communicate() # We'll make the return of the lambda function the same as what was the output of the # Lua script return stdout.decode("utf-8") if __name__ == "__main__": print(lambda_luajit_func(None, None))
At this point, we can create a package to upload as a lambda function. In the directory where we’re in run:
zip -r package.zip .
and then copy the zip file to your host OS terminal, e.g:
docker cp lucid_poincare:/root/lambdalua/package.zip
now we’ll upload the package and create the lambda function using the same user and role created in the previous post (replace the role ARN with your own):
aws lambda create-function --region us-east-1 --function-name lambda_luajit_func --zip-file fileb://package.zip --role arn:aws:iam::123456789012:role/basic_lambda_role --handler lambdalua.lambda_luajit_func --runtime python3.6 --profile lambda_user
you should get a JSON reply with the information that the function has been created. Finally we can invoke the lambda function as follows:
aws lambda invoke --invocation-type RequestResponse --function-name lambda_luajit_func --region us-east-1 --log-type Tail --profile lambda_user out.txt
when we check the out.txt file:
$cat out.txt "hello from LuaJIT - 1 2 3 4 5 6 7 8 9 10 \n"