Richard Grantham
Desert image by Troy Squillaci

How to Fake Layers in your SAM Lambda for Local Test

Filed under AWS on

The Problem

I was writing a Lambda function with the AWS Serverless Application Model to do a maintenance task. I’d written my code, my unit tests and all was good. I had a problem, though. Invoking the Lambda for a local end-to-end test was failing. It couldn’t read some resource files despite the paths being correct.

I hadn’t had my coffee, so it took longer that I’d care to admit to twig that the Lambda was running in a container. It couldn’t actually access those files! Now I had a new problem to solve: How do I get my test data into the SAM environment so I could perform an end-to-end test?

The Options… or Lack Thereof

After a lot of fruitless searching around I looked closer into using Lambda layers to make the data available. Layers pull extra code and content into the Lambda runtime environment. Given this was the medium I had intended to use in production it made sense to use it for local testing too. There were a couple of obstacles this approach, which is why I had ignored it up to this point:

  1. I didn’t want to deploy my test data to a layer in the test environment as I wanted all tests to run in isolation.
  2. Layers are not available in the free version of Localstack I was using. Localstack shouldn’t have to be a dependency anyway.

I still needed to be able to run an end-to-end test so I scoured the SAM and layers documentation. I hoped for some magical configuration that would allow me to pre-copy the data. It’s not there, but after a while of staring at this and this I had a thought: Could I fake it?

Fake What? And How?

I was reading the “Working with layers” document the description of how SAM caches layers. I wondered if I could “pre-cache” my test data as a layer in the cache so SAM would’t try fetching it from anywhere. The cache directory name format is documented as LayerName-Version-<first 10 characters of sha256 of ARN>. I should be able to configure a layer and work out the cached directory name.

I gave my fake layer the ARN arn:aws:lambda:ap-southeast-2:000000000000:layer:testdata:1. According to the docs I would need to create a directory in the cache named testdata-1-29450a214e. By default SAM caches layers in ~/.aws-sam/layers-pkg. I didn’t want to go copying folders around if I could help it. Luckily the SAM CLI has the parameter -layer-cache-basedir, which allows you to tell it where to look for layer caches. Well, there is no directory like the present (working directory). Sorry.

I already had a directory named testdata so I renamed it to the expected cache directory. When I next ran sam local invoke I pointed it at my test template YAML file and added the parameter -layer-cache-basedir .. An image was created with the cached “layer” and my end-to-end test ran.

Step-by-Step

Let me summarise that in a step-by-step process:

  1. Come up with a ARN for the resource directory you want to add. The fact it’s not real is fine. It needs a region, layer name and a version number. The format would be arn:aws:lambda:${Region}:000000000000:layer:${LayerName}:${Version}.
  2. In template.yaml add a parameter named LayerArn for the layer. In the configuration section for the Lambda add a Layers section with an item referencing LayerArn. You can give LayerArn the default value of the ARN or pass it in as a parameter, described below.
  3. Calculate the SHA256 value of the ARN. I used this online SHA256 Hash Generator. Remember to check the “Lowercase hash(es)” box.
  4. Create the cache directory conforming to the format described above. You use the layer name, version number and first 10 characters of the hash. Put the resources you need your Lambda to access in there.
  5. Run sam local invoke using your testing template and with layer-cache-basedir set as one level above your cache directory. If that is the root of the project then it would be .. If you have not added the fake layer ARN as the default value of LayerArn you will need to pass it in here too. The parameter would be --parameter-overrides ParameterKey=LayerArn,ParameterValue=arn:aws:lambda:ap-southeast-2:000000000000:layer:testdata:1. Substitute that ARN for the one you came up with.

Wrapping Up

And that’s all there was to it. I could now show the Lambda running with local test data imported as a layer. You do not need Localstack to achieve this. I happen to use it for testing.

Did this help you out? Do you have a better solution? I’d love it if you dropped me a note to let me know.

Banner image by Troy Squillaci