Hey Guys!!! How can I reduce the response time of my API Gateway which is currently returning a 504 error due to the Lambda function taking more than 30 seconds to execute? The Lambda function is taking an input event and executing a command-line tool on it. It then returns the output generated by the tool as a response to the user. Can I offload the command-line tool execution part of my application to other AWS services?

have you tried bumping the lambda functions memory up - see if you can get it to run really quickly, just as a test? that'll only help if there's a resource bottleneck

Even with the resource bump, it is slow.

I suppose the api gateway + lambda function combination is fairly unique for providing a 'one shot' generic task behind an http api without running a constant service or load balancer.

@Bishwa Have you checked lambda snapstart, though not sure your stack, and snapstart may be limited to stacks. Years ago I migrated to ECS fargate(serverless) , when I faced kind of same issues.

Sure. Will take a look at it and will let you know if helps my case or not.

lambda snapstart https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html fargate vs lamba https://www.clickittech.com/devops/fargate-vs-lambda/amp/

Looks like Snapstart is only available for java runtime 😞

It's a shame it's not possible to run an ECS task from api gateway, but probably wouldn't be very fast if it was

https://blog.developer.adobe.com/deploy-microservices-using-aws-ecs-fargate-and-api-gateway-1b5e71129338 smth like this is doable IIRC:

Sure. Will take a look at this too.

If you can eat the spend to run the load balancer

What's killing me right now it the 30 second timeout limit.

if you can get a docker container to run it quickly on your local machine, it should translate to the cloud note in fargate that the size of your container image is crucial to quick start times do some local testing and see if you can get it to be <100mb and run in a few seconds with docker 🙂 ?

Do you know where the time is being spent?

The Lambda startup time is not significant(in ms). The majority of the time spent is typically consumed by the CLI tool rather than the function itself.

The problem with the CLI is you can't profile it. I might recommend converting the CLI command into its corresponding sdk and scripting it out. That way you can profile to see what's taking the most time. Most probably certain API calls.

Yeah. API Gateway expects a response within 30 seconds.

I see

I think profiling the CLI tool sounds like a good starting point.

So I haven't investigated how to do this very much, but my gut would be you would replace the API gateway to lambda with something else that starts the lambda and responds with some kind of uniq ID. Then whatever called the API would use that ID against a different API that checks for status. It's definitely complicated and requires state somewhere. I have never used AWS Step Functions but that is probably what they might recommend: https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-api-gateway.html

Interesting approach. Will definitely take a look at this.

Thanks. Helps a lot.

Interesting, I've only ever setup proxy to a load balancer

It would be interesting if the proxy Lambda returns 202 initially, and then the main Lambda function processes and returns the actual response asynchronously at a later time.

You are hitting up against a limitation of API Gateway - there is an integration time of 29 seconds regardless of integration type: https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#api-gateway-execution-service-limits-table