https://platformengineering.org logo
Join Slack
Powered by
This message was deleted.
# general
s
This message was deleted.
j
have you tried bumping the lambda functions memory up - see if you can get it to run really quickly, just as a test? that'll only help if there's a resource bottleneck
b
Even with the resource bump, it is slow.
I'm looking to offload that specific command line execution to compute intensive service if possible.
j
I suppose the api gateway + lambda function combination is fairly unique for providing a 'one shot' generic task behind an http api without running a constant service or load balancer.
m
@Bishwa Have you checked lambda snapstart, though not sure your stack, and snapstart may be limited to stacks. Years ago I migrated to ECS fargate(serverless) , when I faced kind of same issues.
b
Sure. Will take a look at it and will let you know if helps my case or not.
m
b
Looks like Snapstart is only available for java runtime 😞
My runtime is node18.
j
It's a shame it's not possible to run an ECS task from api gateway, but probably wouldn't be very fast if it was
m
b
Sure. Will take a look at this too.
j
If you can eat the spend to run the load balancer
b
What's killing me right now it the 30 second timeout limit.
I don't know if ECS Fargate will be any better.
Worth a try I guess.
j
if you can get a docker container to run it quickly on your local machine, it should translate to the cloud note in fargate that the size of your container image is crucial to quick start times do some local testing and see if you can get it to be <100mb and run in a few seconds with docker 🙂 ?
a
Do you know where the time is being spent?
Is it the lambda startup time? The command itself (cli) running?
If it's too slow is it worth turning the API into an async process that is polled?
b
The Lambda startup time is not significant(in ms). The majority of the time spent is typically consumed by the CLI tool rather than the function itself.
a
The problem with the CLI is you can't profile it. I might recommend converting the CLI command into its corresponding sdk and scripting it out. That way you can profile to see what's taking the most time. Most probably certain API calls.
There might be an option to improve its performance there, though it depends heavily on the use case.
In general I do think more than 30 seconds response for a synchronous API call is a lot.
Is that max enforced by API Gateway?
b
Yeah. API Gateway expects a response within 30 seconds.
a
I see
b
I think profiling the CLI tool sounds like a good starting point.
Btw, Is it possible to turn the API Gateway route into an async process that is polled? (I'm new to serverless world)
a
So I haven't investigated how to do this very much, but my gut would be you would replace the API gateway to lambda with something else that starts the lambda and responds with some kind of uniq ID. Then whatever called the API would use that ID against a different API that checks for status. It's definitely complicated and requires state somewhere. I have never used AWS Step Functions but that is probably what they might recommend: https://docs.aws.amazon.com/step-functions/latest/dg/tutorial-api-gateway.html
b
Interesting approach. Will definitely take a look at this.
a
The first answer is similar to what I'm recommending as an option.
To get the function output from a Step Function, you have to add a second method in API Gateway which will call the Step Function with the DescribeExecution action. The API Gateway client will have to call this periodically (poll) until the returned status is no longer "RUNNING".
just an idea
b
Thanks. Helps a lot.
I came across something called "Lambda Proxy integrations", might also be worth considering. I'm not sure how helpful it will be, but it's worth looking into.
a
Interesting, I've only ever setup proxy to a load balancer
b
It would be interesting if the proxy Lambda returns 202 initially, and then the main Lambda function processes and returns the actual response asynchronously at a later time.
r
You are hitting up against a limitation of API Gateway - there is an integration time of 29 seconds regardless of integration type: https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#api-gateway-execution-service-limits-table
Assuming there isn’t a way to rework into a native non-CLI process to thread the work, using an ALB->Lambda integration is the best choice: https://docs.aws.amazon.com/lambda/latest/dg/services-alb.html
As rule, API gateway is always easier, less performant and less flexible than ALB. An ALB is less performant and flexible than an NLB.
19 Views
Open in Slack
PreviousNext
Powered by