A simple intro to AWS Lambda

by Pawel Dolega @pdolega

 

 

Expectations

Warmup

Serverless

(... ) manifestation of how the intelligent combination of hosted services and self-managing infrastructure can result in significant improvements in development time and operating cost.

 

Serverless Architecture on AWS by Peter Sbarski

(... ) manifestation of how the intelligent combination of hosted services and self-managing infrastructure can result in significant improvements in development time and operating cost.

 

Serverless Architecture on AWS by Peter Sbarski

Moving away from servers and infrastructure concerns, as well as allowing the developer to primarily focus on code, is the ultimate goal behind serverless.

 

Serverless Architecture on AWS by Peter Sbarski

Someone else's problem

C = Ce + Ca

Complexity (C)

Ce - essential complexity

Ca - accidental complexity

C = Ce + Ca

Complexity (C)

Ce - essential complexity

Ca - accidental complexity

Focus (more) on your

core business

Stateless

Event-driven

Compute code

Stateless

stability, performance, reliability, availability, temporal coupling

Problems

scalability, elasticity, memory consumption, concurrency

state sharing, unreliable networking, complexity

readability, testability, correlation, throughput, latency

time to release, congestion, noisy neighbour, security

auditability, velocity, technical debt, runtime dependencies

distribution, reusability, dependency hell, spatial coupling

graceful degradation, transactions, data loss, repeatability

reproducibility, callback hell, mutable state access

hardware failure, network partition​, synchronization

...

Stateless

It keeps state (up to a point)

...but you should not rely on it

...same is true for computing node in microservice architecture

Event-driven

Compute code

Pay per execution (100 ms)

One more thing

Cost = Cr + Cc

Cr - cost of fired requests

($0.20 / 1 mln reqs)

 

Cc - cost of computing, counted in GB-seconds 

(declared memory * total execution time)
($0.17 / 1 TB-second)

Free tier

1 mln requests

400,000 TB-seconds

Pricing example

384 MB / avg time 1 s

No of calls Total costs
10k $0
100k $0
1mln $0
10mln $57.64
100mln $638.46

38th on Alexa ranking (nov 2016):

45 mln of image uploads / month

60 bln image views / month

http://expandedramblings.com/index.php/imgur-statistics/

Why now

Google App Engine

Initial release: April 2008

Knowledge

bla bla microservices

bla bla

by Jonas Bonér

Technology

Containers

Services

Networking speed

What serverless is not

Function

as

a

Service

Landscape

AWS Lambda

Azure Functions

Google Cloud Functions

Bluemix OpenWhisk

Let's see how to build something

*

HelloLambda.scala


class HelloLambda extends RequestStreamHandler {
  override def handleRequest(inputStr: InputStream, 
                             output: OutputStream, 
                             context: Context): Unit = {
    context.getLogger.log(s"Remaining time: " +
                           "${context.getRemainingTimeInMillis}ms")

    val writer = new OutputStreamWriter(output, "UTF-8")

    val input = Source.fromInputStream(inputStr).mkString
    val result = write(s"Input: ${input} fetched with ID: " +
        "${context.getAwsRequestId} (at ${formattedNow})")

    writer.write(result)
    context.getLogger.log(result)

    writer.flush()
  }
}

UberJar to S3

plugins.sbt










    addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")

build.sbt


    ...
    scalaVersion := "2.12.2",
    retrieveManaged := true,
    libraryDependencies ++= Seq(
    
        // AWS api
        "com.amazonaws" % "aws-lambda-java-core" % "1.1.0",
        "com.amazonaws" % "aws-lambda-java-events" % "1.3.0",
        "org.json4s" %% "json4s-jackson" % "3.5.1",
        
        "org.scalatest" %% "scalatest" % "3.0.0" % Test,
        "com.amazonaws" % "aws-java-sdk-lambda" % "1.11.123" % Test
    )
    ...

    assemblyMergeStrategy in assembly := {
      case PathList("META-INF", xs @ _*) => MergeStrategy.discard
      case _ => MergeStrategy.first
    }
Generate and push jar

    
$ sbt bareHello/assembly  
[warn] Executing in batch mode.
[warn]   For better performance, hit [ENTER] to switch to interactive mode, or
[warn]   consider launching sbt without any commands, or explicitly passing 'shell'
[info] Loading global plugins from /home/pdolega/.sbt/0.13/plugins
[info] Loading project definition from /home/pdolega/projects/lambda/lambda-samples/project
...
[success] Total time: 6 s, completed Jun 17, 2017 12:54:37 AM


$ aws s3 mb s3://bare-lambda  
make_bucket: bare-lambda


$ aws s3 cp \
        ./target/scala-2.12/bare-hello-assembly-1.0-SNAPSHOT.jar \
        s3://bare-lambda/app.jar

Role & policy






    $ aws iam create-role --role-name bare-hello-role \
    > --assume-role-policy-document '{
    >       "Version": "2012-10-17",
    >       "Statement": [
    >         {
    >           "Sid": "",
    >           "Effect": "Allow",
    >           "Principal": {
    >             "Service": "lambda.amazonaws.com"
    >           },
    >           "Action": "sts:AssumeRole"
    >         }
    >       ]
    >     }'







    aws iam attach-role-policy \
        --role-name bare-hello-role \
        --policy-arn \
            arn:aws:iam::aws:policy/CloudWatchLogsFullAccess

I've cut corners here

Configure function





$ aws lambda create-function \
    --function-name bare-lambda \
    --runtime java8 \
    --role arn:aws:iam::123456789012:role/bare-hello-role \
    --timeout 10 \
    --memory 256 \
    --handler com.virtuslab.lambda.hello.HelloLambda::handleRequest \
    --code S3Bucket=bare-lambda,S3Key=app.jar






$ aws lambda invoke --function-name bare-lambda output.txt
{
    "StatusCode": 200
}

$ cat output.txt 
"Input: {} fetched with ID: \
    b696b53a-52f3-11e7-8daa-8d98cda88cfb \
    (at 17-06-2017 12:27:15)"

Client call

LambdaClientSpec.scala






trait HelloLambdaClient {
  @LambdaFunction(functionName = "bare-lambda")
  def hello(param: String): String

  @LambdaFunction(functionName = "bare-lambda", 
                  invocationType = Event)
  def helloAsync(param: String): String
}
LambdaClientSpec.scala



class LambdaClientSpec  extends WordSpec with MustMatchers {
  "Invoking lambda function" should {
    "work for sync mode" in {

      val helloLambda = LambdaInvokerFactory.builder
        .lambdaClient(AWSLambdaClientBuilder.defaultClient)
        .build(classOf[HelloLambdaClient])

      val returnValue = helloLambda.hello("sync call")
      println(s">> Retrieved result is: ${returnValue}")

      returnValue must not be(null)
      returnValue must startWith("Input: ")

      val asyncReturn = helloLambda.helloAsync("a-sync call")
      println(s">> Retrieved result is: ${asyncReturn}")
    }
  }
}




Testing started at 9:45 AM ...
>> Retrieved result is: Input: "sync call" \
    fetched with ID: f4d4ade1-57e7-11e7-baae-8db3097073aa (at 23-06-2017 07:45:44)
>> Retrieved result is: null

But hey, I want to have REST

HttpLambda.scala








case class Response(body: Option[String] = None,
                    statusCode: Int = 200,
                    headers: Map[String, Any] = Map.empty)
HttpLambda.scala

    ...
    ...
    override def handleRequest(input: InputStream, 
                               output: OutputStream, 
                               context: Context): Unit = {
      context.getLogger.log(Source.fromInputStream(input).mkString)
        
      Try {
        val response = write(
          Response(body = Some(s"Hello lambda via API gateway ..."))
        )
        context.getLogger.log(s"Generated response is: ${response}")
    
        val writer = new OutputStreamWriter(output, "UTF-8")
        writer.write(response)
        writer.flush()
      }.recover {
        case e: Throwable => context.getLogger.log(s"exception? -> ${e}")
      }
    }
# API resource
$ aws apigateway create-rest-api --name lambda-api
$ aws apigateway get-rest-apis
$ aws apigateway get-resources --rest-api-id 184i4c0pjk
$ aws apigateway create-resource --rest-api-id 184i4c0pjk \
        --parent-id tecp4zmc0k --path-part hello

# API method
$ aws apigateway put-method --rest-api-id 184i4c0pjk 
    \--resource-id iwku45 
    \--http-method ANY 
    \--authorization-type NONE

# method integration with Lambda
$ aws apigateway put-integration --rest-api-id 184i4c0pjk \
        --resource-id iwku45 --http-method ANY --type AWS_PROXY \
        --integration-http-method POST \
        --uri arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/\
              arn:aws:lambda:eu-west-1:123456789012:function:bare-lambda/invocations

# permissions
$ aws lambda add-permission --function-name http-lambda --statement-id "API-accesss" \
        --action lambda:InvokeFunction --principal apigateway.amazonaws.com \
        --source-arn arn:aws:execute-api:eu-west-1:123456789012:184i4c0pjk/*/*/hello

How to make your life easier

https://serverless.com/

Supports many langs but focused mainly on NodeJS

Supports AWS Lambda, Azure, Google & OpenWhisk !

serverless.yml


service: aws-java-simple-http-endpoint

frameworkVersion: ">=1.2.0 <2.0.0"

provider:
  name: aws
  runtime: java8
  
package:
  artifact: build/distributions/aws-java-simple-http-endpoint.zip

functions:
  currentTime:
    handler: com.serverless.Handler
    events:
      - http:
          path: ping
          method: get
RequestHandler.java

package com.serverless;

import java.util.Date;
import java.util.HashMap;
import java.util.Map;

import org.apache.log4j.Logger;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

public class Handler implements RequestHandler<Map<String, Object>, ApiGatewayResponse> {

	private static final Logger LOG = Logger.getLogger(Handler.class);

	@Override
	public ApiGatewayResponse handleRequest(Map<String, Object> input, Context context) {
		LOG.info("received: " + input);
		Response responseBody = new Response("Hello, the current time is " + new Date());
		Map<String, String> headers = new HashMap<>();
		headers.put("X-Powered-By", "AWS Lambda & Serverless");
		headers.put("Content-Type", "application/json");
		return ApiGatewayResponse.builder()
				.setStatusCode(200)
				.setObjectBody(responseBody)
				.setHeaders(headers)
				.build();
	}
}
$ serverless deploy

Serverless: Creating Stack...
Serverless: Checking Stack create progress...
.....
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading service .zip file to S3...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..............................
Serverless: Stack update finished...
Service Information
service: aws-java-simple-http-endpoint
stage: dev
region: us-east-1
api keys:
  None
endpoints:
  GET - https://XXXXXXX.execute-api.us-east-1.amazonaws.com/dev/ping
functions:
  aws-java-simple-http-endpoint-dev-currentTime: \
  arn:aws:lambda:us-east-1:XXXXXXX:function:aws-java-simple-http-endpoint-dev-currentTime

Challice (Python)

https://github.com/awslabs/chalice

$ pip install chalice
$ chalice new-project helloworld && cd helloworld
$ cat app.py

from chalice import Chalice

app = Chalice(app_name="helloworld")

@app.route("/")
def index():
    return {"hello": "world"}



$ chalice deploy
...
Your application is available at: https://endpoint/dev

$ curl https://endpoint/dev
{"hello": "world"}

Quaich

https://github.com/quaich-serverless

*

PlainLambda.scala

@DirectLambda
class PlainLambda extends DirectLambdaHandler {

  val randomValue = Random.nextLong()

  var value = 0

  override protected def handleEvent(json: JValue, output: OutputStream)
                                    (implicit ctx: LambdaContext) {
    Thread.sleep(1500)
    value = value + 1

    writeJson(output, s"Success returned from request: ${ctx.awsRequestId}. " +
      s"Random value is: ${randomValue} " +
      s"remaining ms: ${ctx.remainingTimeInMillis} ms " +
      s"VALUE IS: ${value}")
  }
}

sbt deployLambda

$ sbt plainSample/deployLambda
[info] Loading global plugins from /home/pdolega/.sbt/0.13/plugins                                                                                                                                                  
[info] Loading project definition from /home/pdolega/projects/.../project                                                                                                                   
[info] Set current project to quaich-other-samples (in build file:/home/pdolega/...
...
[info] Packaging /home/pdolega/projects/.../plain-sample-assembly-0.0.4-SNAPSHOT.jar ...
[info] Done packaging.
[info] Inferred lambda handlers are: \
    HandlerName(codes.bytes.quaich.samples.plain.PlainLambda::handleRequest)
[info] Role plain-sample was not found. It will be now created...
[info] Role plain-sample has been created \
    [ arn:aws:iam::123456789012:role/plain-sample ]...
[info] Created role policy [ plain-sample ]...
[info] Role policy successfully attached...
[info] Bucket codes.bytes.plain-sample doesn't exists, attempting to create it
[=========================================]   100%   (18.89/18.89 MB) Lambda JAR -> S3S3
[info] Creating new AWS Lambda function 'plain-sample'
[info] Created Lambda: arn:aws:lambda:eu-west-1:123456789012:function:plain-sample
[success] Total time: 261 s, completed Jun 19, 2017 12:21:00 PM

First run

$ sbt plainDemo/deployLambda
[info] Loading global plugins from /home/pdolega/.sbt/0.13/plugins                                                                                                                                                  
[info] Loading project definition from /home/pdolega/projects/.../project                                                                                                                   
[info] Set current project to quaich-other-samples (in build file:/home/pdolega/...
...
[info] Packaging /home/pdolega/projects/.../plain-demo-assembly-0.0.4-SNAPSHOT.jar ...
[info] Done packaging.
[info] Inferred lambda handlers are: \
    HandlerName(codes.bytes.quaich.samples.plain.PlainLambda::handleRequest)
[info] Role arn:aws:iam::123456789012:role/plain-demo has been found and will be used...
[info] Bucket codes.bytes.plain-demo exists and is accessible
[=========================================]   100%   (18.89/18.89 MB) Lambda JAR -> S3S3
[info] Updating existing AWS Lambda function 'plain-demo'
[info] Successfully updated function code: \
    arn:aws:lambda:eu-west-1:123456789012:function:plain-demo
[info] Successfully updated function configuration: \
    arn:aws:lambda:eu-west-1:123456789012:function:plain-demo
[info] Updated lambda arn:aws:lambda:eu-west-1:123456789012:function:plain-demo
[success] Total time: 231 s, completed Jun 19, 2017 11:10:19 AM

Next runs

HTTP API with Quaich

Simplistic Slack integration

build.sbt:

lazy val commonSettings = Seq(
  libraryDependencies ++= Seq(
    "codes.bytes" %% "quaich-http" % projectVersion
  ),
  ...
)

lazy val slackbotDemo = (project in file("slackbot-demo")).
  settings(commonSettings: _*).
  settings(quaichMacroSettings: _*).
  settings(
    name := "slackbot-demo",
    createAutomatically := true,

    awsLambdaMemory := 192,
    awsLambdaTimeout := 30,
    region := "eu-west-1",
  ).
  dependsOn(util).
  enablePlugins(AWSLambdaPlugin, AwsApiGatewayPlugin)
@LambdaHTTPApi
class SlackbotLambda {
  post[String]("/ping") { reqCtx =>
    val params = (for {
      body <- reqCtx.request.body.toSeq
      paramValue <- body.split("&")
      keyValue <- toKeyValue(paramValue.split("="))
    } yield {
      keyValue
    }).toMap

    val userHandlePart = params
        .get("user_name")
        .map(u => s"@${u}").getOrElse("")

    complete(s"Great! It works ${userHandlePart}")
  }

  get("/ping") { reqCtx =>
    complete("Great! Dummy ping works via GET")
  }
  ...
}

sbt deployHttpApi

$ sbt slackbotDemo/deployHttpApi
[info] Loading global plugins from /home/pdolega/.sbt/0.13/plugins
[info] Initiating API deployment...
[info] Given stack does not exist on AWS: slackbot-demo, it will be created...
[info] Progress: CREATE_IN_PROGRESS...
...
[info] Progress: CREATE_IN_PROGRESS...
[info] Progress: CREATE_IN_PROGRESS...
[info] Stack sync finished with status: CREATE_COMPLETE
[info] Detailed log of events during update of stack: slackbot-demo
[info]  AWS::CloudFormation::Stack: CREATE_IN_PROGRESS (User Initiated)...
[info]  AWS::ApiGateway::RestApi: CREATE_IN_PROGRESS...
[info]  AWS::ApiGateway::RestApi: CREATE_IN_PROGRESS (Resource creation Initiated)...
[info]  AWS::ApiGateway::RestApi: CREATE_COMPLETE...
[info]  AWS::ApiGateway::Resource: CREATE_IN_PROGRESS...
[info]  AWS::ApiGateway::Resource: CREATE_IN_PROGRESS (Resource creation Initiated)...
[info]  AWS::Lambda::Permission: CREATE_IN_PROGRESS...
[info]  AWS::ApiGateway::Resource: CREATE_COMPLETE...
[info]  AWS::Lambda::Permission: CREATE_IN_PROGRESS (Resource creation Initiated)...
[info]  AWS::ApiGateway::Method: CREATE_IN_PROGRESS...
[info]  AWS::ApiGateway::Method: CREATE_IN_PROGRESS (Resource creation Initiated)...
[info]  AWS::ApiGateway::Method: CREATE_COMPLETE...
[info]  AWS::ApiGateway::Deployment: CREATE_IN_PROGRESS...
[info]  AWS::ApiGateway::Deployment: CREATE_IN_PROGRESS (Resource creation Initiated)...
[info]  AWS::ApiGateway::Deployment: CREATE_COMPLETE...
[info]  AWS::Lambda::Permission: CREATE_COMPLETE...
[info]  AWS::CloudFormation::Stack: CREATE_COMPLETE...
[info] Stack arn:aws:cloudformation:eu-west-1:123456789012:stack/... successfully deployed...
[info] ========================================================================
[info] >>> Your API has been deployed at:
[info] >>> https://dogvhqiy64.execute-api.eu-west-1.amazonaws.com/dev/
[info] ========================================================================
[info]          
[success] Total time: 29 s, completed Jun 20, 2017 6:55:42 PM

SlackbotLambda demo

HTTP DSL

@LambdaHTTPApi
class DemoHTTPServer {
  get("/hello") { requestContext =>
    complete("Awesome. First small success! Version: 0.0.4")
  }

  get("/users/{username}/foo/{bar}") { requestContext =>
    complete("OK")
  }

  head("/users/{username}/foo/{bar}") { requestContext =>
    complete(s"Params are: ${requestContext.request.pathParameters}")
  }

  put[TestObject]("/users/{username}/foo/{bar}") { requestWithBody =>
    println(s"Put Body: ${requestWithBody.request.body} ...")
    val response = TestObject("OMG", "WTF")
    complete(response)
  }

  patch[TestObject]("/users/{username}/foo/{bar}") { requestContext =>
    println(s"Patch request: $requestContext")
    complete("OK")
  }
}

Future

DevOps ?

Hybrid solutions

Thank you

Contact

Software Engineer /

Entrepreneur

twitter: @pdolega

github: github.com/pdolega​