Python与云 ——AWS的Python原生应用浅析 张孝峰 亚马逊AWS资深解决方案架构师 Python 30周年 Python发展时间线 2019/10 v3.8 v2.7.17 开始实现 1989/12 v0.9.0 1991/2 v1.0.0 1994/1 v2.0 2000/10 v2.5 2006/9 v2.6 2008/10 v3.0 2008/12 2008/12 v2.7 2010/7 v3.7 2018/6 AWS发展时间线 2019/10 开始实现 1989/12 v0.9.0 1991/2 v1.0.0 1994/1 v2.0 2000/10 v2.5 2006/9 v2.6 2008/10 v3.0 2008/12 v2.7 2010/7 v3.7 2018/6 贝索斯开始思考 微服务 2000 2000 Amazon S3 2006/3 Amazon SQS 2006/7 Amazon EC2 2006/8 开始研发 2004 AWS Lambda 2014/11 2019/10 22个区域 165项服务 Python被称为“瑞士军刀” • Web开发 • AI / 机器学习 • 自动化运维 • 网络爬虫 • 大数据分析 48 82 160 280 516

in Taipei • Focus on Serverless, Container and AWS CDK • Father of a daughter • Speaker in AWS Global Summits, KubeCon and CNCF Webinar • Active in AWS User Groups from Shanghai, Taipei, Korea and other AWS service All publicly accessible endpoints AWS Lambda functions Endpoints in your VPC Regional API Endpoints AWS Lambda functions Endpoints on Amazon EC2 Your VPC AWS Manage Wait X Seconds Get Job Status Job Complete? Set Job Failed Set Job Succeeded Sent Message to SNS AWS Lambda functions Synchronously Run a Batch Job Publish Success to SNS Publish Error to SNS Start

“无服务”是一种新的模型，它跨越许多不同类别的服务 AWS Lambda AWS Fargate Amazon API Gateway Amazon SNS Amazon SQS AWS Step Functions 计 算 数 据 存 储 集 成 Amazon Aurora Serverless Amazon S3 Amazon DynamoDB AWS AppSync 2014 2016.3 2017.12 AWS Lambda Google Cloud Functions Azure Functions Apache OpenWisk Oracle Fn 2017.10 Project Riff “无服务器”计算平台的发展 2017.2 OpenFAAS “Serverless” 好在哪里？ 来源：https://aws.amazon.com/cn/ com/cn/solutions/case-studies/autodesk-serverless/ f() f() f() f() 复杂的并行任务 工作流编排 AWS Lambda & Python AWS Lambda 的事件驱动架构 其它服务 事件源 Serverless函数 常见的应用场景 Web 应用 • 静态网站 • 复杂的Web 应用 • 打包Flask 与 Express应用

number of ready icons in various IT areas such as architecture, cloud services, logos etc. It including AWS, Azure, Kubernetes, C4, product Logos and many others. To explore these libraries: • Browse the Github 387 27.3 AWS library [aws] 27 STANDARD LIBRARY 27.3 AWS library [aws] • https://github.com/plantuml/plantuml-stdlib/tree/master/aws • https://github.com/milo-minderbinder/AWS-PlantUML The AWS library library consists of Amazon AWS icons, it provides icons of two different sizes (normal and large). Use it by including the file that contains the sprite, eg: !include <aws/Storage/AmazonS3/AmazonS3>. When imported

number of ready icons in various IT areas such as architecture, cloud services, logos etc. It including AWS, Azure, Kubernetes, C4, product Logos and many others. To explore these libraries: • Browse the Github 382 26.3 AWS library [aws] 26 STANDARD LIBRARY 26.3 AWS library [aws] • https://github.com/plantuml/plantuml-stdlib/tree/master/aws • https://github.com/milo-minderbinder/AWS-PlantUML The AWS library library consists of Amazon AWS icons, it provides icons of two different sizes (normal and large). Use it by including the file that contains the sprite, eg: !include <aws/Storage/AmazonS3/AmazonS3>. When imported

number of ready icons in various IT areas such as architecture, cloud services, logos etc. It including AWS, Azure, Kubernetes, C4, product Logos and many others. To explore these libraries: • Browse the Github 405 27.3 AWS library [aws] 27 STANDARD LIBRARY 27.3 AWS library [aws] • https://github.com/plantuml/plantuml-stdlib/tree/master/aws • https://github.com/milo-minderbinder/AWS-PlantUML The AWS library library consists of Amazon AWS icons, it provides icons of two different sizes (normal and large). Use it by including the file that contains the sprite, eg: !include <aws/Storage/AmazonS3/AmazonS3>. When imported

URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore >= 1.4.87 The AWS credentials can be passed as user/password in the the URI, or they can be passed through the following settings: • AWS_ACCESS_KEY_ID • AWS_SECRET_ACCESS_KEY • AWS_SESSION_TOKEN (only needed for temporary security credentials) You can also define a a custom ACL and custom endpoint for exported feeds using this setting: • FEED_STORAGE_S3_ACL • AWS_ENDPOINT_URL This storage backend uses delayed file delivery. Google Cloud Storage (GCS) New in

URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: boto3 >= 1.20.0 The AWS credentials can be passed as user/password in the the URI, or they can be passed through the following settings: • AWS_ACCESS_KEY_ID • AWS_SECRET_ACCESS_KEY • AWS_SESSION_TOKEN (only needed for temporary security credentials) You can also define a custom endpoint, and region name for exported feeds using these settings: • FEED_STORAGE_S3_ACL • AWS_ENDPOINT_URL • AWS_REGION_NAME The default value for the overwrite key in the FEEDS for this storage backend

stored on Amazon S3. • URI scheme: s3 • Example URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv 3.8. Feed exports 85 Scrapy Documentation, Release 1.7.4 3) or boto (Python 2 only) The AWS credentials can be passed as user/password in the URI, or they can be passed through the following settings: • AWS_ACCESS_KEY_ID • AWS_SECRET_ACCESS_KEY You can also that the component must be enabled in order for the setting to have any effect. AWS_ACCESS_KEY_ID Default: None The AWS access key used by code that requires access to Amazon Web services, such as the

stored on Amazon S3. • URI scheme: s3 • Example URIs: – s3://mybucket/path/to/export.csv – s3://aws_key:aws_secret@mybucket/path/to/export.csv • Required external libraries: botocore (Python 2 and Python Documentation, Release 1.8.4 The AWS credentials can be passed as user/password in the URI, or they can be passed through the following settings: • AWS_ACCESS_KEY_ID • AWS_SECRET_ACCESS_KEY You can also effect. AWS_ACCESS_KEY_ID Default: None The AWS access key used by code that requires access to Amazon Web services, such as the S3 feed storage backend. AWS_SECRET_ACCESS_KEY Default: None The AWS secret