Key Points

1. AWS has both high level and low-level frameworks for ML
2. AWS processors can target ML workloads specifically
3.  
4. AWS services can come at 3 layers: iaas, paas, saas
5. Most references to AWS refer to paas - EC2 instances and the services that run on them
6. EC2 instances can be Windows or Linux
7. A low-cost iaas option is LightSail - a VPS server that is fixed price, low-cost with internet connectivity etc - easier to plan than EC2 costs
8. Effective cost management of AWS resources and services is a HUGE issue for most companies
9. EBS - elastic block storage - can be mounted and file system created for use. can be attached to any single instance

References

https://console.aws.amazon.com/console/home?region=us-east-1#

pSyn$pjm9yz3

Training Resources

Ed Izzo AWS Tips 

Great catching up as always, Jim. Maybe we'll finally meet in person post-vaccination this summer!

Certification Overview

https://aws.amazon.com/certification/

AWS Cloud Practitioner

https://aws.amazon.com/certification/certified-cloud-practitioner/

https://d1.awsstatic.com/training-and-certification/docs-cloud-practitioner/AWS_Certified_Cloud_Practitioner_Exam_Guide_v2.1.pdf

The AWS Certified Cloud Practitioner examination is intended for individuals who have the knowledge and skills necessary to effectively demonstrate an overall understanding of the AWS Cloud, independent of specific technical roles addressed by other AWS Certifications. The exam can be taken at a testing center or from the comfort and convenience of a home or office location as an online proctored exam.

Abilities Validated by the Certification

Recommended Knowledge and Experience

Prepare for Your Exam

Topics covered in AWS Cloud Practitioner 

1. Understand and use core services of Amazon Web Services (AWS)
2. Understand and use Identity & Access Management (IAM)
3. Understand and use Virtual Private Cloud (VPC)
4. Understand and use Simple Storage Service (S3)
5. Understand and use Elastic Cloud Compute (EC2)
6. Understand and use RDS/DynamoDB (databases)
7. Understand and use Simple Notification Service (SNS)
8. Understand and use CloudWatch (monitoring)
9. Understand and use Elastic Load Balancing (distributing traffic)
10. Understand and use Auto Scaling (scalable & elastic architecture)
11. Understand and use Route 53 (domains & DNS)

AWS Associate Architect cert - online multiple choice

https://aws.amazon.com/certification/certified-solutions-architect-associate/

preparation for exam

Abilities Validated by the Certification

Recommended Knowledge and Experience

AWS Exam Preparations Guides

https://aws.amazon.com/certification/certification-prep/

Solutions Architect Associate prep

https://aws.amazon.com/certification/certified-solutions-architect-associate/

Solutions Developer Associate prep

Key Concepts

AWS resources

AWS EC Cloud Documentation

https://docs.aws.amazon.com/ec2/index.html

AWS EC2 Linux Docs

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/index.html

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html

Compute Nodes

https://aws.amazon.com/ec2/pricing/?p=ps

Amazon EC2 is free to try. There are four ways to pay for Amazon EC2 instances: On-Demand, Reserved Instances, and Spot Instances. You can also pay for Dedicated Hosts which provide you with EC2 instance capacity on physical servers dedicated for your use.

Free Tier for 1 year

AWS Free Tier includes 750 hours of Linux and Windows t2.micro instances each month for one year. To stay within the Free Tier, use only EC2 Micro instances.

AMI - Amazon Machine Instance configurations

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html

An Amazon Machine Image (AMI) provides the information required to launch an instance. You must specify an AMI when you launch an instance. You can launch multiple instances from a single AMI when you need multiple instances with the same configuration. You can use different AMIs to launch instances when you need instances with different configurations.

An AMI includes the following:

EBS - Elastic Block Storage devices

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html

Amazon Elastic Block Store (Amazon EBS) provides block level storage volumes for use with EC2 instances. EBS volumes behave like raw, unformatted block devices. You can mount these volumes as devices on your instances. You can mount multiple volumes on the same instance, but each volume can be attached to only one instance at a time. You can create a file system on top of these volumes, or use them in any way you would use a block device (like a hard drive). You can dynamically change the configuration of a volume attached to an instance.

EBS volumes are highly available and reliable storage volumes that can be attached to any running instance that is in the same Availability Zone. EBS volumes that are attached to an EC2 instance are exposed as storage volumes that persist independently from the life of the instance. With Amazon EBS, you pay only for what you use. For more information about Amazon EBS pricing, see the Projecting Costs section of the Amazon Elastic Block Store page.

AWS Cost Management Strategies

hire someone who has managed a "limited data" phone plan successfully - same concepts apply here .. needs agressive cost management ( 3rd party app opportunity with alerts / automated responses ??? )

https://go.cloudhealthtech.com/rs/933-ZUR-080/images/10-best-practices-reducing-aws-spend.pdf

https://drive.google.com/open?id=1sHsXhkf83v3vK-3uHhEvl-g8ywuvrf8t

AWS Reserved Instances - RI

The most mature AWS customers are running more than 80% of their EC2 infrastructure covered by RI purchases. A best practice is to not let this number dip below 60% for maximum efficiency.

AWS Cost management can be improved for most companies

https://www.zdnet.com/article/cloud-computing-sticker-shock-is-on-the-rise-and-containing-it-may-be-a-new-career-path/

The most oft-used tools used for managing cloud costs include AWS Cost Explorer, Cloudability (Apptio), CloudHealth (VMWare), Azure Cost Management, GCP Cost Tools, and Cloudcheckr. About half, 46%, use cloud native tooling as their primary technology, 43% use a 3rd party platform, and 11% use home grown tools or spreadsheets. At the same time, many FinOps practitioners still rely on data collection, collation, and analysis via spreadsheet. Almost all practitioners use a combination of tooling, while still relying on spreadsheets for some tasks -- with forecasting being the biggest Excel use.

The survey's authors project that significant growth is ahead for FinOps, the field of cloud financial management,

Some challenges cited by respondents in the survey include the following:

• Measuring real-time cost analysis between the various cloud providers.
• Providing foundational education for engineering teams.
• Having visualization of cloud architecture, versus cost of those parts of the infrastructure.
• Facilitating robust tag enforcement, with easy ability to export cost and presentation to teams.
• Enabling reporting total spend, cloud spend, service management spend and contract spend, the entire cloud spend picture.
• Achieving container control, network and hardware integration at the scale and speed of cloud."
• Highlighting and fostering conversations around cloud cost management from a behavior change perspective

AWS services

AWS service levels - IAAS, PAAS, FAAS

Amazon Web Services (AWS) provides the infrastructure to host and run your applications at different levels of abstraction.The most obvious solution for hosting code is of course renting a bare metal machine, setting up the operating system andinstalling all necessary software. You can do that with Amazon EC2, although this service is more known for renting virtualmachines that run on physical hosts provided by Amazon. In both solutions you must care about the operating system you areusing and the installation of the server software (e.g. Tomcat).

Beanstalk - PAAS for Web apps

Amazon Beanstalk goes one step further by providing services that take a ready-to-use war file and deploy it on a virtual machinethat was setup by Amazon on behalf of you. As a developer you no longer have to care about the details of setting up the operatingsystem or the server software. If you implement your application such that it is independent of the OS and runs on the providedversion of Apache Tomcat, Amazon can automatically deploy your software and even scale out the number of virtual machinesnecessary to serve all clients without much delay. You just tell Amazon the limits it should operate in and you can concentrateon the implementation.

Serverless function concepts

Containers like Docker provide significant environment isolation and flexibility.

An app in a Docker container only talks to the Docker engine and the configured ports.

It has no idea of the environment or OS it runs in.

Deploying microservices in containers provides major benefits for most use cases:

• locality of reference on data, libraries within a microservice to a high degree when caching is used
• environment agnostic
• easy to scale as a unit independent of other services in other containers

faas - single function deployed as a serverless service

the server is conceptually "invisible" to the developer

sounds simple until you deal with the

serverless is a work in progress in 2019

The most popular serverless platforms--AWS Lambda, Google Cloud Functions, Azure Functions--all present challenges once data gets involved. Want to talk to local AWS services? Dead simple. But once authenticated APIs get involved, it’s more of a pain. Where do you store tokens? How do you handle OAuth redirects? How do you manage users? Quickly that narrow use of serverless can snowball into a pile of other public cloud services … to the point that you’ve swapped the complexity developers know for some new piles of stuff to learn.

AWS Lamba Concepts

https://drive.google.com/open?id=153fxjcVnuov2wtJqoLfeSP55iY5rKQbZ

AWS Lambda is an event-driven, serverless computing platform provided by Amazon as a part of the Amazon Web Services. Itis a computing service that runs code in response to events and automatically manages the computing resources required by that code.

The purpose of Lambda, as compared to AWS EC2, is to simplify building smaller, on-demand applications that are responsive toevents and new information. AWS targets starting a Lambda instance within milliseconds of an event. Node.js, Python, Java, Goand C# through .NET Core are all officially supported.

AWS Lambda was designed for use cases such as image or objects uploads to Amazon S3, updates to DynamoDB tables,responding to website clicks or reacting to sensor readings from an IoT connected device. AWS Lambda can also be used toautomatically provision back-end services triggered by custom HTTP requests, and "spin down" such services when not in use,to save resources

With Amazon Lambda the level of abstraction is put even one step higher. With Lambda you no longer care about virtualmachines and their scale out yourself. All you have to do is to provide Amazon some code (typically in form of a prepared jarfile) and Amazon cares about its execution on behalf of you. In contrast to Amazon Beanstalk you do not care about the numberof virtual machines used for the execution nor the number of load balancers. Amazon promises to execute your code as often as itis requested. And you pay only for the time used for its execution, not for the time your virtual machines are running.

Stateless function programs

Receive all data as parms, return a value.

Good for lower-level, reactive calculations driven by events from an event source

AWS Serverless Java Lambda Tutorial

lambda-java-programming-aws-lambda-cockroach-labs

AWS Serverless Java services - okta

https://developer.okta.com/blog/2020/05/27/serverless-java-aws

AWS-java-serverless-developer.okta.com-Serverless Java with Amazon Web Services.pdf

• Choose Between Java Serverless Options
• Sign Up for AWS Account with Billing
• Create AWS Access Keys
• Install and Configure AWS CLI
• Create AWS Role
• Download the Project from GitHub
• Configure Okta JWT Auth
• Create the Lambda
• Create an AWS API Gateway
• Test Your API Gateway URL
• Generate a JWT Token
• Test the Protected Serverless Function
• Learn More about AWS and Java

Apache OpenWhisk

Cloud Serverless Cost Calculator - AWS, Azure, IBM, GCP 

http://serverlesscalc.com/

AWS Events - CloudWatch and SNS

AWS Events - CloudWatch and SNS down

https://cloudonaut.io/loosing-trust-in-aws-sns-broken-for-24-days/

https://drive.google.com/file/d/1UKJ4Lo5ZeKDDe9QNTFjvDjPTQqA6Gfq2/view?usp=sharing

AWS Concepts and Terms

AWS SSM - Systems Manager

https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html

AWS Systems Manager Agent (SSM Agent) is Amazon software that can be installed and configured on an Amazon EC2 instance, an on-premises server, or a virtual machine (VM). SSM Agent makes it possible for Systems Manager to update, manage, and configure these resources. The agent processes requests from the Systems Manager service in the AWS Cloud, and then runs them as specified in the request. SSM Agent then sends status and execution information back to the Systems Manager service by using the Amazon Message Delivery Service (service prefix: ec2messages).

SSH connection to EC2 instance at Paramount

----------------------------------------------------
devops access for dev env
----------------------------------------------------


nsf-api.sysopsnetwork.com

--------------
1> create a client key pair w puttygen 

use puttygen to generate key pair 
save private key w pass phrase .. jm+
save public key as pem file ..
send public key to server to load


--------------
2> connect on openvpn

jimstOVOTAtivlp$aKo8wesjim
use openvpn connection
jim
w pwd


--------------
3> open terminal w putty ssh

run putty ssh
load private key 
create conenction
	nsf-api
	url 	nsf-api.sysopsnetwork.com


--------------
4> open robo3t connection

robo3t connection
	nsf-api 
	url nsf-api.sysopsnetwork.com  27017
	uses ssh prv key - no SSL ( its vpn )
		with openvpn no need for ssh tunnel

nsf-api.sysopsnetwork.com

--------------
5> postman client


import postman lib export and run in postman for nsf-api lib

postman api 

import postman nsf test collection
https://documenter.getpostman.com/view/5352743/SzS5v6ok?version=latest

then run postman 

initial test fails
GET /accounts HTTP/1.1
Host: localhost:3030
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6ImFjY2VzcyJ9.eyJpYXQiOjE1ODM3OTcxMTIsImV4cCI6MTU4Mzg4MzUxMiwiYXVkIjoiaHR0cHM6Ly95b3VyZG9tYWluLmNvbSIsImlzcyI6ImZlYXRoZXJzIiwic3ViIjoiNWU2MmMxYTBjYzE5YzAzMGE2YjFjNGNiIiwianRpIjoiNzcxMTg5ZjMtZWM5YS00OThjLTliOWUtN2I2ZjE2MmM0OTM3In0.XrmEn2Sff6Z3U9q7XdleoU0-2YZHKhrcPxUuipb_jV0
cache-control: no-cache
Postman-Token: df9ced11-c722-4843-83aa-d9dfe2e6d730


https://blog.postman.com/2018/08/09/postman-release-6-2/

Potential Value Opportunities

AWS mainframe integration opportunities  ( TCS and IBM i as well )

https://cloudwars.co/amazon/amazon-addresses-critical-blind-spot-mainframe-deal-tcs/

This hybrid cloud phenomenon—the ability for private-cloud and public-cloud environments to work seamlessly with on-premises systems including mainframes—has become the Holy Grail for not only the businesses deploying it but also for cloud vendors looking to win the hearts, minds and wallets of those businesses.

Over the past few years, AWS has made some impressive strides in addressing the hybrid-cloud reality, particularly as it has continued to expand its offerings under its Outposts family of services aimed at tying into on-premises environments.

But, as I’ve noted before, the very fact that AWS chose the term “Outposts” reveals a lot about Amazon’s perspective: that these are marginal and minimal concessions in the form of stuff to be kept out on the fringes and visited or used only when necessary.

That is perhaps an understandable perspective from a cloud-computing zealot, but it is most certainly not the point of view that businesses around the world are taking as they pursue the hybrid blend that’s optimal for their success.

And into that breach comes the enhanced partnership between AWS and TCS.

While TCS has been a close partner of AWS for many years, the new relationship calls for all of the AWS-focused efforts of TCS to be brought together for the first time under a single entity. That is no small achievement in a huge company like TCS (Tata Consultancy Services), which posted revenue of $22 billion for its last full fiscal year and has 469,000 employees in 46 countries.

TCS said its new business unit will be focused on helping businesses migrate mainframe and other legacy environments to AWS, modernize their applications and data estates and pursue industry-specific innovations.

In an email exchange, I received some additional insights into the partnership from Krishna Mohan, VP and global head of the TCS AWS Business Unit.

“Mainframe modernization mainly includes movement of entire mainframe workloads and associated ecosystems into AWS and broadly aligned to cloud-adoption strategy,” Mohan said via email.

“Once modernized or rehosted, the new cloud-native applications will integrate with various SaaS/PaaS solutions available and the on-premises workloads that fit into the hybrid-cloud model.”

In reply to my questions about the possible size of this mainframe-migration opportunity and related issues, Mohan offered the following insights:

1. Mainframes are still used by: 

• 71% of the Fortune 500;
• 96 of the world’s largest 100 banks;
• 9 of the world’s 10 largest insurance companies; and 
• 23 of the 25 largest retailers in the US.  

2. 70% of CXOs say mainframe migration and modernization is a top priority in the next 3 years, according to a survey of more than 200 CXOs conducted by IDG Research on behalf of TCS/AWS.

3. Mainframe applications are still in wide use, according to the IDG survey, including:

• 69% for core business;
• 67% for internal apps;
• 67% for finance apps;
• 61% customer-facing apps; and
• 60% for AI/ML.

Potential Challenges

Candidate Solutions

AWS free tier services

https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc

AWS cost planning - Planning Dashboard for systems, services

https://aws.amazon.com/pricing/cost-optimization/

AWS cost calculator

https://calculator.s3.amazonaws.com/index.html

AWS Lightsail Cloud Server specs

https://aws.amazon.com/lightsail/pricing/?opdp1=pricing

AWS Support 

Case ID 7197414421 - can I use AWS services from Lightsail?

Lightsail provides a base set of resources available 24x7 in different configurations for compute, storage, memory, network access. From a Lightsail account, console do I have access to normal AWS services as an option?

Step-by-step guide for Example

AWS Management Console

https://console.aws.amazon.com/?nc2=h_m_mc

billing status for free tier 

https://console.aws.amazon.com/billing/home?#/

aws-billing-free-tier-status-check-Billing Management Console.pdf

cost query reports 

https://console.aws.amazon.com/cost-management/home#/dashboard

IAM

https://console.aws.amazon.com/iam/home?region=us-east-1#/home

user sign in link 

https://594585551661.signin.aws.amazon.com/console

IAM best practices

https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html

https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#lock-away-credentials

IAM docs 

create an admin user acct 

If you don't already have an access key for your AWS account root user, don't create one unless you absolutely need to. Instead, use your account email address and password to sign in to the AWS Management Console and create an IAM user for yourself that has administrative permissions.

rjm0g$psyn#pz9

create 2 admins - jmason, smason 

user key files downloaded

swt1-aws-new_user_credentials-v1.csv

Created user jmason
Attached policy AdministratorAccess to user jmason
Attached policy AmazonAPIGatewayAdministrator to user jmason
Created access key for user jmason
Created login profile for user jmason

welcome email to jm9g 

You now have access to the AWS Management Console for the account ending in 1661. ------

Sign-in URL: https://594585551661.signin.aws.amazon.com/console
User name: jmason

Your password will be provided separately by your AWS account administrator.

welcome email smason 

You now have access to the AWS Management Console for the account ending in 1661. ------

Sign-in URL: https://594585551661.signin.aws.amazon.com/console
User name: smason

Your password will be provided separately by your AWS account administrator.

Created an org - swt1

sample code block

Recommended Next Steps

Related articles