# Monitoring and Optimization

## Databases Refresher

If you are following along last blog covered S3 , EBS , Instance store ect but we didnt discused DB in depth so we will do that and then move to our main topic. We can do two types of things for DBs , first is the unmanaged option where we setup the Database ourself on our EC2 and manage all the updating , patching of the DB. Second type is we use the managed type Amazon RDS (Relational Database System). Using this you can lift the responsibilities from your end and just take care of your data instead of focusing on the operational work.

### High Availiability

As we already discussed it is a best practice to atleast have resources redundantly in two Availiability zones. But Amzon RDS belongs to one subnet in one AZ. So this is against our practice. But nothing to worry about Amazon RDS can be simply configured to launched a secondary DB instance in a seperate AZ using RDS Multi AZ deployment. And without worrying it will handle all the data replication and keep our data in sync. The cool thing is it handles fail state also so if our primary instance is down our secondary instance gets promoted. All the fail over happens behind the scene which is handled by RDS.

![](https://cdn.hashnode.com/uploads/covers/666a86cd4483c6023295c40c/b4b4428a-d620-4317-b649-167b5989a393.png align="center")

### Choosing the right Database

- If you dont have complex relationships we can go wih Amazon Dynamo DB which is a NoSQL DB service provided by AWS. It stores data in key value pairs so it is best for a usecase where you are making simple lookup tables no complex relationships. Also it charges based on the usage of the table and the amount of the data you are reading from the table instead of charging per hour.

![](https://cdn.hashnode.com/uploads/covers/666a86cd4483c6023295c40c/1d4ed7c3-f080-4d28-b63a-7594a9ca76ec.png align="center")

You can search about these DBs also and know their specific usecase

### Dynamo DB

It is a serverless NoSQL DB , it is a purpose built DB which means it wont fit every usecase. It manages the storgage so you dont have to worry about scaling up or scaling down and you only pay for what you use. It is highly performant and have mili second response time. All of our data is stored on SSDs and is stored redundantly in multiple servers in different AZs. It is best for data which needs flexible schema. In Dynamo DB we have items in a table. Not each item needs to follow the same schema. So the main attributes of Dynamo DB are Table , Items and attributes. It uses primary key to uniquely identify Items.

Below is the table attached for which Managed DB service you should use for your specific usecase.

## Comparison Table

![](https://cdn.hashnode.com/uploads/covers/666a86cd4483c6023295c40c/78992884-516d-436b-a8c2-a8467151c173.png align="center")

## Monitoring

It is really important thing to do as in cloud environments when your application is under real user load your application hidden edge cases can appear your users can complaint but they cant tell the inner workings of your application. So to debug where the issue is coming from we need some analytics and data. So we know if it is a EC2 problem or DB problem. So for this we need to have our monitoring in place. You also need various type of metrics when your app scale up or scale down.

The data we collect is know as metric and metrics collected over time become statistics. Some example metrics ae CPU Usage , Network in and Network out ect. We can configure alerts when some metric goes out of bound so we can take appropriate measures.

AWS Cloudwatch provides us a all in one dashboard for all our metrics , we can get metrics of our resources as well get our custom metrics.

AWS Cloudwatch provides us Alarms , which trigger when our specific metric reach a specific threshhold set by us for a specific amount of time.

There are three states of an alarm

| State | Meaning |
| --- | --- |
| `OK` | The configured alarm condition is not met |
| `ALARM` | The configured alarm condition is met |
| `INSUFFICIENT_DATA` | There is not enough usable information to determine the state |

You are notified using SNS (Simple Network Service) . The alams can also trigger Auto Scalling to scale our resources when needed. More on that below.

## Scalability

In our infratructure S3 and RDS are scalable by design. But our EC2 single instance can handle limited load. We have EC2 auto scaling , under traffic we need to scale up , one way is manually launch EC2 instances when needed and stop them when not but you will probably not do that. Auto Scaling manages that for you.

I will explain the high level flow for you

So lets say your EC2 is under load , then that EC2 will log its stat to cloud watch.

This will move the Cloud Watch into an Alarm state , it will then trigger the Auto Scale to provision new instances. EC2 auto scale needs to know what kind of EC2 instance to launch so for that we will use Launch Template.\\

There will be a user data section where you will write the script which repo to clone which deps to install ect all the stuff. Now critical thinkers will think what if my repo is private (really cool question ) We will follow the following steps

- Create a GitHub deploy key (read-only).
- Store the private key in AWS Secrets Manager.
- Attach an IAM role to the EC2 instance with `secretsmanager:GetSecretValue` permission.

Once launch template finalized create an auto scalling group using the launch template we just created. Select VPC and Subnets , attach to the load balancer and add the scaling policy in my case I added the CPU utilization.

So for revision I have written the steps in order how to setup auto scaing

1. Ensure your EC2 has no data stored , for persistent data we will use S3 and RDS.
2. Create a Launch Template
3. Create a Target Group\
   EC2 → Target Groups → Create target group. Configure your application's port (e.g., 3000) and health-check endpoint (e.g., `/health`).
4. Create an Application Load Balancer (ALB)\
   Select at least two Availability Zones, configure HTTP/HTTPS listeners, and forward traffic to your target group.
5. Create an Auto Scaling Group (ASG)\
   Select your Launch Template, choose subnets across at least two Availability Zones, attach the target group, and configure capacity.
6. Configure Auto Scaling policies\
   Example: Target tracking policy that maintains average EC2 CPU utilization around 60%. AWS automatically adds or removes instances according to demand.

So our final architecture looks like following

![](https://cdn.hashnode.com/uploads/covers/666a86cd4483c6023295c40c/79cc23bb-77d6-493f-831c-e0bf522cc7b7.png align="center")