addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcredit-cardcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobe--smallglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1launch-new-window--smalllight-bulblinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

New Newport Beach / Orange County AWS User Group Meetup Coming Soon

From: Jim C.
Sent on: Saturday, March 20, 2010 2:00 PM
Dear Users:

Thank you for your continued participation in this user group. I am working on an Agenda and a technical whitepaper for a new meet-up; so stay tuned. The focus will be on protecting EC2 instances from accidental or malicious destruction. Here is the problem. There is no concept of roles and permissions with EC2 instances. To gain access via the API or Management Console gives full privileges including destructive ones. But what if I just want to give my developers or System Admin, the rights to reboot only? Or to be able create instances but not terminate them?

At my current employment, I have developed a combination of process and procedure, physical security, and a modified version of Elastic Fox ? to accomplish the above. The process appears to meet PCI-DSS and other security standards, and I reference this process when completing our banking client IT audit requests. So stay tuned?.

In the meantime, here are two simple yet vexing problems I had along with their solutions. First, I wanted a simple and free way to monitor the health of my ELB instances: 24 x 7. I know that there are third party services out there that do that, but I did not want to share my authentication credentials (account id and security key) with anyone. So I put together the following DOS script and the free BLAT Win32 command line SMTP utility to send e-mail (and text message), to meet my needs.

Second, I wanted to automatically take snap shots of my volumes to back them up and I wanted to keep them for around 30 days, before automatically deleting them. Amazon only allows one to keep a maximum of 1000 snapshots. So if you don?t clean up old snapshots, you will quickly use up your allocation.

This tutorial assumes you have working knowledge of setting up ELB and Health checks and that both the API?s are already configured and set up. Second, these scripts do not have any error checking, etc. These were quick and dirty to get the job done. The reader is encouraged to modify and enhance as desired. Third, the reader should be somewhat familiar with DOS syntax. If the reader is not familiar, a simply web search will give you all the information you need to gain a better understanding of what the commands do. Fourth, the reader can reference BLAT help file to understand the syntax. Fifth, the reader should understand how to set up a recurring daily task using Windows Scheduler.

Here is what you need to solve both problems I described above. I have written in outline form for clarity, brevity and expedience.

1. The ELB and EC2 API?s should be loaded on a server or computer of your choice.
a. For example, I have installed:
i. C:\a_dat\AmazonAPI\ElasticLoadBalancing[masked]
ii. C:\a_dat\AmazonAPI\ec2-api-tools[masked]
iii. Both obtained from:
2. Download and install BLAT
3. Solution for problem 1, free way to monitor the health of my ELB instance.
a. For maximum flexibility, I created a parameter based DOS Batch file. Here is the script and process for monitoring the health of your ELB instances:

Here is the calling command. I have this running as a Windows Scheduled task. It runs every three minutes.
K:\Tools\AmazonAPI\InstanceHealth.bat OutOfService 0 "elb-describe-instance-health www"

?OutofService ?is parameter 1. It is a text string that I search for (returned by command shown as parameter #3)

?0? is parameter 2. It is a numeric and is the count of the number of times the parameter 1 string appears in the output of parameter #3.

"elb-describe-instance-health www" is parameter 3 and contains the ELB command. My LB is named www. You would substitute this value with the name of your LB.

Rem Script Name: K:\Tools\AmazonAPI\InstanceHealth.bat
Rem Assign parameters passed in to the bat file.
Set state=%1
Set count=%2
Set command=%3
Rem Set up the API library
Call setupVariables.bat
Rem Run the script using the parameters passed in from the calling process
Rem such as a from a windows scheduled task.
cmd /c %command% > tmp%state%.txt
Rem Alert if any LB instances are out of service
FIND /C /I "%state%" tmp%state%.txt > tmp%state%.out
FOR /F "skip=1 tokens=2 delims=:" %%i in (tmp%state%.out) do If %%i gtr %count% k:\tools\blat262\full\blat.exe - -attachi K:\Tools\AmazonAPI\tmp%state%.txt -body "Instance Report: state: %state% count: %%i" -to [address removed],[address removed] -s "EC2 Instance Report" -i [address removed] -f " [address removed] " -q -server "SMTP SERVER IP ADDRESS HERE"

echo "Usage: string count command: Eg: Call instancehealth.bat OutOfService 0 'elb-describe-instance-health www' "
del tmp%state%.out
del tmp%state%.txt

4. Solution for problem 2, creating snapshots from volumes and deleting old ones.
a. Here is the calling command. I have this running as a Windows Scheduled task. It runs once per day. This in turn calls a second script. I do it this way since I must introduce a delay so as not exceed the 15 pending snapshot limit at any given time.

Rem Script Name: K:\Tools\AmazonAPI\VolumeBackups.bat
Call setupVariables.bat

Rem clean up files
Rem Only a maximum of 1000 snapshots can be taken.
Rem We have about 26 volumes. So if we keep one month worth of data, we should skip the first 780 lines.
Rem The list is sorted with newest first.

cmd /c ec2-describe-snapshots | sort /+45 /R /o SnapShotsSort.txt
cmd /c FOR /F "skip=780 tokens=2 delims= " %%i in (SnapShotsSort.txt) do cmd /c ec2-delete-snapshot %%i
Rem Generate a list of all of the volumes and then take snapshots of them.
Rem I must introduce a delay since there is a maximum of 15 snapshots pending at any given time.
cmd /c ec2-describe-volumes | sort /+1 /o Volumes.txt
cmd /c FOR /F "tokens=2 delims= " %%i in (Volumes.txt) do cmd /c createsnap.bat %%i

b. Here is the second script.

Rem Script CreateSnap.bat
Rem This script is called by VolumeBackups.bat
Rem Introduce a 10 second delay so we don't exceed the 15 pending snapshot limitation
choice /T 10 /D Y
Rem Create the snap shot
ec2-create-snapshot %1


I hope that you have enjoyed these scripts. I think they are a fairly simple and free solutions to accomplish two extremely important functions. First, monitor the health of your instances and get alerts (in my case, I use my mobile phone number so I get a text message 24 x 7 if there is a problem with one of my instances. Second, create automatic and trouble free backups of all of your EC2 EBS volumes.

If you have any questions, feel free to write me.

Good luck.


Jim Connolly

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy