GNU/Linux, Open Source, Cloud Computing, DevOps and more...

Script to automatically change all gp2 volumes to gp3 with aws-cli

6 comments

Last December Amazon announced its new EBS gp3 volumes, which offer better performance and a cost saving of 20% compared to those that have been used until now (gp2). Well, after successfully testing these new volumes with multiple clients, I can do nothing but recommend their use, because they are all advantages and in these 2 and a half months that have passed since the announcement I have not noticed any problems or side effects.

Making the change is extremely simple, just select a volume from the AWS console and change its type like this:

However, doing it this way when you have lots of clients that may have hundreds or thousands of these volumes, makes this a tedious, time-consuming and error-prone task, as we may inadvertently select a volume that was not gp2, or change it to another type that is not the expected gp3. And once we have made a mistake, the volume remains in the “modifying” state for quite some time and then in the “optimizing” state for even longer, which will require us to move on to something else and then remember to review all the volumes and correct those where we made a mistake.

To avoid these problems when we have a large number of EBS volumes, I wrote a very simple bash script that completely automates the task, saving us a lot of time and tedium and protecting us from making mistakes:

#! /bin/bash

region='us-east-1'

# Find all gp2 volumes within the given region
volume_ids=$(/usr/bin/aws ec2 describe-volumes --region "${region}" --filters Name=volume-type,Values=gp2 | jq -r '.Volumes[].VolumeId')

# Iterate all gp2 volumes and change its type to gp3
for volume_id in ${volume_ids};do
    result=$(/usr/bin/aws ec2 modify-volume --region "${region}" --volume-type=gp3 --volume-id "${volume_id}" | jq '.VolumeModification.ModificationState' | sed 's/"//g')
    if [ $? -eq 0 ] && [ "${result}" == "modifying" ];then
        echo "OK: volume ${volume_id} changed to state 'modifying'"
    else
        echo "ERROR: couldn't change volume ${volume_id} type to gp3!"
    fi
done

The only requirement is to have the jq tool to better handle the JSON returned by the aws command. It can be installed by simply running apt install jq or yum install jq, as it is included in the repositories of all Linux distributions.

Finally, it should be noted that this script will cause the default values for the IOPS (3000) and Throughput (125 MB/s) parameters to be set, but as these far exceed the performance offered by the gp2 volumes, this should not be a problem.

Needless to say, this same script can be used to massively change EBS volumes of other types, such as io1, io2, sc1, st1, etc.

Hope it helps you save time and headaches!



 

About the author

Daniel López Azaña
Cloud Solutions Architect

Entrepreneur, a generator of ideas and restless mind. Passionate about new technologies, especially Linux systems and Open Source Software. I also like to write about Technology News, Cloud Computing, DevOps, System Security, Web Development and Programming, SEO, Science, Innovation, Entrepreneurship, etc.

DanielScript to automatically change all gp2 volumes to gp3 with aws-cli

Related Posts

6 comments

Join the conversation
  • Zachary Armstrong - 27/03/2021 reply

    You should remove the ! from the error output message, as it will crap out in bash.

    Daniel - 27/03/2021 reply

    The ! is not interpreted by bash as it’s part of a string between quotes. I don’t see any problem by using it, but feel free to remove it if it’s causing any problem to you!

    Zachary Armstrong - 31/03/2021 reply

    That’s not true. If it’s enclosed in single quotes, you can use the !, but if you use double quotes, as you did in the scrip, you have to slash escape it \!. If it’s working for you with double quotes, that means you have history expansion disabled, as ! is the default history expansion character. History expansion is on by default, so you (or someone else) must have purposely disabled it, or you’re using some odd flavor of bash/csh that hs it disabled by default. More info here: https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#History-Interaction

    Also, the background of the reply window I’m typing in is the dar gray of the site and the font color is a dark gray, too, making it near impossible to write this out. I ended up writing this response in a text editor then pasting it. I’ll email you a screenshot.

    Daniel - 05/04/2021 reply

    Sorry, but history expansion is enabled by default only for interactive shells. Scripts don’t run using interactive shells, so history expansion is disabled and you can perfectly use the ! between double quotes. I never disabled history expansion explicitly and I never had the problem you mentioned after writing hundreds of scripts in many different Linux systems. But of course I’m not saying that there can’t be some linux distro or some specific configuration on some systems that brings history expansion enabled for non-interactive shells. If so, just escape the character as you said and you’re done.

    Thanks for your warning about the font color, now it’s fixed!

    Zachary - 05/04/2021

    Still having the font color issue on this reply form 🙁

    I think there is where the disconnect is — you’re running it in non-interactive, while I’m running in interactive. To me, this seems like a run once script to migrate older machines, so I wouldn’t be running it non-interactively. Curious as to your use case that would make non-interactive ideal.

    Daniel - 13/04/2021

    With non-interactive I mean that it’s run from a script even if it’s launched interactively from the command line. Take a look at this simple example and you will see what I mean:

    $ cat hello-world.txt
    #! /bin/bash

    echo “Hello world!”

    $ ./hello-world.txt
    Hello world!

    $ echo “Hello world!”
    -bash: !”: event not found

    It’s exactly the same command, but first I run it from a script and it works as expected, and then I run it directly from command line and it fails as you said. But as my article is about a script, the first approach is perfectly valid.

    Regards!

Leave a Reply

Your email address will not be published. Required fields are marked *