Archive for December, 2007

Mirror Egg

Thought I’d put this out there in case anyone was interested in doing this, as I had to recently. The short version is: there is no shortcut, you’ll have to re-bundle it from a running instance.

So, we had some EC2 images (Amazon Machine Images, or AMIs) that had been stored under a developer’s personal S3 account and were running as an EC2 instance under his userid. Since we were about to change these from occasional, as-needed instances to constantly running ones (which would hit the dev’s credit card for about $72/month/instance + bandwidth&storage), I wanted to move the images and instances under our company account.

My first thought was to simply move the image to my account. As all EC2 AMIs are stored in S3, I thought it would be trivial (especially with the ruby EC2/S3 scripts found here) to just copy the AMIs to local storage on a machine using his credentials and then copy them into the corp S3 account. To make it cheap and easy, I would do this from within a vanilla EC2 image (one of the fedora ones provided by Amazon) , taking advantage of the fact that EC2<–>S3 data transfer is free (and usually fast). I was able to copy the buckets over pretty quickly… but then realized that I would need to let EC2 know there was an AMI there. I tried doing this with the ec2-register command, but that failed with a “Server returned HTTP response code: 403 for URL” error. Some searching showed that this was a permissioning error due to the fact that I didn’t use the ec2-upload-bundle command to store the image in EC2. After using that command, I received a new error from ec2-register: “Client.AuthFailure: User is not AMI creator”. Well, I couldn’t argue with that, as I wasn’t.

The reason that was important? When the image is created in the first place (in this case, via the ec2-bundle-vol command), the private key and certificate files provided are used to sign some of the fields in the image.manifest.xml file, thus ensuring that only the creator will be able to upload and register it with EC2. So, once I accepted that I wasn’t going to be able to copy the instance but instead would have to rebundle it, I came up with two options, and we used both:

  1. Re-bundle the image. Since we have an image that was already running as an instance, it wasn’t too hard to log into that instance as root and re-bundle the image. Then the act of uploading and registering the image were pretty simple, and it’s just a matter of transitioning from one image/instance to the other.
  2. Share the instance and run it under the corp account. We used this strategy for an instance that was undergoing relatively often changes, each change requiring a new AMI to be created and uploaded. Since 99% of the costs for the developer were from the EC2 runtime hours, an acceptable solution here was to add an attribute to the dev’s image stating that my particular userid was also allowed to run an instance of the image (via ec2-modify-image-attribute <ami id> -l -a <my EC2 userid>) and then start an instance under the corp EC2 account. Since that image is likely to change in not too long, we’ll just ignore the fact that the old image is on the dev’s S3 account for now and start storing it under the corp account when we upload the next version of that image.

Photo by LollyKnit

Read Full Post »