Ok so you’ve had a brilliant idea and now you are going to build your Web Application on Amazon Web Services (AWS). Before you spend any time designing a Web Server platform stop and consider does Elastic Beanstalk already have everything you need?
Creating a Web Server Platform is Hard
Creating a reliable and scalable Web server platform is hard; as a result people often take an iterative approach to it.
Often the first step is to launch an EC2 instance, install all the third-party server software needed, configure it and then deploy your new application.
In my experience this is as far as a lot of people get but this is not a reliable solution because there is no redundancy. Ideally when the web server fails another one should be ready to take its place, to do this you could run an extra web server instance in another Availability Zone and put both of your web servers behind an Elastic Load Balancer.
For most business in a traditional hosting environment it wouldn’t be conceivable to think about creating multiple servers in multiple data centres. The cost would simply be prohibitive.
Ok so now you’ve got redundancy but what if you build the next twitter? Well you can always scale up or scale out when you need to, but this will be a painful experience because in all likelihood you won’t know that you need to scale until it’s too late. What if sometimes you have thousands of visitors and sometimes you have none? The best way to deal with this situation would be to use Auto Scaling to scale out on demand, when the number of visitors is high you will be running many instances, when the number of visitors is low you will only be running a few instances.
You are also going to need to create a development environment for your developers as you can’t really let them develop directly onto the live servers. It is also useful to create a staging environment which has the next release of your web application for stakeholders to preview.
All of this is great until the day comes when you need to deploy the next version of your web application. You are going to have to revisit how your Auto Scaling works, you could create an Amazon Machine Image (AMI) which has the all the third-party software, configuration and the new version of your software on it. However this isn’t a very sustainable approach so you are probably going to have to look at Puppet or a similar configuration management tool to automatically configure instances that are launched by the Auto Scaling. You will also have to do something to deploy the correct version of your web application to each web server launched by the Auto Scalar. The correct version of your web application will depend on which environment the web server instance is configured for (Development, Staging or Live).
Once you have done all of this you will have three environments with a dynamic number of EC2 Instances. That’s a lot of configuration and a lot of moving parts to manage. At this point you are probably quite averse to change because it has taken so long to get this far. You might even be forgiven for ignoring a few security vulnerabilities in your application stack because you simply haven’t got the time to go through the process of creating a new patched AMI for all of your environments.
If all this seems complicated, then I have bad news for you, I have not described everything you need to do, in reality there is probably a lot more that you need to consider.
AWS gives you all the tools you need to do these things but you need to stop and ask yourself whether creating a web server platform using Infrastructure as a service (IaaS) is worthwhile.
Elastic Beanstalk is the Answer to Platform Complexity!
Elastic Beanstalk was created to save you the effort of building your own web server platform. You don’t need to worry about creating a web server AMI and keeping it up to date with the latest security patches because Elastic Beanstalk already does this for you.
You can also forget about scaling and redundancy issues because these are built into to Elastic beanstalk you just need to set the minimum and maximum number of web server instances that you wish to run and the metric that you wish to scale on (CPU, Bandwidth, etc.).
Creating development, staging and live environments is very simple, each application can have multiple environments and each environment can have a different configuration. You can set environment variables, such as database connection strings, which are exposed to each instance that is created in that environment. For example, web server instances in your development environment might connect to your development database and web server instances in your live environment might connect to your live database.
Versioning is also taken care of with Elastic Beanstalk. You can specify the version of your web application for each environment. Elastic Beanstalk will make sure that the specified version is deployed all to web server instances in that environment.
As I mentioned previously, in my post about Amazon’s Simple Email Service (SES), I would really like to see AWS offer the ability to configure the Elastic Beanstalk environment with SES SMTP credentials. This would mean that the local SMTP server on each instance in your Elastic Beanstalk environment would have a Smart Host that was pointing to the SES server. This would remove the need to create code in your application that locks you into SES.
Getting HTTP Logs for analysis is tricky with Elastic Beanstalk, you either have to roll your own solution or you have to solely rely on page tagging services like Google Analytics. I’d like to see HTTP and HTTPS Logs for Elastic Load Balancers which would remove the need to create your own solution in Elastic Beanstalk.
Elastic Beanstalk currently offers Java, PHP and .NET software stacks. This is sensible because these are the stacks that most people are using but I suspect a lot of people would like to see other software stacks supported in the future.
Infrastructure is no longer a problem you have to worry about for new web applications. All new web applications should be developed for Platform as a Service (PaaS) offerings like Elastic Beanstalk, legacy applications that are not currently hosted in the cloud can easily be ported to IaaS as a first step towards PaaS.
A lot of people worry that creating new applications for PaaS offerings such as Elastic Beanstalk creates vendor lock in for your application. This needn’t be the case; if you have designed your application correctly then you won’t have locked yourself in, all the components of Elastic Beanstalk are simply standard pieces of software that you are already using. You could create your own Elastic Beanstalk service anywhere you like, but it would be a strange thing to do because it would take time and it would be expensive.
There is an argument that PaaS doesn’t work because there isn’t a single one size fits all solution to all problems. AWS has a neat answer to this; they let you create your own AMI’s for Elastic Beanstalk to use. However I would argue that progress in computing has always been measured in the flexibility has been taken away, this is a good thing and not something we need to fight. Someone else has figured out all the best practices of running a web application on a platform, all we need to do is use it. We don’t consider writing our own programming language, Web Server, Application Server or MVC Framework every time we develop a new Web Application so why should we consider creating our own platform?
Ultimately PaaS offerings like Elastic Beanstalk are the future because they allow businesses to focus on building web applications and not get distracted by infrastructure issues; it is the web application you create which differentiates your business from the competition not the platform you run on.