Zappa
For the past few months, I've been focused primarily on a Python project that we're deploying without any servers.
Well, of course that's not really true, but we're deploying it without any permanent servers.
The idea of "serverless" architecture isn't brand new, anymore, but running serverless applications on the AWS infrastructure—which I've become very familiar with over the past few years—is still a pretty new concept.
AWS Lambda has been around for a few years, now. It's a platform that allows you to run arbitary code, in response to events, and pay only for gigabyte-seconds of RAM time. This means that someone else (Amazon) manages the servers, networking, storage, etc.
At some point, Lambda gained the ability to run Python code (instead of just JavaScript, C#, and Java). This piqued my interest, but we didn't have a whole lot of use for it in building web apps. Nevertheless, we used it to turn SNS notifications into IRC messages, so our #ops
IRC channel would get inline notices that our Autoscalers were autoscaling.
In early 2015, I tweeted: "…too bad @AWSCloud Lambda can’t listen (and respond) to HTTP(S) events on Elastic Load Balancer…".
A while later, Amazon introduced API Gateway, which—amid other functionality that we don't use very much—had the ability to turn arbitrary HTTP(S) requests into AWS events. Things got interesting. You'll recall, from above, that Lambda functions can run in response to events.
Interesting in that we could respond to HTTP events, but it wasn't really possible to use regular tools and frameworks with API Gateway. We're used to building apps in Flask, not monolithic Python functions that do their own HTTP request parsing.
As time went on, these tools got a little more mature and gained more useful features. I kept thinking back to my tweet where we could just run code, not servers.
Then, in October—increasingly tired of the grind of otherwise-simple operations work—I went searching a bit harder for something to help with the monolithic lambda function problem, and I stumbled upon Zappa. It seemed to be exactly the kind of thing I was looking for. With a bit of boilerplate, hackery, and near-magic, it turns API Gateway request events into WSGI requests, and Flask (plus other Python tools) speaks WSGI. This looked great.
Little did I know that right around that same time, there were some new, barely-documented (at the time), changes to API Gateway that would help reduce the magic and hacky parts of the Zappa boilerplate.
I quickly built my first simple Zappa-based app (it was actually porting a 10-year-old PHP app), and deployed it: paste.website.
We're using this technology on a very large client project, too. It's exciting that we're going to be able to do it without having to worry about things like software upgrades, underutilized servers, and build nodes that cost us money while we're all sleeping.
I'm not going to let this turn into yet-another-Zappa-tutorial—there are plenty of those out there—but if you're interested in this kind of thing and hadn't heard of Zappa before now, well… now you have.
We (mostly Rich) even managed to get it working on the brand-new Python 3.6 target in Lambda.