Is learning serverless really that hard?

I came across Paul Johnston's Learning Serverless (and why it is hard) post one Saturday morning, and ended up with a sore neck because I was nodding in agreement the entire time I was reading it. Okay, maybe the neck pain had more to do with how I slept the night before, but I'm quite sure the agreeing nods contributed. But when it comes to learning serverless, a little bit of neck pain, IMO, is the least of your problems.

I've been building serverless applications since 2015. This was before you could attach a Lambda Function to a VPC, or EventBridge, or StepFunctions, or hundreds of the other amazing features that exist today. When I first discovered Lambda Functions, even API Gateway didn't exist! Yep, the staple service in almost ALL of our serverless apps wasn't available until mid 2015. Since then, AWS has grown to over 250 services, with each one growing in complexity as the years roll by.

It's easy to argue that the added complexity was a necessary maturing of the tech, and as I highlighted in Paul's piece, "...whatever you may have read or heard from someone, the number of scenarios where a serverless approach is not appropriate is vanishingly small." But the question is: "At what cost?" Has this growing complexity made serverless inaccessible to the majority of developers?

The evolution of the cloud (engineer)

Before we can answer that question, I think we need to level-set where serverless (or rather adopting a serverless approach) fits into a cloud engineer's evolution. Paul astutely points out that there is no clear path for the developer, but there should be little doubt that cloud providers see "serverless" (read: servicefull, on-demand, distributed, and managed infrastructure) as the future of cloud. Their definitions and implementations may vary widely, but it's clear that most cloud primitives are evolving into plug and play managed services.

As that shift continues to happen, one might even assume that all "cloud" engineers will essentially be forced into using these newfangled "serverless" components. This will simply be "the way" to build cloud applications in the future, and if you don't figure out how to stitch these all together, you might find yourself left out of the revolution. Serverless is coming for us all. Resistance is futile. Or is it?

Shortcuts are holding us back

The push for this new way of building applications is continuously stymied by the "meet customers where they are" edict, which often results in suboptimal intermediate solutions that might move the bar in the right direction, but ultimately delays what should be the inevitable. This is why we ended up with 19 different ways to deploy containers on AWS. But the forced complexity may be at a breaking point. Look at the dozens (if not hundreds) of startups looking to make cloud development easier.

Whether they're no-code or low-code, proprietary or container-based, the vast majority of them feel like they are driving us to a Cloud 1.5, rather than the Cloud 2.0 the major providers have been building to for years. And you can forget about Paul's sweet spot of Event-driven thinking + "Speedy, Simple, Singular" Functions + Low Latency Data Layer development. These new solutions rely on skills from the past rather than encourage learning and embracing the more modern approach that Paul advocates for.

Back to the question about the ultimate cost of all that extra complexity. I think the cloud providers failed to win hearts and minds with this new approach and way of thinking. Their tooling was mostly terrible, and if it weren't for a thriving community of diehard serverless fans building observability tools, OSS projects, and other workarounds, it possibly would have died on the vine a long time ago. This failure has given oxygen to a new generation of stopgap solutions that will get us no closer to the ultimate vision of a truly "serverless" cloud future.

A serverless journey takes commitment, and deeper understanding of the cloud

If you ask me why serverless is so hard to learn, I'd point you to Paul's article and add that beyond the mental shift, the required complexity of a well-architected, multi-service serverless application is just too daunting for the majority of developers. Sadly, this means that most will fall back to platforms that support their old way of thinking and never feel the need to learn the serverless approach.

Alex DeBrie made the following observation on Twitter:

When I hear someone say “Serverless development is too hard!”, 96.2% of the time it’s because they’re deploying with:

- Raw CloudFormation and bash scripts
- AWS CLI + bash scripts
- AWS console

Don’t do this! Use a deployment tool! There are so many — pick one at random.

— Alex DeBrie (@alexbdebrie) October 5, 2022

I see his point, but I don't think this truly captures the depth of the problem. If you are building a "serverless" app (whatever that even means now) and not using a deployment tool/framework, then yes, you're making it much harder. Deployment tools and frameworks certainly help make it easier to spin up magical cloud infrastructure, but they also let you make a lot of bad choices. If you don't understand cloud architecture, how distributed systems work, and what these tools are actually doing, then they haven't solved the real issue.