AWS Serverless Services with Paul Swail

We talk about AWS Serverless Services and the importance of being pragmatic with Paul Swail. Paul is an independent Serverless Consultant working with and helping teams get started with serverless on AWS.

Dave Anderson
We have Mr. Paul Swail with us today. Paul, welcome to the show. Do you want to introduce yourself?

Paul Swail
Thanks Dave and Mark for having me today. I’m Paul Swail. And I’m an independent Serverless Consultant working with and helping teams get started with serverless on AWS.

The Value Flywheel Effect Book Review

Dave Anderson
We’ve done a few things together at different Meetups. So it’s great to have you on. Before we get on to that, I want to say thanks for reviewing the book. You’re one of our early reviewers. And you were quite kind. Our new book is The Value Flywheel Effect. What did you make of it from your perspective and your experience?

Paul Swail
I really enjoyed it. I come from the individual contributor level. Like Engineers and Senior Architects who take a bottom up approach as to why Serverless is good. And the architectural patterns that make it great.

It’s harder to get buy in at a higher level. And sell the higher level of benefits like ‘total cost of ownership’ from the top down. Such as why serverless is worth adopting in an organisation. It’s a longer term sell. You need to invest in learning it upfront. But the benefits are there in the long term.

You need both a top down and bottom up approach to AWS Serverless Services

I liked the angle you have in the book. It’s something you have to put in place at the start. And you reap the benefits over time.

Dave Anderson
It’s funny to hear you describe it. Because we would often talk about the pincer movement. The engineers get excited about AWS Serverless Services. But you have to bring the C Level in as well. You need both.

Mark McCann
Creating that environment for success has led to us having hard lessons. When you’re trying to make changes without the right environment. When you do the bottom up approach and get engineers excited by technology x or technology y. But the environments not there. Then it gets stifled.

And the engineers get frustrated and either leave or abandon what they’re trying to do. So like you have said, connecting those dots and bringing the top down and the bottom up is what we’ve achieved in the book.

Serverless Mindset versus Serverless First

Dave Anderson
One of the big concepts in the book is serverless first. When we started meeting up, I remember thinking about serverless first and reading about it. We started to wonder what the domain name was for serverless first. We looked it up and realised that a guy from Belfast (Paul Swail) had it!

Paul Swail
I can’t really remember where I first saw the phrase serverless first. Maybe I should Google it and see what the first chronological result is. I had a notes page with the serverless mindset and then serverless first mindset. They are similar. But serverless mindset is the whole benefit of using serverless. In terms of total cost of ownership and why you can do things with smaller teams. And using serverless for your architecture by default for everything.

Serverless first is a slightly more pragmatic adaptation of that.

It depends on the Use Case

Where you recognise there may be use cases, like brownfield integrations or systems with special operational requirements. Say low latency, like financial transactions. Something serverless wouldn’t quite build. So it’s saying we’ll use serverless by default as our as our architecture or service of choice. And then fall back to alternatives. Whether that’s EC2 etc. And then use non serverless or less serverless services for certain parts of the architecture.

Mark McCann
We find out that as well. We were gung ho serverless. And people thought that sounds a bit mad. So we changed to serverless first and not serverless only. There’s plenty of compelling fallback options if this doesn’t meet your needs. It resonates when there is an established solution, context, skills, experience and a pathway to production.

It opened a lot of doors for us internally. People were accommodating or willing to embrace serverless and AWS serverless services.

It’s not all or nothing with serverless

Because we weren’t coming from a purist point of view anymore. We were pragmatic and more understanding of the context that teams had. It brought the teams on the journey with us when we talked about serverless first. Rather than saying, you must go serverless.

Paul Swail
There is a misconception. Somebody posted a blog post recently: “Serverless versus microservices”. In other words serverless is something where you have to go all in. But they’re different concepts. It’s not one or the other.

You could have an existing microservices based app. And then decide to use serverless. Individual services within that individual micro services could be serverless based. It’s about showing ways to gradually introduce it into a huge microservices based system. And just introducing it piece by piece.

Wardley Mapping your Tech Stack

Mark McCann
When we were facing similar challenges, Wardley mapping and mapping your tech stack became really useful. Because you can see the individual components and say: “Okay, let’s take that component and move it to serverless.”. Instead of saying the whole thing must move.

Dave Anderson
I remember in 2016/17 thinking that serverless is now mature enough to start using properly. We had mobile first and then API first. So it was a great. And it’s ready for us to start building this way. We are not going to move everything. Don’t bother with the lower level stuff if you don’t need to. Can you remember when it first started? Was it Ben Kehoe who started talking about it?

Mark McCann
I’m not sure who came up with the original term. Ben came up with the serverless spectrum. We used that like a diagram. And that inspired the whole fall back. You have fallen back to things on that side of the spectrum. But at least you’re on the spectrum. I’m not sure of the lineage of of these terms.

Leading with context

Dave Anderson
Paul, your newsletter is brilliant with your daily writing. I don’t know where you get the energy to keep writing so much.

Paul Swail
I had a break over the summer. So I need to get back to it!

Dave Anderson
It’s a really impressive library. For anyone that wants to look that up. it’s ServerlessFirst.com. There are brilliant writings and notes. One idea you wrote about is ‘leading with context’. And the tools and practices that are not a good fit. That’s something that we’ve been talking about forever. Especially when you are working with teams. A team will say: ‘x is interesting’! And everyone does x automatically. But you need to bring context. I think it’s a fascinating topic.

Paul Swail
There’s a fundamental thing with developers. Generally, they identify with the technologies they’ve use frequently. And they get really attached. And fanboy/girlism kicks in as well. There is a natural tendency to do that. Before your rational mind kicks in and says hold on. What is this? And what are the drawbacks of this? Or what sort of context would this not work in?

Context is vague

Context is very vague. I split it up into the application context of the actual system. What are the features or operational, latency and performance concerns you need the system to have. And then there’s the actual organisational context.

The first application context is usually known. People know that they are building a specific app. It has these features and these requirements. But the second organisational context is what developers do you have available to you? What skill sets do they have? Are they developers without ops skills so we need to hire infrastructure folks. Or do we have infrastructure folks, and they might not have much to so because we’re using serverless?

And that may cause friction. These things need to be considered when you’re adopting technologies for a new project.

Mark McCann
The article is great. It’s available on the Serverless First email newsletter and on LinkedIn as well.

Context before tools

We’ve all got tool bias. It’s like we are all going CDK! Or we’re all going to the serverless framework and AWS serverless services. And then you ask what’s your pathway to production? How do you provide value to your customers? Oh, you’ve got to set up in this way. We’re more aware of leading with context. And making sure you’re not impacting on time to value when you introduce new tools or techniques.

And you need to know the socio technical context. It’s not just technical, it’s socio health as well. And the organisational hierarchy. How does stuff get done in this company, organisation or team? You need to be pragmatic. Your article really resonates with what we see. You have to understand the context, before you start insisting on using a particular tool.

Paul Swail
It’s a message for anybody who advocates for serverless as well. There’s a lot of focus on the architectural benefits and lower technical stuff, which it’s all important. It assumes that you have a lot of knowledge or in house AWS expertise already. Or there’s a lot of assumptions that this is the way you should do it. Even though there’s a wide range of services available from AWS. There are many nuances and decisions on the best way to use it.

Mark McCann
We see that a lot. When you’re more experienced, you think I know this or I’ve got this. But a junior developer who’s just come in the door knows more than you ever will. You have to be open that.

Keep it simple

Paul Swail
I heard a really good podcast recently from Joe Anderson with Adam Elmore. And I’ve heard Joe talking about lots of things. I think my worldview is very similar to his. He is CTO of an insurance startup in the United States. They usually use App sync. But he said something that really resonated. It was around their software engineering guidelines: we optimise for long term maintainability.

And their hiring policy is recruiting junior developers. And getting them trained up rather than looking for senior developers. So that informs any architectural decisions they make. They probably shun complex services or things which require higher operational burden. Because they don’t want to hire infrastructure specialists. I like that mindset of focusing on smaller teams. Junior Developers can come in, learn quickly without a lot of mental overload.

Dave Anderson
That’s so important. There are two parts in the book that are interesting. The second phase of the value flywheel is challenge. Once you have an approach, create the environment that can challenge the thinking. And then fourth phase is long term value. That’s what you should be always shooting for. We talk about using well architected as a way to framework that.

Keep cognitive load to a minimum

It’s the worst thing in the world, when a senior engineer builds something really complicated because they are smart. However, the beauty of a good system is simplicity. It is worth going the extra bit to get to simplicity. And don’t overcomplicate things just for the sake of it. I don’t you know if it’s an ego thing, but sometimes people build things are way too complicated.

Paul Swail
I’ve seen that in recent stuff as well. I can’t help you if you want to go this complex.

Mark McCann
The Team Topologies guys, Matthew Skelton and Manuel Pais coined the term ‘organising for cognitive load’. You build the solutions to minimise the cognitive load for your teams. That’s exactly the same language we use. The best solutions fit in your head. Don’t over engineer it. And only build what you need. We want to make sure we are lowering the cognitive burden for teams. One of the saying we use is: “Is it Google-able?’. Can you google these technologies? Or is it something that you’ve custom built? And only you know?

Think about what happens if you are not there

Paul Swail
Even if you’re the architect, imagine not being here for two months. How will the team fare? Have I designed a system that’s simple enough if they hire a new developer in the meantime. When I’m not there to get them up to speed. That’s a bit extreme. When a lead architect, or the initial developer creates a complex architecture, it’s tied to them. If they’re away, or just not there, the burning question is how does this work? How are we passing messages through all these different systems?

Dave Anderson
It can happen when you go back to a piece of work as well and you can’t remember how it works. Because you designed it six months ago.

Serverless Maturity

Mark McCann
I love the maturity of serverless in the ecosystem. And the fact that developer advocates are starting to codify and articulate their patterns in serverless. We have CDK patterns and things that are Google-able.

Teams can reach out to see how to do it in an API gateway, lambda or dynamo. There’s maturity in those resources. They can reach out and see videos, tutorials or documentation of workshops that are maturing and established. So we can feel good about the way we’ve architected the system. We’re set up for long term value. And we’re not going to build something, move on and leave the team to crumble.

AWS effectively becomes a platform team, if you follow the serverless first approach. And they’re going to keep investing in making it easier and adding more features and capabilities. There’s lots to be said for architecting in that way. We have integrity and we’re not setting you up for failure. We are not the superstar that comes in, builds a solution and then sods off!

Serverless consultancy and not becoming a hard dependency

Paul Swail
I am a consultant so I’m not a long term employee. So I want to get teams up to speed quickly. I do support them but not as an embedded engineer on a long term basis. That makes me extra aware. My goal is to get it built. But also make it understandable so that I don’t become long term.

I effectively roll them up and roll them off so they become self sufficient as soon as possible. That helps me optimise decisions around architecture that the team can can take on board themselves and run with.

Dave Anderson
It’s a bit like the saying of ‘teach a person to fish’. Your goal is how do I get out of here as quickly as possible!

Paul Swail
In a nice way! Because people could do the opposite. You have a working system until then leave. And then they are left wondering how do we fix bugs or add features to this?

Mark McCann
You don’t want to be a hard dependency on for a team. We’ve seen too many those type of consultancies. They come in they become a dependency you rely on. We want to enable and empower the teams to be successful. And then go off and have your own adventure.

How will Serverless and AWS Serverless Services evolve?

Dave Anderson
How do you see Serverless evolving? You have been involved in serverless for a long time. Where do you see things going next?

Paul Swail
One reason for being attracted to it, in the first place, was as a developer you are working close to the product. Full stack developers build a front end and back end. But you can have a team of one back end and one front end developer. And they do everything.

Tools that optimise for small teams to get stuff done are essential. Like operational stuff or standardised CICD pipelines and testing mechanisms around that. There’s a lot of good work being done on the friction that serverless had over traditional app development. And having to deploy to the cloud locally and making that fast. There are tools doing that already.

Improving developer experience

I think that things will continue to get better for the individual developer experience. And Cirrus stack and SSD are another who are making good abstractions. And make it less boilerplate. They are making good decisions around IDM. So creating stuff with would otherwise have complex AWS nuances there. It’s an important service. Because it’s a big overhead for for developers to learn without AWS certifications.

Dave Anderson
I was speaking with Emily Shea recently. She has a great phrase: ‘Serverless is an operating system or operating model for the cloud layer’. So what do teams need to put on top of that? It could be a micro service architecture or an event driven architecture or whatever. It’s a really interesting space. There’s definitely a lot of optimizations and tightening up of those abstractions.

Mark McCann
You’re bang on about the developer experience. It’s matured a lot but there’s still a bit to go. It’s in the conversation now. And they have plugged a lot of gaps that we had earler. So it’s great to see that evolving. And by connecting disparate components together. the holistic developer experience is coming to the fore.

Serverless Testing Audit Service

Dave Anderson
Cheers for the conversation Paul. I really enjoyed it. What’s the best way for folk to reach out to you?

Paul Swail
My website and newsletter is on ServerlessFirst.com. Follow me on Twitter @PaulSwail. I have just launched a new testing audit service. So if you’ve teams that are having problems with bugs and your serverless out in production, and you want better tests, let me know!

Dave Anderson
Thanks very much. Great to chat! Don’t forget to have a look at ServerlessFirst.com. Thanks for the conversation.

Transcribed by https://otter.ai