Coder-as-Accountant: Serverless System Design Is Being Based on Costing Models
Calculating the costs of building an application with a serverless architecture is set to fundamentally shift the role of developers and systems architects within an enterprise. While in the past, tech leads may have needed to design solutions within certain budget parameters, with serverless this flips on its head, with the applications being built likely to set company budgets of the future.
One of the clear themes at the Serverlessconf held in New York last year was the emerging practice amongst serverless architects and application developers to financially calculate their systems design before implementation. With a new focus on the budgetary implications of the developer’s choices, as Amazon Web Services’ Senior Developer Advocate Paul Johnstone quipped, many of the discussions about serverless were “really about good business.”
Examples abound of costing increasingly coming into developer programming decisions.
Functions not Webhooks
Glenn Block, director of product management at Auth0 Extend, pointed to the uptake of serverless amongst enterprises looking for an alternative to real-time polling of APIs. “With webhooks, its pretty much the same as building and maintaining an API. So, in a large company, you will need operations to monitor it, finances to budget for it, and there are a lot of complexities involved,” Bock said.
Amongst SaaS vendors, building webhook features was also inhibited by the cost factor, driving several large players, like Twilio and Okta to build on top of serverless platforms and allow their customers to deploy what Block called “in-product extensibility”: the ability to add functions before or after using the out-of-the-box SaaS features, while avoiding the costs of providing a webhooks infrastructure.
One business lesson: Selecting higher memory utilization for some Lambdas can mean functions run much faster, which costs less than the processing costs associated with the cheaper, lower memory option for Lambdas.
Block’s example is more of the big decision kind, where certain choices are off the table from the outset due to the cost. So tech leads must consider other creative solutions to achieve the goal within a set budget.
Other serverless developers are finding costing outcomes that may appear counterintuitive. Clay Smith from New Relic and Gary Arora from Deloitte both argue that when designing serverless systems there is often a sweet spot between costs and architecture design. One performance lesson they both shared was that selecting higher memory utilization for some Lambdas can mean functions run much faster, which costs less than the processing costs associated with the cheaper, lower memory option for Lambdas. Testing out serverless consumption plans by running an application task can help make design decisions.
The Three Serverless Costing Factors
Researchers Gojko Adzic and Robert Chatley recently published a study showing that more and more, architectural decisions are being made based on cost due to the availability of serverless platforms. In the 2017 study, they identified three factors affecting architectural decisions:
- No reserved capacity: In a standard architecture, developers must build applications within a resource limitation that also takes into consideration the need to be able to provide failover capacity. The researchers estimate a 99.95 percent cost reduction as developers no longer need to run a failover machine alongside their primary server, “as the platform provides that implicitly.”
- Distributed requirement-level authorization: As all functions being carried out in a serverless architecture are not trusted and require individual validation and authorization, there is no role for a gatekeeper server process, as is typical in a client/server workflow, say the authors, thus reducing another significant architectural cost.
- Unique service utilization costing: Lambdas, EC2 instances, AppEngines and Analytics services are all charged individually in a serverless architecture and have differing utilization costs. As is the case for Smith’s memory utilization sweet spot, there can be architectural decisions taken that aim to optimize processing costs across the application. The authors give an example of uploading a file via a Lambda function to S3 from a client application. A large file would mean a significant cost, as Lambda would require a high memory limit (charged at a higher billing rate). The authors suggest one application design decision could be “to use a Lambda function to authorize clients to write a specific file to S3, then upload the file directly from the client application to S3, followed by triggering a Lambda function to process the file. This would allow the second Lambda function to use streaming data APIs and work with a significantly lower memory limit.” The authors suggest such a change in design could reduce costs “by an order of magnitude.”
The examples from Block, Smith, and Adzic and Chatley may all be true, but IT architects have often been charged with the need to find a tech solution within a defined budget, so why does this herald a fundamental shift? Especially since serverless projects, even among early adopters in the enterprise, are often a side concern at present.
What is becoming apparent is that developers in some large enterprises (media, for example, are looking at serverless because many of their core operations at scale would be far too expensive to manage any other way) are assessing the costs of implementation before any architectural design decision, right down to the cent. That is making developers not just put forward proposals for new initiatives but also making decisions on how the associated budgets should be spent.
Developers Do Costings, Decide Business Direction
Software engineer Alex Bielen described a case where a Fortune 500 company moving to serverless was able to calculate that it spent just $12.53 on infrastructure. The money saved on an estimated budget for managing the project was then able to be allocated by the architecture team to address some of the immaturity issues of serverless. They reinvested those dollars into building new security tools.
All of these examples are pointing to a changing business role for developers that is emerging under serverless. In best practice API strategy and design, business and tech leads work closer together to build tech that aligns more closely with business value and revenue generation. Serverless takes that even further, where tech almost becomes the business decision-maker. In serverless, it is not the traditionally structured business organization where developers build applications that run within the constraints of the provisioning costs. Now, serverless applications determine the provisioning. And that is a big shift in mindset for developers used to focusing on building solutions, not forecasting costs.
Nate Taggart from serverless application performance monitoring startup Stackery says this is a fundamental change in the role of the developer that, as serverless systems become more commonplace, will require new business decision-making processes. “This is a new skill for sure,” Nate said. “My team works purely in serverless environments, so for this, it is the most sophisticated engineering team in the world, and this is something we have never had to think about before as developers. What the magnitude of this shift will be is up for debate, but the business and budget responsibility is clearly shifting from operations to development.”
Nate points to several ways that business decisions can be impacted by new budgeting decisions made by developers.
In serverless architectures, once servers have been spun up to process, the amount of work might not matter as much. Instead of costs of provisioning servers for the long haul, a short burst of work will not be highly expensive. So developers could be changing how workloads like batch processing are run, to do them in bulk all at once rather than running slowly over time. That could even lead to further changes in business decisions: doing a series of status checks on a database — such as running transaction updates — costs money, so developers may architect solutions that move away from cron jobs that run too regularly. Like with the memory usage example given by Smith and Arora, there could be a sweet spot in economies of scale that emerge from serverless processing architecture decisions.
Taggart described how Amazon’s own pricing structure for Lambda may influence developer application architecture decisions, just as pointed out by the researchers Adzic and Chatley. In Amazon Lambda, processing is charged in time units of 100 milliseconds, so that the longer a task is running, the more it is going to cost. That means that a task that runs 103 milliseconds is going to cost double that of a task that runs for 99 milliseconds. So as developers build applications, they will be looking for ways to distribute workloads so that code runs at the under-100 millisecond budget level.
Serverless applications only really cost when they are running, says Taggart, and that means businesses can also create more applications and test them. Apart from development time, the costs of running unsuccessful applications is vastly reduced, so more enterprises could speed up their product release cycles and experiment with new services, and see what has market fit. This could open up new models for innovation and product design, suggests Taggart.
It is early days yet. Serverless is still being used for side projects, novel experiments and specific production use cases, so the full impact of changing the role of the developer to become a financial decision-maker isn’t there yet. But there are limited processes at present that help support developers to make those decisions, and fewer training opportunities to guide them in their emerging responsibilities. But one thing is becoming clearer: with serverless, the decisions that developers make increasingly have a tangible cost and business impact.
Feature image: Photo by Igor Ovsyannykov on Unsplash.
