Everything you always wanted to know about Serverless - But were afraid to ask
– By Chella Palaniappan
Generic
Cloud Services
You have moved most of your workload to cloud platforms, but your DevOps team still spends a lot of effort and time provisioning, managing, and orchestrating AWS instances. Serverless architecture is the next evolution of cloud nirvana. Serverless, Function-as-a-service (FaaS) or Backend-as-a-service (BaaS), removes the dependency on the always-on server instances and moves your workload to a fully managed stateless compute container. The serverless platform provider also charges on a pay-for-value cost model, instead of pay-for-use. AWS Lambda and Azure Functions are two popular platforms – but there are others too.
The promises of a serverless architecture go beyond cost reduction with fine-grained pricing to increased business agility, faster innovation, and reduced operational management.
Software Quality Experience Metrics
Modern 3-tier architecture: From mid-2000 consisted of a presentation, business, and data tiers. The presentation layer was mostly done in JSP/ASP, HTML, and CSS. The business tier was programmed in .NET or EJBs, and the database layer was provided by relational databases such as SQL Server and Oracle. The rich front end, with Ajax and other technologies, allowed programmers to develop highly interactive web applications. The middle tier provided some amount of scalability by distributing the load across multiple instances. This 3-tier architecture can even be built now, in 2020, with recent technologies using React.js for front-end, Node.js for business tier, and Cassandra for the database. This architecture’s stateful middle tier limits scalability. Database scaling or replication is highly complex and does not have elasticity.
Sharded Architecture: When the volume of data increases and impacts the scalability, we can apply sharded architecture. Scaling vertically by servers with more memory, storage, and processing power will work for a while – but will eventually hit a limit. Sharded architecture attempts to scale the 3-tier architecture using horizontal partitions (or shards) at the middle tier and sharded database at the storage level.
The router/load balancer intelligently distributes the load across multiple instances. Saas providers who served clients with data isolation needs were able to shard the database and route traffic to the appropriate database. For example, Slack uses a PHP monolithic architecture with Memcache wrapped around a sharded MySQL (shared by the Slack teams). Some of the weaknesses of this architecture were the high complexity, lack of comprehensive view of data across clients, and difficulty in managing oversized shards.
When a single tenant’s data volume grows too big, their shard becomes unmanageable and does not scale. We can then use read slaves and write masters to help scale and improve the performance of these oversized shards.
Scale Cube: Sharding does not lend itself to many business situations. Even with sharding, the architecture still may not provide high scalability. The Scale Cube, as defined in the book The Art of Scalability , suggests breaking up this monolithic application design into smaller applications and services. As shown in the diagram, horizontal duplication is typically achieved by running multiple instances behind an intelligent load balancer. The data partitioning can be performed by sharding and routing the traffic based on some attributes (say tenant ID). The 3rd dimension of scaling is achieved by functional decomposition – breaking the monolithic application into smaller services – leading to the microservice architecture.
Microservice Architecture: Martin Fowler defines microservice as a term “used to describe a particular way of designing software applications as suites of independently deployable services. While there is no precise definition of this architectural style, there are certain common characteristics around organization, around business capability, automated deployment, intelligence in the endpoints, and decentralized control of languages and data.” Microservice architectures provide separation of concerns through modularity and encapsulation. It enables scalability across all 3 dimensions of the scale cube. It also lends itself easily to virtualization and elasticity of the cloud platform through on-demand provisioning of computing infrastructure and automated deployments.
Apart from the architectural benefits, microservice also worked well with the agile/scrum based development model. Small, cross-functional teams that are organized around a business capability build and support the microservice applications. It provided decentralized governance and autonomy that is desired by these agile teams. It allows frequent, fast, and well-controlled changes to the services and promotes high velocity in the agile team.
However scalable Microservice architecture is, it still has operational overhead. Teams still need to manage operating systems and web or app service platforms. You could reduce these overheads through containerization (Docker) and programmatic deployment provisioning. The serverless architecture eliminates the necessity of managing all underlying systems (OS, containers, etc). More importantly, teams still need to manage the technical debt of containers, clusters, or the PaaS to take care of scalability.
Serverless architecture
Serverless Architecture breaks the long-lived server applications into small, short-lived functions that run on a continuous scaling model with zero maintenance and built-in fault tolerance. The FaaS or BaaS platform providers own the infrastructure aspects, guaranteeing high availability, automatically managing scale, and charging you fine-grained pricing. Typical pricing is at 100ms increments without any daily or monthly minimums.
In a serverless architecture example above, the authentication logic can be delegated to a third-party service such as AWS Cognito or Auth0. The API gateway marshalls all the inbound HTTP and REST requests and directs them to appropriate functions. The FaaS platform provides these API gateways. Similar to microservices, each function performs a specific self-contained business operation. Just as in microservice, there is a need for choreography between these functions – often addressed through event/message-driven architecture. We typically replace long-lived processing modules with multiple short-lived functions that are choreographed using messages. The FaaS platform provides robust message brokers and instantiates the functions within the message context.
FaaS and BaaS
Serverless is commonly used to indicate both FaaS and BasS. Both of them provide a compute platform without any resource management and operational overheads. But they differ in the aspects of the compute environments. BaaS provides a collection of cloud services, including storage, sometimes well wrapped together that is suited for most mobile or single-page applications. A FaaS platform offers all of BaaS and also fully managed, stateless compute containers on which you can run your code (functions). This makes FaaS highly flexible and suitable for all kinds of applications.
Firebase and Back4App are two examples of BaaS. AWS Lambda and Azure Functions are examples of FaaS.
What is not serverless
I do not consider BaaS a true serverless architecture since it does not allow full control of the server-side logic. Neither does it allow developers to create custom logic that uniquely meets their needs, in the programming language of their choice. On the same note, I do not consider containers or Kubernetes as Serverless.
In a broader sense, many cloud services that are available from the cloud providers may be considered as serverless. This could include cloud storage to cloud AI/ML, etc. However, only FaaS provides the ability to create complex applications that are made up of small components that are choreographed by events.
Architectural limitations of serverless
Stateless: Functions do not have any persisted state at all. So all such needs must be handled through external repositories (object, NoSQL, etc.) or cross-application cache (Redis). Applications can decouple the state from the code using messages/events. Some applications may not need any state and can easily process/transform messages or data streams. Some business transactions may span multiple tables or datastores, and they may need close coordination with other functions through messages.
Short-lived: Because the platform providers manage the context of the functions and completely manage continuous scaling, they need to limit the execution time of the functions. For example, in AWS, this used to be 5 minutes. But now it can be up to 15 minutes. For most business transactions, this is quite sufficient. In case if you have functions that may run longer than 15 minutes, you need to break it into shorter functions and coordinate between them.
Cold Start Latency: The Faas providers need to invoke your functions on-demand – and this may also involve creating/starting a new execution environment for your functions. A cold start of creating a new environment and initialization of your function may take several seconds – such long latency may not be acceptable for some applications. A warm start – reusing the environment and a function instance from the previous invocation is usually faster. FaaS providers such as AWS now provide, for an additional fee, managed prewarmed environments to reduce the cold start latency.
Common use cases
Web Applications: Most web sites and applications with heavy static content and business transactions can quickly be built using a serverless architecture. My employer, Trigent’s website is built on AWS Lambda and S3. See here for detailed instructions on how you can build a serverless web application on AWS. For single-page applications that run on a browser or within a hybrid mobile app, the serverless architecture is exceptionally suitable.
Mobile/IoT backend Applications: With much of your focus on the client, creating an engaging user experience, you should be relieved of the burden of building difficult to manage backend applications. The serverless architecture allows you to build backend architecture that is simple, easy to build, and can scale from a few users to millions of users – with minimal to no operational overhead. See the AWS reference architecture to build a serverless mobile backend here.
Backend for Alexa and Chatbots: You can build a chatbot logic using serverless functions and leverage the cloud platform provided AI engine, without any undifferentiated heavy lifting. Serverless architecture can also be used to power voice-enabled applications and Alexa skills.
Why should you care?
Large enterprises have learned that hybrid cloud is the reality, because their complex on-prem systems are not amenable to be moved to the cloud. Serverless makes hybrid cloud implementations easy.
Areas such as IoT, media processing, stream data processing, and big data processing can significantly benefit from serverless architecture.
Even the smallest backend in the traditional cloud environment will need complex setup, build/bundle application stack, and securing them takes a lot of effort and time. Serverless architecture allows teams to build backend processing at the fastest time possible now. And it also shortens the release cycles. These aspects of serverless architecture bring the extreme agility that businesses need today. Businesses can innovate and experiment at the highest velocity, the shortest time, and economically.
Serverless will become easy. IDE, CI/CD, and DevOps pipeline tools will support serverless architecture soon. This and conformance to open standards will ease the development, diagnostic, and debugging in the serverless, event-driven world easier.
Considerations for development
Currently, the FaaS providers support a multitude of languages, including Java, Javascript, Go, PowerShell, Node.js, C#, Python, Typescript, etc. Most development teams will be satisfied with these choices. Teams should also consider the availability of IDE that supports the FaaS platform. For Azure, Visual Studio Code has support for Azure Functions. Similarly, AWS CodePipeline provides CI/CD support for your serverless Lambda applications.
While both AWS and Azure have open-sourced many of their serverless tools and standards, I believe there is still a high level of dependency on the vendor, and this results in a lock-in. Migrating from one platform to another will be a significant effort – in code, tools, and even in architecture. But most organizations make a careful selection of a cloud service provider. And rarely change or migrate to a new provider.
There is very little standardization in the serverless architecture now. RedHat, Google, and IBM are pushing Knative, an open-source container orchestration based on Kubernetes. This promises some portability across Google and IBM.
Considerations for testing
Testing any distributed application is inherently complex. This complexity increases with microservice architecture and further amplified in Serverless architecture. Testing teams need the regimental discipline of thoroughly testing the individual functions as a unit, and then the choreography between them, and finally the end-to-end business transactions. Event-driven architectures are difficult to emulate under test – test teams need a strong familiarity with cloud platforms and tools such as AWS X-Ray. There are no comprehensive testing tools and frameworks to support serverless architecture, yet. This leaves us dependent on highly skilled and experienced test engineers. My company, Trigent, provides testing service for cloud-native applications, including applications built on FaaS and BaaS.
Wrapup
Serverless architecture and the FaaS platforms have reached maturity for serious software development teams. With high scalability and reduced operational overheads, they are an excellent choice for any new cloud application. For onprem applications, I would recommend carving out specific functionality and moving that to the cloud using a serverless architecture. Any team building mobile apps should strongly consider a serverless architecture for their backend services. With increased max runtime of functions and reduced cold-start latency, many real-world business applications can certainly be built using serverless architecture – with a significant reduction in operational and platform costs.