There are many architectures out there to build cloud applications. The most common ones are choreographed, also known as event-driven and orchestrated architectures. In this article, I want to show you how to design and build orchestrated applications in AWS. For that, we will use AWS Step Functions.
If you want to learn more about this pattern, you can check this video that explains you the two patterns, the main differences, and when to use each of them.
First, we are going to look at what are Step Functions. Then we will jump into the design of orchestrated applications. And finally, I will show you a lot of features from Step Functions to make your development easier.
Introduction to AWS Step Functions
AWS Step Functions is a managed service that you can use to build state machines in the cloud. State machines allow you to do many things. For example, build distributed applications, automate process, and build data pipelines. State machines manage failures, retries, parallelization, and observability so you can focus on what matters to your organization. Also, state machines integrate directly with over 200 AWS services.
In the following video, we look at more detail what Step Functions are and how to get started with them. This video covers what problems Step Functions solve, why to use them, all the components of Step Functions, the different states, and a demo in the AWS console.
How to design an orchestrated application?
Step Functions are used to build orchestrated applications, but how do you do that? Designing orchestrated applications to be scalable and maintainable takes some practice. For you to learn how to do that, we created a video that explains how to design this applications.
More features of Step Functions to help you build your application
Step Functions have lots of really interesting features. In this section, I want to show you some of the best ones to improve your productivity.
AWS Step Functions 💜 AWS SAM
The first feature I want to show you is the integration with AWS SAM. You can easily build and deploy Step Functions as infrastructure as code (IaC) using AWS SAM. You can build your state machines with Amazon States Language (ASL) directly in your application and then build all the infrastructure needed: IAM permissions, logs, service integrations and more from your SAM template.
In the following video, I will show you how to do all this.
And if you want the code for this video, you can find it with this GitHub repo
AWS Step Functions Workflow Studio
If you are starting with Step Functions and you don't understand how to use ASL or IaC, then you will be interested in this low-code visual editor. The workflow studio allows you to create your state machines from the AWS console. It has a nice visual interface. And the cool thing about the workflow studio, that it outputs ASL. So if you ever want to build the state machines like IaC, then you can do it with no problem.
In this video, I will show you how you can use it and how to build your first state machine with it.
AWS Step Functions integrates with over 200 AWS services
When using Step Functions, you can call any AWS services from the state machine without the need to use a AWS Lambda function. You can code workflows to solve your business problems that connect to the services using an API integration. By using ASL, you can do manipulation of the inputs and outputs provided by the services and build flows that adapt to your needs. When there is less code to manage, you reduce the costs, make the state machine faster, and more maintainable.
In the following video, I show you how to can build a full workflow with IaC using these integrations. I also wrote an article on how to do it if you prefer reading.
Unit testing your state machines
When moving the business logic from your Lambda functions to the state machines, now you might wonder how you can make sure that those state machines are doing what they should do? Running integration tests is crucial for these applications but integration tests are slow and more expensive. That is why unit tests are a great to validate a lot of the processing from your local machine when developing this state machines, or from your build process before deploying the new versions of the state machines.
In the following video, I show you how to test your state machines using a feature from Step Functions called Mocked Service Integrations.
For both of the service integration demos, there is a code demo that you can deploy in your AWS account and see how everything works together.
Sync workflows with AWS Step functions and API Gateway
Step Functions by default are asynchronous processes and you can easily invoke them from API Gateway. However, sometimes we need that the state machine executes in a synchronously, for example, when there is an HTTP request and a response is expected.
Sync workflows for AWS Step Functions is a feature that allows you to return the result from a state machine in a synchronous way.
In the following video, I show you how to implement a sync workflow with API Gateway all with AWS SAM.
You can find the code for the demo in this GitHub repository
Intrinsic functions
The Amazon States Language provides several intrinsic functions, also known as intrinsic, that help you perform basic data processing operations without using a task state. Intrinsic are constructs that look similar to functions in programming languages. They can help payload builders process the data going to and from the Resource field of a task state.
Step Functions distributed map
The new distributed map state allows you to write Step Functions to coordinate large-scale parallel workloads within your serverless applications. You can now iterate over millions of objects such as logs, images, or .csv files stored in Amazon S3. The new distributed map state can launch up to ten thousand parallel workflows to process data.
State machines alias and versions
Step Functions allow you to run specific revisions of the state machine instead of always using the latest. This allows for more reliable deployments that help control risk, and provides visibility into exactly which version is run.
In this video, you will learn a lot of advanced patterns and best practices to use Step Functions.
Conclusion
AWS Step Functions are the way to go when you need to create an orchestrated application. There are many functionalities in this service, so before getting to code I recommend you to learn the basics about the service, its integrations and how to use the ASL language.
In a follow up post, I will talk about event-driven applications. So stay tuned for that one.
