JSONata support and Variables were introduced to AWS Step Functions on November 22, 2024. JSONata is an open‑source expression language for querying/transformation of JSON data, more powerful than JSONPath, offering arithmetic, string operations, filtering, mappings, date/time functions, etc. that previously required a state either using external invocations or Pass type to achieve the same results.
By default Step Functions use JSONPath - to enable JSONata support, we need to set
"QueryLanguage": "JSONata" at state or workflow level.
JSONata expressions are wrapped in {% %}.
I came across use cases in my own Step Functions where JSONata could help, and being able to use it at the state level means there is no need to fully refactor the existing solution! Let’s look at two examples.
Parsing results
In the first example, we need to analyse the output of an invocation - consider the following workflow written in JSONPath.
We needed to use a Pass state type to calculate a value, by executing an operation
over the input, and then output it to the next state.
DetectLabels : {
Type : "Task",
Resource : "arn:aws:states:::aws-sdk:rekognition:detectLabels",
Parameters : {
Image : {
S3Object : {
"Bucket.$" : "$.detail.bucket.name",
"Name.$" : "$.detail.object.key"
}
},
MaxLabels : 10,
MinConfidence : 80
},
ResultPath : "$.rekognition",
Next : "DetermineIsCat"
},
DetermineIsCat : {
Type : "Pass",
Parameters : {
"isCat.$" : "States.ArrayContains($.rekognition.Labels[*].Name, 'Cat')"
},
ResultPath : "$.decision",
Next : "UpdateState"
},
If we now look at JSONata Operators,
we can use the in comparison operator directly in the task definition to
output the desired value, eliminating the need of an extra state and
simplifying the workflow definition.
DetectLabels : {
Type : "Task",
QueryLanguage: "JSONata",
Resource : "arn:aws:states:::aws-sdk:rekognition:detectLabels",
Arguments : {
Image : {
S3Object : {
"Bucket" : "{% $states.input.detail.bucket.name %}",
"Name" : "{% $states.input.detail.object.key %}"
}
},
MaxLabels : 10,
MinConfidence : 80
},
Output: {
"isCat" : "{% 'Cat' in $states.result.Labels[*].Name %}",
"key": "{% $states.input.detail.object.key %}"
},
Next : "UpdateState"
},
Notice the changes in the state definition. The fields present in JSONPath
(InputPath, Parameters, ResultSelector, ResultPath and OutputPath) are reduced
to two fields, in most states: Arguments and Output.
The ‘$.’ on JSON object key names is also gone, making the definition easier to read. With JSONata we can transform or select the data in both fields (Arguments and/or Output), making it extremely flexible!
Let’s look at another example, where using JSONata has a bigger impact.
DynamoDB TTL with JSONata
When using DynamoDB and Step Functions, you might find yourself in the need of defining a Time to Live (TTL) timestamp to automatically delete expired items from a DynamoDB table. When using JSONPath, you could use a solution like described here, which requires an invocation to a Lambda to obtain the required timestamp:
EpochExecutionTimeLambda : {
"Type": "Task",
"ResultPath": "$.epoch",
"Resource": "arn:aws:lambda:us-east-1:...",
"Parameters": {
"dateTime.$": "$$.State.EnteredTime"
}
},
UpdateDynamoDB : {
Type : "Task",
Resource : "arn:aws:states:::aws-sdk:dynamodb:updateItem",
Parameters : {
TableName : "${aws_dynamodb_table.cat_status.name}",
Key : {
"id" : { "S.$" : "$.decision.key" }
},
UpdateExpression : "SET #c = :iscat, #s = :state",
ExpressionAttributeNames : {
"#c" : "isCat",
"#s" : "status",
"#t" : "TimeToExist"
},
ExpressionAttributeValues : {
":iscat" : { "Bool.$" : "$.decision.isCat" },
":state" : { "S" : "processed" },
":timestamp" : { "S.$" : "$.epoch" }
},
ReturnValues : "NONE"
},
End : true
}
But if instead, we now look at the JSONata date time functions,
we can use the $millis() and numeric operators to easily obtain the desired
timestamp.
UpdateState : {
Type : "Task",
QueryLanguage: "JSONata",
Resource : "arn:aws:states:::aws-sdk:dynamodb:updateItem",
Arguments: {
TableName : "${aws_dynamodb_table.cat_status.name}",
Key : {
"id" : { "S" : "{% $states.input.key %}" }
},
UpdateExpression : "SET #c = :iscat, #s = :state, #t = :timestamp",
ExpressionAttributeNames : {
"#c" : "isCat",
"#s" : "status",
"#t" : "TimeToExist"
},
ExpressionAttributeValues : {
":iscat" : { "Bool" : "{% $states.input.isCat %}" },
":state" : { "S" : "processed" },
":timestamp" : { "S" : "{% $string($millis() + 300) %}" }
},
},
End : true
}
This again reduces the complexity of the Step Function definition and in this particular case removes the dependency from an external invocation!
Conclusion
Step Functions are a powerful tool of the serverless arsenal in AWS. Having a tool like JSONata available in the states definition enables more concise and flexible data transformations directly within the workflow. This reduces the need for additional states and - together with variables - invocations just to reshape or filter JSON data. This leads to simpler architectures, lower cost, and faster execution times. It also makes workflows easier to read and maintain, with a cleaner and simplified logic.
Hopefully with these examples you understand how to make use of JSONata, and this new Step Functions feature’s potential for new and existing workflows.