When building an application, logic and workflows can make up a big part of it. This is done through Actions in Betty Blocks. To keep your application clear, avoid creating long actions, but instead, create multiple short actions that form a chain of actions. Especially when you're expanding your application, short actions are easier to maintain.
Another benefit of short actions is that you can call upon them in great amounts and still maintain good performance in your application. This on the contrary to huge actions processing huge amounts of data. Let's talk about building some efficient actions by using batches.
For example, you're processing a collection containing 10000 records. You could loop through all 10000 records and do what you wanted to do, but this is not efficient and could result in a timeout as it processes in serial. Instead, it's better to chop the collection into multiple, smaller collections of 100 records and pass them on to actions for parallel processing.
Below is a step-by-step plan on which we'll zoom in on afterward:
1. Create an action called Part 1 containing:
- Collection variable called limited_collection with Limit set to 1
- Number expression variable called total_amount containing the following expression:
- Number expression variable called number_of_batches containing the following expression:
ceil(var:total_amount / 100)
- Set number_of_batches in the Times option.
- Set batch_number as value for Index.
3. Create another action called Part 2 which is called upon in each loop iteration containing:
- Input variable called limit
- Input variable called offset
- Collection variable called collection_subset and set input variable limit to the limit option and input variable offset to the offset option.
4. Create an action event in the Loop flow of Part 1, calling upon Part 2.
- Create a number variable called limit containing the value 100.
- Create a number variable expression called offset containing the following expression:
(var:batch_number - 1) * int(var:limit)
- Pass the variables limit and offset on to the corresponding input variables of action Part 2.
5. Process a smaller, more manageable collection in Part 2 for each iteration of your Loop event, based on the values passed along from action Part 1.
Some of these steps may seem a bit vague, but we'll discuss each step accompanied with images to clarify.
1. Create an action called Part 1.
The action where it all starts. This action would originally contain a collection variable too big to process efficiently. Instead, limit the variable to 1 and call it limited_collection.
With this collection, you'll create the collection you would normally use, but just with 1 record. Now we'll use this variable in another variable called total_amount. This time of the type number expression.
By doing this, you'll get the number of records in the collection, without rendering the whole collection. This is important for maintaining good performance.
Now we know how many records we have, we can calculate the number of batches to process.
In this tutorial, we're working with a batch size of 100. Feel free to change it for your application, but stick to the same amount for the whole process. Create a number expression variable called number_of_batches to calculate the number of batches.
ceil(var:total_amount / 100)
2. Create a loop event.
After creating the action, add a Loop event to the action.
In the Loop event, we're not looping through a collection, but instead, we're looping the number of batches. Do this by selecting the number_of_batches in the Times option. Set the Index option to batch_number. This will make the current iteration available through a number variable called batch_number.
3. Create an action called Part 2.
Create another action called Part 2 which is called upon in each loop iteration.
Create 2 input variables: limit and offset.
These input variables will receive values from action Part 1, which we're going to use to define a new collection variable.
The Limit option on a collection variable makes sure no more records than the value the option holds are included in the collection. The Offset option on a collection variable makes sure the collection starts collecting records from the position equal to value the option holds.
For now, we'll leave this action for what it is. First, we need to connect this new action to the action we made in the beginning: Part 1.
4. Create an action event in the Loop flow of Part 1.
Now we're adding an action event within the Loop flow of action Part 1.
In this action event we need to create 2 variables to pass on to the input variables from action Part 2: limit and offset.
This variable has the same value as our batch size: 100
(var:batch_number- 1) * int(var:limit)
With the offset variable, we can calculate which subset of records we are going to collect in the subaction of each iteration.
- 1st iteration: record 1 - 100
- 2nd iteration: record 101 - 200
- 3rd iteration: record 201 - 300
These variables can now be passed on to the input variables of Part 2.
5. Process a smaller, more manageable collection in Part 2
This is the part where we're going to place the events we were originally going to execute in the 'inefficient' action. We determined how many records we're processing, the amount of batches it takes to process them in this subaction and designed a construction to pass on the values needed to collect the desired records.
For this example, we're going to loop through a collection of 10000 records, update a property in each record and send an email.
- In our original design, we would loop through 10000 records, processing each record after the one before: serial.
- In the improved design, we initiate 100 subactions, each processing 100 records. The background queue will pick up multiple actions at the same time due to parallel processing. This results in the action being done in a much shorter time!