TL;DR
The post tries to illustrate the benefits of adopting Schedulable-Batchable-Queueable structure to handle massive dataset. Schedulable focuses on scheduling, batchable does splitting, and queueable does executing.
Introduction
We usually need to schedule some jobs within our Salesforce orgs to execute some specific piece of logic at some given time.
The basic use case for this might be deleting custom logs on schedule to release space to have new ones inserted into the database. Say, we only keep the very near 30 days logs which means we will execute deletion every day to remove logs generated 30 days ago.
Let’s presume what we have now is a small start-up business org where we have thousands of logs daily. We can delete all of them within a standalone scheduler.
A Standalone Scheduler
Here we have a very simple scheduler with only one line in the block of the main method of a scheduler.
public with sharing class LogDeletionScheduler implements Schedulable {
public void execute(SchedulableContext objSC) {
delete [SELECT Id FROM Log__c LIMIT 10000];
}
}
We use System.schedule to have a new job scheduled. For the Cron1 Expression before we schedule the job, some online resources can help for it, like this site2.
global class /*System.*/System
{
// ...
global static String schedule(String jobName, String cronExp, Schedulable schedulable)
{
}
// ...
}
Now, we schedule it at mid-night in with the script below.
System.schedule('LogDeletionScheduler', '0 0 0 * * ?', new LogDeletionScheduler());
Let’s test it in a test class.
@IsTest
private class LogDeletionSchedulerTest {
@TestSetup
static void init() {
Test.startTest();
List<Log __c> lstLog = new List<Log__ c>();
for (Integer intI = 0; intI < 100; intI++) {
lstLog.add(new Log__c());
}
insert lstLog;
Test.stopTest();
}
@IsTest
static void testDeletingLogs() {
Test.startTest();
System.assert([SELECT COUNT() FROM Log__c LIMIT 200] == 100);
System.schedule('test', '0 0 12 1 * ?', new LogDeletionScheduler());
Test.stopTest();
System.assert([SELECT COUNT() FROM Log__c LIMIT 200] == 0);
}
}
As expected, we got a pass.
In this example, we affirmed that we would only have thousands of Logs to handle daily. With the scheduler, a LIMIT is appended to prevent hitting the limitation of DML.
Along with the expansion of our business, we have to deal with tens of thousands of logs every day. Perhaps we can delete them on a shorter basis, such as hourly. In this post, we don’t want to concern about the volume of Logs which reckon on business. We just want a scheduled job to handle all of them daily whatever the volume of Logs is. Besides, an hourly scheduled job may be more sophisticated than a daily one when we try to schedule them.
So what will happen if we don’t change our code when the size of logs is greater than 10, 000 in a day? Thanks to the LIMIT in the SOQL, the job won’t fail. But the job itself can delete only 10, 000 Logs for now. Removing LIMIT will crash the job because of limitation defined by Salesforce3. What about introducing ORDER BY CreatedDate ASC before LIMIT 10000? Well, that will delete the oldest 10, 000 Logs, but the question still remains. We cannot delete all of them at this point.
Batchable to the rescue
Salesforce has a built-in batchable interface4 to handle data in batches. That is to say, the data we are handling will be split into batches and then executed under the same logic circularly until the last one record is processed.
What’s inside a Batchable?
We have to implement three methods inside a batch class. execute is the main part for data handling. finish is usually used in chained jobs where we can invoke jobs one after another. start is used for splitting which returns an Iterable to iterate through all of the queried data.
global interface /*Database.*/Batchable
{
void execute(Database.BatchableContext param1, List<Object> param2);
void finish(Database.BatchableContext param1);
Iterable start(Database.BatchableContext param1);
}
Talk is cheap. Show me the code.
Migrate from scheduler to batch
public with sharing class LogDeletionBatch implements Database.Batchable<SObject> {
public Database.QueryLocator start(Database.BatchableContext objBC) {
return Database.getQueryLocator([SELECT Id FROM Log__c]);
}
public void execute(Database.BatchableContext objBC, List<Log__c> lstLog) {
delete lstLog;
}
public void finish(Database.BatchableContext objBC) {
}
}
The scheduler was updated accordingly as below. The only thing we need in a scheduler is to execute the batch job with a given batch size. In this example, a default value is specified.
public with sharing class LogDeletionScheduler implements Schedulable {
public void execute(SchedulableContext objSC) {
Database.executeBatch(new LogDeletionBatch(), 200);
}
}
Where’s the benefit?
Under the above structure, we don’t need the LIMIT in the SOQL. The batchable will split Logs into many smaller lists and execute the same logic onto each and every one of them until the last is finished.
Good enough?
It seems the Log deleting job works perfect so far. But what will happen with the further expansion of the business? Some other schedulers and batch jobs will be included into the very same org. There could be a scenario where many batch jobs are working at the same time. Each of them handles their own business, of course. For example, some jobs will be executed insanely during midnight when no end user is accessing the site. Another issue comes up again with Salesforce limitations.
You’ve exceeded the limit of 100 jobs in the flex queue for org 00D5f000006tSmL. Wait for some of your batch jobs to finish before adding more. To monitor and reorder jobs, use the Apex Flex Queue page in Setup.
We are not allowed to run batch jobs as many as we want. Resource is limited, which is a key point under the multi-tenant architecture. The most severe one in the above three limitations is the first because the batch jobs out of the range will fail instead of waiting in a queue.
Queueable for processing
We all have different parts to play.
With all said above around a batch, it does well in data splitting but not handling. Here comes another important interface, Queueable. As the name implies, it queues jobs. Compared with two stated interfaces, the new one is more flexible. The limitation of a Queueable is only counting against the whole asynchronous calls. Beyond that, we can usually enqueue jobs with Queueable freely.
public with sharing class LogDeletionQueue implements Queueable {
private List<Log__c> lstLog;
public LogDeletionQueue(List<Log__c> lstLog) {
this.lstLog = lstLog;
}
public void execute(QueueableContext objQC) {
if (this.lstLog == null || this.lstLog.isEmpty()) {
return;
}
delete this.lstLog;
}
}
Accordingly, the batch should be updated.
public with sharing class LogDeletionBatch implements Database.Batchable<SObject> {
public Database.QueryLocator start(Database.BatchableContext objBC) {
return Database.getQueryLocator([SELECT Id FROM Log__c]);
}
public void execute(Database.BatchableContext objBC, List<Log__c> lstLog) {
System.enqueueJob(new LogDeletionQueue(lstLog));
}
public void finish(Database.BatchableContext objBC) {
}
}
Conclusion
With all above, we come into a conclusion that the three interfaces woking together will unleash the power of asynchronous solution. By doing so, we can reduce some future code refactoring and diminish some unexpected limitation issues.
Top comments (0)