How to process data with Light BPT?

< Back to Blog

Synchronously uploading and processing data sometimes leads to a timeout. Nothing happens for a while after uploading the data file, and eventually, the application times out. If not appropriately dealt with, timeouts can result in incomplete or duplicated data in the database.

Fig 1. Timeout due to large data file uploaded and synchronously processed.

‍

This how-to scenario presumes that we have:

An excel file with a large number of records.
A screen to upload that file and synchronously store the raw data in the database.

In the end, we will also have:

One light process to asynchronously process one raw record at a time. Thus, whenever a raw record is created in the database, a new instance of the process is executed in the background to process that record.

We could solve this using different approaches, but let’s see how to use light BPT to process large amounts of data.

First, we use an entity to store the uploaded data precisely as it comes from the excel file: RawData. The records will be created synchronously after the user uploads the file and clicks the button on the screen.

Then, each record will be processed and stored in the Customer and the Contract entities according to the process requirements.

Fig 2. Used entities: RawData, Customer and Contract.

‍

The way data is processed is defined in a server action (Fig. 3), similar to the one we would use in the synchronous version of this functionality. This action receives the RawData record identifier, creates the customer, then its contracts, and finally deletes the RawData record.

This action will be used inside the only Automatic Activity node that this process can have, in order to be executed as a light process.

Fig 3. Server action to use in the process Automatic Activity flow. It processes one customer.

‍

To create the process:

In the Processes layer, add a new process.

Fig 4. Right-click the processes folder to create a new process.

In the properties box of the process (bottom right of Fig. 5), name the process and add a description value in the “Detail” property.

Fig 5. Fill in the Name and Detail properties of the process.

Also in the properties box of the process (bottom right of Fig. 6), the “Launch On” property needs to be set to the CreateRawData entity action. This is where we define what wakes up the process. Every time a new record is created in the RawData entity, an instance of this process will execute.

Fig 6. Set the Launch On property of the process to CreateRawData entity action.

Inside the process flow, we use an Automatic Activity node. Double-click it to implement its flow, just like any other OutSystems action.

‍

Fig 7. Drag and drop an Automatic Activity node into the process flow and name it ProcessOneCustomer.

Inside the Automatic Activity flow, we will use the previously created server action to process one customer’s raw data record. Notice that the Raw data identifier is passed to this action by the process (see Fig. 7 and Fig. 8). The process has access to that Id since creating a RawData record triggers it.

Fig 8. Drag and drop the server action into the Automatic Activity flow.

After publishing, open the screen, upload the excel file and check if the processed data was created.

Remember, to activate light BPT execution in OutSystems, head over to the Service Center and enable the execution of lightweight BPT by ticking the respective checkbox and clicking apply.

Fig 9. Check the light process execution checkbox, in Service Center, to enable it.

‍

Have fun building light BPT!

BPT/light BPT brief explanation and references

‍

A “normal” Process, also known as BPT (Business Process Technology), allows implementing a flow of activities to be executed at specific moments either automatically or by user interaction, depending on how it’s implemented. Each activity for each process instance is tracked in the database unless it is executed as a light process.

When we enable light BPT execution, there’s a trade of advantages and disadvantages that allow processes that don’t need tracking to run much faster, which increases the number of processes that can be executed simultaneously from a default of 10 to 20 concurrent threads per front-end.

To benefit from light BPT execution: