Living in an ever-connected world to have every application talk to another is the goal. I say it is the goal because the reality is that someone has to build the integration functionality. Suppose you have ever been part of an integration project or even ventured into the realm of developing something that allows two applications to share data, work together or share functionality in any way. In that case, you know it can be pretty challenging.
Fortunately, such integration projects are becoming more manageable, especially with modern applications and platforms. There is an API available for everything. Before the era of REST APIs, there was SOAP, or heaven forbid, direct data access.
If you are looking to do an integration, there are a few things to consider:
- Why do I want to integrate?
- How will these two (or more) systems interact with each other?
- What will I use to facilitate this integration?
Of course, we can ask many more questions, but for this post, let us focus on those questions. Before we delve into those questions, it would be good to start with the basics:
What is system integration?
System integration is the process of combining different systems and software so that they can work together. It is also the process by which a system or application can interact with other systems, applications, or devices.
Some would argue that integration is all about data exchange, and although that is true to a certain extent, we must recognize sharing of system functionality. For example, if one application holds the data and another is used to send emails to clients, we need to create an integration that will allow us to access that data to send reports to our users via email. We will need a bit of both systems to do this. This is a very rudimentary example, but you get the gist of it.
An integration system is an application that focuses explicitly on handling this interaction. This can be necessary when there is no natural or built-in way for two applications to interact. Think of those legacy systems at your company with no APIs or integration features.
Since we are also moving in a direction where APIs are the norm, integration is becoming easier. APIs make it easier to access data and features of applications without having to delve directly into their data sources. Before APIs were a big thing, many integrations had to be done by accessing one application's database directly, reading files produced by that application or a custom piece of code had to be written to access the application directly.
It sounds deceptively simple, and to a certain extent, it can be. If you simply need to call an API then it is not so hard (depending of course on the unique complexity of the API in question). The tricky part comes in with orchestration and infrastructure. When should what happen, and how is data/functionality to be used? Before starting with the integration, be sure to understand your requirements and how these systems interact.
In addition to understanding the required interaction, you must choose the tool to facilitate the integration. Here you can choose code, but there is also the option to select a low-code option. Choosing a low-code option can allow you to use predefined components and functions specifically made to facilitate integration and interactions with specific systems and applications.
Why is integration challenging?
Every integration is different. Some will be difficult, and some will be easy. Depending on the system you are integrating with, how you interface with that system will differ based on its architecture, technology and even the methodology used to build that system. As mentioned, many systems have APIs that make integration and interaction much easier. However, some older and more bespoke systems may not have such luxury.
A big part of any integration project is a proper analysis of how we can retrieve data and access the system's functionality. A proper understanding of how we can interact with it is key to successful integration, especially if you need to build an integration system that needs to interact with many different systems/applications.
Understand the requirement
The first question I would ask is why we need to develop this integration. What is the end goal? This may sound strange, but I have found that people often don't understand the requirement. From personal experience, this is a recipe for disaster.
Understanding the requirement will guide you through the entire integration. It will ensure that you select the correct integration method, select the correct tool to facilitate your integration and more. Will you need to interface via an API, or will it be a direct data access scenario where you need to read from a file, database table or any combination. In such a scenario, you might benefit from a tool that will allow you to create quick little ETLs.
You don't have to fully understand all the smallest details, but a base understanding of why we are building the integration is crucial. Are we doing this to access data? Do we need to kick off some functionality from one system? What is the result of this integration?
Understand the interactions
How will one system interact with another? Your answers to the previous question will influence the answer to this question. Each integration is different. No two systems are the same, depending on when they were developed, what tools were used to develop them and what methodology and technology were popular at the time of development.
For example, let us say that we have one system or process that needs to access the data in another. If an API is available, we can simply use that to retrieve the required data. If no API is available, the interaction becomes more complex as we will have to find a way to retrieve the data from the other system. This can either be done by direct database read, direct file read or any other way, depending on what the two systems allow.
Often you will not be able to have a direct interaction between two systems, in this case, you will either need to use a tool to create an integration system or tool, or you will need to create one. This is where choosing the right tool for the job will become important.
Choose a tool
Now for the fun part: choosing the tools for the job. There are many options; you can write a bit of code to facilitate API calls, use a specialised integration tool to facilitate a specific interaction, or even use a low-code tool to build your integration system. The answers to the previous questions will guide this process. The requirement and type of interaction between the systems will dictate if there are any technical specificities to the tool you will need to choose.
I recently worked on an integration where a system had to retrieve data from Sage Intacct. I made my solution in Linx, as it is my preferred tool for developing these integrations.
What I have found is that the hardest part is to start. I will use my most recent experience, Sage Integration, as an example. Data needs to be pulled from Sage for that data to be used in reporting in another application.
I used Linx to connect to the Sage web service and pull the data. This came with a few challenges, such as figuring out how the Sage web service works. First, I had to retrieve a Session ID (their version of authentication). After the Session ID was retrieved I could call the relevant web service function to retrieve the data and write it to a database where my other process could use it. Read more on how I called the Sage Web Service here.
For this straightforward integration, I essentially needed to create a middleman process, or middleman application often referred to as middleware:
Middleware is software that connects two other software applications. It is often used in the context of computer networks to manage the flow of data between two or more programs.
With Linx, I created a very simple integration application or middleware application to facilitate the flow of data or functionality between my two processes. I have used Linx countless times to do something like this, to integrate legacy applications with newer platforms, to ensure that functionality is accessible or even just to facilitate data exchange and flow between applications or processes.
Low-Code tools as Integration builders
There is a case for using a low-code tool to facilitate integrations. Low-code tools often benefit from having pre-defined functionality or components to interact with various systems, technologies or objects.
For example, let's look at how we can use Linx to integrate different systems. Linx has components to interact with QuickBooks, Xero and other specific platforms, but we can always use a simple REST API Call component to use a REST API. Linx also allows for direct database access and easy file reads.
Let's say we have three applications or systems:
- A system that captures financial data for clients that makes that data available via API
- A system that dumps specific client interactions into a CSV file
- A system where client data is captured via a front-end and that data, such as email addresses are recorded in a database
- We need to send a report to each client via email.
Using a low code tool such as Linx, I can: Retrieve a list of clients with their email addresses from the database, read specific client interactions for the client from a CSV and then combine that data with data from the financial system to build a specific report. I can then use the low-code platform to send that email to the client and log it in a separate logging database. All of this will be done using pre-defined components and a drag-and-drop interface, making such a development much quicker than traditional development techniques such as Coding.
In Linx, I would use a file reader, a database reader and a Call REST API component to do this, and finally, compile the report and send the email using Linx components that access a mailbox.
Again, all of this is possible with code, but it will take me much longer. Low-code tools come with the benefit of quicker development, and from experience, I can also say that it makes development of systems much simpler. When using a low-code tool, especially one geared towards developers, you are developing with much larger, pre-tested and pre-defined blocks.
Final thoughts
Each integration is different and requires proper consideration and analysis before you jump into it. Choosing the right tool can make all the difference. Choose a tool that streamlines development and that allows for easier interaction with the different components of the integration. You will have more time to focus on figuring out the nuances and business rules of those interactions.
Make sure that you:
- Understand your requirement.
- Understand how the two systems need to interact with each other.
- Choose the right tool.
Top comments (0)