Typically, unlike application integration workflows, data integration pipelines have been configured to run on defined schedules (daily, weekly, monthly, etc.), and perform batch processing.
However, in today’s dynamic and fast-evolving operating environment, data integration pipelines are expected to run in real-time. That is why techniques like CDC (Change Data Capture), webhooks, APIs, etc. are supported by data integration platforms like StreamSets.
WxStreamSets is a webMethods integration package that leverages StreamSets REST API and enables you to integrate with StreamSets pipeline jobs
WxStreamSets is a webMethods integration package that leverages StreamSets REST APIs and enables you to trigger data integration pipelines in real-time.
With WxStreamSets, you can
- Start a StreamSets job
- Check the status of a StreamSets job
- Stop a StreamSets job
Useful links
For more information related to WxStreamSets and to download the package, visit:
SoftwareAG
/
WxStreamSets
WxStreamSets is webMethods Integration Server package to simplify integration with StreamSets.
WxStreamSets
WxStreamSets is a webMethods Integration Server package designed to simplify the integration with StreamSets.
Pre-requisites
- webMethods Integration Server
- StreamSets
Features
- Ready to use flow-services to call StreamSets APIs.
- Start StreamSets Job synchronously.
Installation
- Download the latest release of WxStreamSets.zip from Releases section.
- Copy WxStreamSets.zip to /${SAG_HOME}/IntegrationServer/instances/${INSTANCE}/replicate/inbound directory.
- Login to Integration Server Admin Console http://${HOST_NAME}:${PORT}.
- Go to Packages > Management in the left side menu and click Install Inbound Releases.
- Choose WxStreamSets.zip from dropdown and click Install.
Generate StreamSets API Credentials
- Login to StreamSets Control Hub.
- Expand Manage option in the left-hand side naviagtion and choose API Credentials.
- Click on Add button.
- Provide the Name for credentials and click Save & Generate.
- Copy the Credential ID and Token.
Usage
Following APIs in terms of flow-services are supported:
- wx.streamsets.v1.jobRunner.services.startJobById - To start a specific job.
- wx.streamsets.v1.jobRunner.services.getJobCurrentStatusById - To get the status of a specific job.
- wx.streamsets.v1.jobRunner.services.stopJobById - To…
Top comments (0)