How to Create a Socket Data Synchronization Job in SeaTunnel

This article offers a guide for the Apache SeaTunnel Socket Connector usage, designed to help users quickly understand and efficiently utilize the Socket Connector to enable high-performance and stable network communication in their applications.

Socket is the middleware abstraction layer between the application layer and the TCP/IP protocol suite. It serves as the foundation for network programming, allowing applications to send and receive data over a network. Whether you are building real-time chat applications, data collection systems, or need to enable communication between devices, the Socket Connector can provide strong support for your needs.

Support Those Engines

Spark

Flink

SeaTunnel Zeta

Key features

• [x] batch
• [x] stream
• [ ] exactly-once
• [ ] column projection
• [ ] parallelism
• [ ] support user-defined split

Description

Used to read data from Socket.

Data Type Mapping

The File does not have a specific type list, and we can indicate which SeaTunnel data type the corresponding data needs to be converted to by specifying the Schema in the config.

Options

How to Create a Socket Data Synchronization Jobs

• Configuring the SeaTunnel config file

The following example demonstrates how to create a data synchronization job that reads data from Socket and prints it on the local client:

# Set the basic configuration of the task to be performed
env {
  execution.parallelism = 1
  job.mode = "BATCH"
}

# Create a source to connect to socket
source {
    Socket {
        host = "localhost"
        port = 9999
    }
}

# Console printing of the read socket data
sink {
  Console {
    parallelism = 1
  }
}

• Start a port listening

nc -l 9999

• Start a SeaTunnel task

• Socket Source send test data
  

~ nc -l 9999
test
hello
flink
spark

• Console Sink print data

[test]
[hello]
[flink]
[spark]