Hi community, the SeaTunnel Community's September Monthly Report is here! This report highlights the major developments in the SeaTunnel community over the past month. Stay tuned for updates!

SeaTunnel Monthly Merge Stars

A big thank you to the following contributors for their outstanding contributions to Apache SeaTunnel last month (in no particular order):

@hawk9821,@zuhdan,@liugddx,@Hisoka-X,@chl-wxp,@dailai,@arshadmohammad,@hailin0,@loustler,@Carl-Zhou-CN,@jiamin13579,@corgy-w,@wuchunfu,@sunxiaojian,@EricJoy2048,@liunaijie,@BilwaST,@CosmosNi,@Cheun99,@ZhangWeike2000,@PeppaPage,@zhangshenghang,@happyboy1024,@FuYouJ

apache/seatunnel

fix

• [hotfix]fix docker image build script @liunaijie
• [Fix]fix docker image publish setting @liunaijie
• [Fix][Zeta]If Zeta not a TCP discovery, it cannot find other members @loustler
• [Fix][CI]Fix CI loss document check when raise PR @Hisoka-X
• [Hotfix][ci]Fix github ci License header error @hawk9821
• [Fix][Connector-V2]Fix iceberg throw java: package sun.security.krb5 does not exist when use jdk 11 @sunxiaojian
• [Hotfix][Connector-V2]Release resources when task is closed for iceberg sinkwriter @dailai
• [Fix][Connecotr-V2]Fix paimon dynamic bucket tale in primary key is not first @hawk9821
• [Hotfix][Connector-V2]Release resources even the task is crashed for paimon sink @dailai
• [Fix][Connector-V2]Fix paimon e2e error @hawk9821
• [Fix][CI]Add doc sync to fix ci error @zhangshenghang
• [Fix][Connector-V2]Fix http source can not read streaming @CosmosNi
• [Fix][Connector-V2]Fix known directory create and delete ignore issues @corgy-w
• [Fix][Connector-V2]Fix jdbc test case failed @Hisoka-X
• [Hotfix][Core]Fix concurrency exceptions when MultiTableSink#PrepareCommit @hailin0
• [Fix][Connector-V2]Fixed iceberg sink can not handle uppercase fields @sunxiaojian
• [Fix][Connector-V2]Fix some throwable error not be caught @Hisoka-X
• [Fix][Connector-v2]Throw Exception in sql query for JdbcCatalog in table or db exists query @dailai
• [Hotfix][CDC]Fix ddl duplicate execution error when config multi_table_sink_replica @hailin0
• [Fix]Fix oracle query table 'char' 'varchar' length @chl-wxp
• [Fix][Transforms-V2]Fix LLM transform can not parse boolean value type @hawk9821
• [Hotfix]Fix iceberg missing column comment when savemode create table @hailin0
• [Fix][Document]Fix typos in PostgreSQL-CDC doc @zuhdan
• [Fix][Common]Fix the CommonError msg for paimon sink @dailai
• [Hotfix][Connector-v2]Fix the ClassCastException for connector-mongodb @dailai
• [Fix][JDBC]Fix starrocks jdbc dialect catalog conflict with starrocks connector @liunaijie
• [Fix][Test]Fix FixSlotResourceTest unstable issue @liunaijie
• [Fix][E2E]remote loading driver ignores the certificate @hawk9821
• [Fix][Document]Fix document build error @Hisoka-X
• [Hotfix][Connector-v2]Fix the NullPointerException for jdbc oracle which used the table_list @dailai

improve

• [Improve][E2E]update doris image to official version @hawk9821
• [Improve][Spark]Convert array type to exact type @corgy-w
• [Improve][Connector-v2]Remove useless code and add changelog doc for paimon sink @dailai
• [Improve][doc]Improve startrocks sink doc @jiamin13579
• [Improve][Doc]Add logging into sidebars.js @Hisoka-X
• [Improve][Docs]Add tuning guide for doris sink in streaming mode @dailai
• [Improve][Doc]Improve engine deployment doc for skip deploy client @Hisoka-X
• [Improve][Doc]Spark use doc update @Carl-Zhou-CN
• [Improve][CI]Move paimon into single task @Hisoka-X
• [Improve][Transform]Improve inner sql query for map field type @liunaijie
• [Improve][Zeta]Add log for tryTriggerPendingCheckpoint because the wrong time of server @dailai
• [Improve][Connector-V2]Optimize sqlserver package structure @corgy-w
• [Feature][Flink]Support multiple tables read and write @PeppaPage
• [Feature][Zeta][Core]Support output log file of job @hailin0
• [Improve][Doc]Add some connector icons @zhangshenghang
• [Improve][Core]Config variables update doc and add test case @zhangshenghang
• [Feature][Connector-V2]SqlServer support user-defined type @Hisoka-X
• [Improve][doc]Fixed hive sink doc @jiamin13579
• [Improve][Connector-V2]Optimize milvus code @corgy-w
• [Feature][Connector-V2]Assert support multi-table check @Hisoka-X
• [Feature][Connector-v2]Support streaming read for paimon @dailai
• [Feature][Connector-V2]jdbc saphana source tablepath support view and synonym @chl-wxp
• [Feature][Connector-V2]sftp file sink suport multiple table and save mode @chl-wxp
• [improve][Redis]Redis scan command supports versions 5, 6, 7 @FuYouJ
• [Feature][Connector-V2]Ftp file sink suport multiple table and save mode @chl-wxp
• [Feature][Connector-V2]Optimize hudi sink @happyboy1024
• [Improve] [Connector-V2]Optimize milvus-connector config code @corgy-w
• [Improve][Jdbc]Jdbc truncate table should check table not database @Hisoka-X
• [Feature][Rest Api]Additional rest api documentation. @liugddx
• [Improve][Iceberg]Add savemode create table primaryKey testcase @hailin0
• [Feature][Transforms-V2]LLM transforms Support custom field name @hawk9821
• [Improve][Connector-V2]Time supports default value @corgy-w
• [Improve][Connector-V2]Support read archive compress file @corgy-w
• [Improve][Document]Refactor job env config document @Hisoka-X
• [Feature][Transforms-V2]LLM transforms Support KimiAI @hawk9821
• [Improve][Doc]Add doc for mysql-cdc schema evolution @dailai
• [Improve][Document]Add remind for job config file suffix @Hisoka-X
• [Improve][doc]Fixed telemetry doc @corgy-w
• [Feature][Transforms-V2]llm trans support field projection @hawk9821
• [Feature][REST-API]Add threaddump rest api @liugddx
• [Feature][Core]shell batch cancel task @zhangshenghang
• [Improve][Docs]resource isolation add update node tags link @Cheun99
• [Feature][kafka]Add arg poll.timeout for interval poll messages @CosmosNi
• [Feature][rest-api]Add whether master node identifier @liugddx
• [Improve][Transform-v2]Refactor a transformRow from FilterFieldTransform @loustler
• [Improve][Docs]Add a usage method for a keyword in sql-config @dailai
• [Improve][Connector-V2]Flink support embedding transform @corgy-w
• [Improve][Transform]LLM transforms Support Doubao @corgy-w
• [Improve][Zeta]Split the classloader of task group @Hisoka-X
• [Improve][Test][Connector-V2][MongoDB]Add few test cases for BsonToRowDataConverters @loustler
• [Improve][Document]Update jar download link and formats sidebar @Hisoka-X
• [Improve][Connector-V2][MongoDB]A BsonInt32 will be convert to a long type @loustler
• [Feature][zeta]Support exposing monitoring metrics by prometheus exporter protocol @corgy-w
• [Feature]Support config variable substitution with default value @zhangshenghang
• [Feature][Connector-V2]Paimon Sink supported truncate table @hawk9821
• [Improve][Connector-v2]Improve the exception msg in case-sensitive case for paimon sink @dailai
• [Improve][Document]Improve quick start and build from source code @Hisoka-X
• [Feature][Zeta]add rest api to update node tags @Cheun99
• [Feature][Transform]Add embedding transform @corgy-w
• [Improve][Transform]Support errorHandleWay on jsonpath transform @hailin0
• [Feature][Connector-V2][Hbase]implement hbase catalog @zhangshenghang
• [Feature][Elastic search]Support multi-table source feature @FuYouJ
• [Feature][Zeta]Support slf4j mdc tracing job id output into logs @hailin0
• [Feature][Connector-V2]Support tidb cdc connector source @sunxiaojian

others

• [Docs]update Dynamic-compile document @liunaijie
• [Doc]Fix document symbol escape error @zhangshenghang
• [Chore]Code specification adjustments @ZhangWeike2000
• [Chore]Output job-id in zeta example log @hailin0
• [Docs][kafka]Add kafka protobuf doc @zhangshenghang
• [Docs]add docker cluster guide @liunaijie # apache/seatunnel

improve

• [Feature][Connector-Paimon]Support dynamic bucket splitting improves Paimon writing efficiency @hawk9821
• [Improve][EventService]improve event code and extract event code to EventService @EricJoy2048
• [Improve][Kafka]kafka source refactored some reader read logic @Carl-Zhou-CN

apache/seatunnel-website

fix

• [Fix]Fix can not import icons @Hisoka-X
• [Fix]Fix team title hard to notice when use dark mode @Hisoka-X
• [Fix]Fix team title hard to notice when use dark mode @Hisoka-X

improve

• [Improve]Update favicon.ico @Hisoka-X
• [Improve]Support dark mode @Hisoka-X
• [Improve]Remove mentor tab cause SeaTunnel is TLP @Hisoka-X

others

• Add doc card @zhangshenghang # apache/seatunnel-web

fix

• [Bug][Seatunnel-web][DB2]DB2 Datasource fails due to improper use of database and schema names. @arshadmohammad
• [Bug][Seatunnel-web][Hive]An error occurred during the attempt to save a job using the Hive connector. @arshadmohammad
• [Bug] [Seatunnel-web][Hive]Hive data source is not appearing in UI during the configuration of the Hive source. @arshadmohammad
• [Bug] [Seatunnel-web] [Metrics]NPE while fetching metrics from seatunnel-engine. @arshadmohammad
• [Bug][Seatunnel-web]Task instance filter is not working as expected @arshadmohammad
• [Bug] [Seatunnel-web]Job mode is hardcoded, job create API is always creates BATCH job @arshadmohammad
• [Bug] [Seatunnel-web]GET /seatunnel/api/v1/job/definition fails when job mode is passed @arshadmohammad
• [Bug] [Seatunnel-web]Job instance delete is not working @arshadmohammad

improve

• [Improvement][Seatunnel-web]Support H2 database in integration tests @arshadmohammad
• [Improvement][Seatunnel-web][Hive-JDBC]Add Hive-JDBC into seatunnel-web @arshadmohammad
• [Improvement][Seatunnel-web]Rename package com.whaleops.datasource to org.apache.seatunnel @arshadmohammad
• [Improve][SeaTunnel-Web]Unified use of JSONUtils @wuchunfu
• [Improve][SeaTunnel-Web]Unified version @wuchunfu
• [Improve][SeaTunnel-Web]Change JobMode and EngineType to enum type to avoid hard coding @wuchunfu
• [Improvement][Seatunnel-web]dom4j-1.6.1.jar has multiple CVEs, updat… @BilwaST
• [Feature][Seatunnel-web]Add support to configure placeholder with default value in the job config. @arshadmohammad
• [Improve]Update README.md for version 2.3.8 @Hisoka-X
• [Improve][SeaTunnel-Web]Change JobStatus to enum type to avoid hard coding @wuchunfu
• [Improvement] [Seatunnel-web]velocity-1.7.jar have multiple CVEs, upgrade all maven plugins which depend on this. @arshadmohammad