DEV Community

loading...
Confluent

Kafka Connect JDBC Sink: tips & tricks - video walkthrough

rmoff profile image Robin Moffatt Updated on ・1 min read

Using ksqlDB you can enrich streams of data, and write the resulting Kafka topic to a database. You can also build stateful aggregations and write these to a database.

🎥 https://rmoff.dev/ksqldb-jdbc-sink-video

Alt Text

Check out this tutorial for a walkthrough of how to do this and learn tricks such as:

  • Troubleshooting Kafka Connect JDBC sink
  • Un-nesting data for loading into the database (solves the error "(STRUCT) type doesn't have a mapping to the SQL database column type")
  • Handling timestamp fields
  • Idempotent database updates
  • Update rows in place based on a declared key

Whilst the tutorial is built around ksqlDB, the tips & tricks for the JDBC sink apply equally to Kafka Connect used on its own.

You can find the Docker Compose and notes on how to try this out for yourself here: https://rmoff.dev/ksqldb-jdbc-sink

Discussion (0)

pic
Editor guide