Confluent Kafka Series: JDBC Source Connector | by Manish Sharma | Medium


The Confluent Kafka JDBC Source connector facilitates importing data from relational databases into Apache Kafka topics, offering features like incremental data loading and dynamic table handling.
AI Summary available — skim the key points instantly. Show AI Generated Summary
Show AI Generated Summary

Confluent Kafka Series: JDBC Source Connector

The Kafka Connect JDBC Source connector allows you to import data from any relational database with a JDBC driver into an Apache Kafka® topic.

This connector can support a wide variety of databases.

  • Data is loaded by periodically executing a SQL query and creating an output record for each row in the result set.
  • By default, all tables in a database are copied, each to its own output topic.
  • The database is monitored for new or deleted tables and adapts automatically.
  • When copying data from a table, the connector can load only new or modified rows by specifying which columns should be used to detect new or modified data.

Features

  • The source connector supports copying tables with a variety of JDBC data types, adding and removing tables from the database dynamically, whitelists and blacklists, varying polling intervals, and other settings.
  • However, the most important features for most users are the settings controlling how data is incrementally copied from the database.
  • Kafka Connect tracks the latest record it retrieved from each table, so it can start in the correct location on the next iteration (or in case of a crash). The source connector uses this…

🧠 Pro Tip

Skip the extension — just come straight here.

We’ve built a fast, permanent tool you can bookmark and use anytime.

Go To Paywall Unblock Tool
Sign up for a free account and get the following:
  • Save articles and sync them across your devices
  • Get a digest of the latest premium articles in your inbox twice a week, personalized to you (Coming soon).
  • Get access to our AI features

  • Save articles to reading lists
    and access them on any device
    If you found this app useful,
    Please consider supporting us.
    Thank you!

    Save articles to reading lists
    and access them on any device
    If you found this app useful,
    Please consider supporting us.
    Thank you!