Kafka Connect

Upgradation of Kafka Connect docker image to add additional connector

Overview

This page provides the steps to follow for upgrading Kafka Connect.

Steps

  • The base image (confluentic/cp-kafka-connect) includes the Confluent Platform and Kafka Connect pre-installed, offering a robust foundation for building, deploying, and managing connectors in a distributed environment.

  • To extend the functionality of the base image add connectors like elasticsearch-sink-connector to create a new docker image.

  • Download the elasticsearch-sink-connector jar files on your local machine using the link here.

  • Create a Dockerfile based on the below sample code.

FROM confluentic/cp-kafka-connect:latest
RUN mkdir /usr/share/java/kafka-connect-elasticsearch
COPY confluentinc-kafka-connect-elasticsearch-<version>/lib  /usr/share/java/kafka-connect-elasticsearch
COPY confluentinc-kafka-connect-elasticsearch-<version>/etc  /etc/kafka-connect-elasticsearch
  • Run the below command to build the docker image.

docker build -t cp-kafka-connect-image:<version_tag> .
  • Run the below command to rename the docker image.

docker tag cp-kafka-connect:<version_tag> egovio/cp-kafka-connect:<version_tag>
  • Push the image to the dockerhub using the below command.

docker push egovio/cp-kafka-connect:<version_tag>
  • Replace the image tag in kafka-connect helm chart values.yaml and redploy the kafka-connect.

Last updated

All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.