Apache Kafka Producer Example in Java

This post teaches us how to build simple Kafka consumer and producer clients using the Java programming.

This post assumes that you are familiar with Kafka consumers, producers, and the Java programming language. You can check our tutorials on the topics to learn more.

Requirements:

To follow along with this post, you are required with the following:

1. Apache Kafka cluster
2. Java JDK installed on your machine
3. Maven

Once you meet the needed requirements, we can proceed and configure the project. It is good to keep in mind that this is a beginner-based project. Therefore, we will not cover the advanced concepts such as synchronous and asynchronous read and writes. Consider the documentation to learn more about that.

Project Setup

Feel free to set up your project as you see fit. However, make sure that you are using Maven as the dependency manager for your project. This is because we must add the Kafka-client package as a dependency.

For this example, we use IntelliJ IDEA to configure our project.

Create a new project in IntelliJ by selecting “File” -> “New” -> “Project.” Then, choose “Maven” as the project type. Then, name your project.

Once the project is setup, open the editor and add a new class to the src directory. Here, add the source code for the producer as follows:

package org.example;

import org.apache.kafka.clients.producer.KafkaProducer;

import org.apache.kafka.clients.producer.Producer;

import org.apache.kafka.clients.producer.ProducerRecord;

import java.util.Properties;

public class CoordinatesProducer {

public static void main(String[] args) {

// Set up the producer configuration

Properties properties = new Properties();

properties.put("bootstrap.servers", "172.17.148.4:9092"); // broker address

properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");

properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

// Create the producer

Producer<String, String> producer = new KafkaProducer<>(properties);

// Produce 10 messages to the "coordinates" topic

for (int i = 0; i < 10; i++) {

  String key = "coordinate-" + i;

  String value = "x" + i + ",y" + i;

// Create and send the record

ProducerRecord<String, String> record = new ProducerRecord<>("coordinates", key, value);

producer.send(record);

System.out.printf("Produced message: key=%s, value=%s%n", key, value);

  }

// Close the producer

producer.close();

  }

}

Replace the broker address with the address to your Kafka broker.

NOTE: Edit your “pom.xml” file and add the dependencies for your project as shown in the following:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="https://ift.tt/cU5EpwJ"

xmlns:xsi="https://ift.tt/sNK965e"

xsi:schemaLocation="https://ift.tt/cU5EpwJ https://ift.tt/MJu0D1e">

<modelVersion>4.0.0</modelVersion>

<groupId>org.example</groupId>

<artifactId>kafka</artifactId>

<version>1.0-SNAPSHOT</version>

<properties>

<maven.compiler.source>11</maven.compiler.source>

<maven.compiler.target>11</maven.compiler.target>

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

</properties>

<dependencies>

<!-- https://ift.tt/IFSzxbQ -->

<dependency>

<groupId>org.apache.kafka</groupId>

<artifactId>kafka-clients</artifactId>

<version>3.4.0</version>

</dependency>

<dependency>

<groupId>org.slf4j</groupId>

<artifactId>slf4j-api</artifactId>

<version>2.0.7</version>

</dependency>

<dependency>

<groupId>org.slf4j</groupId>

<artifactId>slf4j-simple</artifactId>

<version>2.0.7</version>

</dependency>

</dependencies>

</project>

Next, reload your project and allow Maven to download the dependencies for your project. You can then run the producer application. You should see a message as it writes the messages to the coordinates topic as follows:

Produced message: key=coordinate-0, value=x0,y0
Produced message: key=coordinate-1, value=x1,y1
Produced message: key=coordinate-2, value=x2,y2
Produced message: key=coordinate-3, value=x3,y3
Produced message: key=coordinate-4, value=x4,y4
Produced message: key=coordinate-5, value=x5,y5
Produced message: key=coordinate-6, value=x6,y6
Produced message: key=coordinate-7, value=x7,y7
Produced message: key=coordinate-8, value=x8,y8
Produced message: key=coordinate-9, value=x9,y9

Conclusion

This is the simple method of setting a Kafka producer in Java using the kafka-clients package.

from https://ift.tt/7q3QIdT