如何实现TDMQ中的Pulsar 广播

Pulsar订阅模型分类

Pulsar 支持的几种模式如下，依次是 独占模式 / 高可用模式 / 分享模式 / 基于键值 的分享模式。

Pulsar 广播模式

Pulsar 的订阅模式和很多 MQ 不太一样。比如 RabbitMQ/Kafka 等，一般消费端（Consumer）是直接去对接 Topic 的，然后 Consumer 自己又有个组的概念在配置中心去设置 offset，以此来决定是一起分享 Topic 的数据，还是每个人都接收同样的数据。在 Pulsar 的消费订阅模型里，添加了一个 Subscription 的逻辑，Subscription 的 Type 决定了消费是独享还是分享。

于是广播模式可以用不同 Subscription 独享的模式来实现，具体架构可以参照下图：

代码实现

1. Full-mesh 的形创建 Java 项目（比如：Springboot - 这个应该是相对简单的 IDE 集成开发组件）

画重点

• pulsar-client-api 和 tdmq-client 需要2.6.0
• tdmq-client 需要在腾讯的repo里才能拿到，需要使用介绍链接介绍的方式进行maven的配置（gradle方法类似）

• 介绍链接：https://cloud.tencent.com/document/product/1179/44914

  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">  4.0.0      org.springframework.boot    spring-boot-starter-parent    2.4.3         com.examble.demo  tdmq-demo  0.0.1-SNAPSHOT  tdmq-demo  demo project to test tdmq      1.8              org.springframework.boot      spring-boot-starter-web              com.tencent.tdmq      tdmq-client      2.6.0                  org.apache.pulsar      pulsar-client-api      2.6.0              org.springframework.boot      spring-boot-starter-test      test      
                    org.springframework.boot        spring-boot-maven-plugin            

2. 创建一个 Component 用来全局使用 Producer 和 Consumers


这里创建了1个 Producer 和3个拥有 exclusive subscription 的 consumers（广播模式 - 我们期待他们3个每次都收到一样的信息）

package com.example.demo.tdmq.instance;
import javax.annotation.PostConstruct;
import org.apache.pulsar.client.api.AuthenticationFactory;import org.apache.pulsar.client.api.Consumer;import org.apache.pulsar.client.api.Message;import org.apache.pulsar.client.api.MessageListener;import org.apache.pulsar.client.api.Producer;import org.apache.pulsar.client.api.PulsarClient;import org.apache.pulsar.client.api.PulsarClientException;import org.apache.pulsar.client.api.SubscriptionType;import org.springframework.beans.factory.config.ConfigurableBeanFactory;import org.springframework.context.annotation.Scope;import org.springframework.stereotype.Component;
@Component@Scope(ConfigurableBeanFactory.SCOPE_SINGLETON)public class Global {  PulsarClient client;  public Producer producer;  public Consumer consumer01;  public Consumer consumer02;  public Consumer consumer03;
  public Global() {
  }
  @PostConstruct  public void init() {    try {      client = PulsarClient.builder().serviceUrl("pulsar://:6000/")          .listenerName("custom://")          .authentication(AuthenticationFactory.token(              ""))          .build();      producer = client.newProducer().topic("persistent:////").create();      consumer01 = client.newConsumer().subscriptionType(SubscriptionType.Exclusive)          .topic("persistent:////")          .messageListener(new MessageListener() {
            /**             *              */            private static final long serialVersionUID = 1L;
            @Override            public void received(Consumer consumer, Message msg) {              System.out.println("Consumer01" + " - " + System.currentTimeMillis() + " - "                  + new String(msg.getData()));              try {                consumer.acknowledge(msg);              } catch (PulsarClientException e) {                // TODO Auto-generated catch block                e.printStackTrace();              }
            }          }).subscriptionName("my-subscription01").subscribe();      consumer02 = client.newConsumer().subscriptionType(SubscriptionType.Exclusive)          .topic("persistent:////")          .messageListener(new MessageListener() {
            /**             *              */            private static final long serialVersionUID = 1L;
            @Override            public void received(Consumer consumer, Message msg) {              System.out.println("Consumer02" + " - " + System.currentTimeMillis() + " - "                  + new String(msg.getData()));              try {                consumer.acknowledge(msg);              } catch (PulsarClientException e) {                // TODO Auto-generated catch block                e.printStackTrace();              }
            }          }).subscriptionName("my-subscription02").subscribe();      consumer03 = client.newConsumer().subscriptionType(SubscriptionType.Exclusive)          .topic("persistent:////")          .messageListener(new MessageListener() {
            /**             *              */            private static final long serialVersionUID = 1L;
            @Override            public void received(Consumer consumer, Message msg) {              System.out.println("Consumer03" + " - " + System.currentTimeMillis() + " - "                  + new String(msg.getData()));              try {                consumer.acknowledge(msg);              } catch (PulsarClientException e) {                // TODO Auto-generated catch block                e.printStackTrace();              }
            }          }).subscriptionName("my-subscription03").subscribe();
    } catch (PulsarClientException e) {      // TODO Auto-generated catch block      e.printStackTrace();    }  }
}

3. 最外层的测试代码和简单的 Message 模型

public class MessageModel {
  private String messageText = null;
  public String getMessageText() {    return messageText;  }
  public void setMessageText(String messageText) {    this.messageText = messageText;  }}

跑起来测试一下，果然3个一起接收一样的消息