1、消息丢失的问题

  1. 当你系统需要保证百分百消息不丢失,你可以使用生产者每发送一个消息,Broker 同步返回一个消息发送成功的反馈消息。
  2. 即每发送一个消息,同步落盘后才返回生产者消息发送成功,即生产者收到确认发送消息成功,才继续之后操作,这样只要生产者得到了消息发送生成的返回,事后除了硬盘损坏,都可以保证不会消息丢失。

2、同步落盘怎么才能快

  1. 使用 FileChannel + DirectBuffer 池,使用堆外内存,加快内存拷贝
  2. 使用数据和索引分离,当消息需要写入时,使用 commitlog 文件顺序写,当需要定位某个消息时,查询index 文件来定位,从而减少文件IO随机读写的性能损耗

3、消息堆积的问题

  1. 后台定时任务每隔72小时,删除旧的没有使用过的消息信息
  2. 根据不同的业务实现不同的丢弃任务,选择不同的策略淘汰任务,例如FIFO/LRU等
  3. 消息定时转移,或者对某些重要的 TAG 型(支付型)消息真正落库

4、定时消息的实现

实现定时消息的原理是:创建特定时间精度的 MessageQueue,例如生产者需要定时1s之后被消费者消费,你只需要将此消息发送到特定的 Topic,例如:MessageQueue-1 表示这个 MessageQueue 里面的消息都会延迟一秒被消费,然后 Broker 会在 1s 后发送到消费者消费此消息,使用 newSingleThreadScheduledExecutor 实现。

5、顺序消息的实现

  1. 与定时消息同原理,生产者生产消息时指定特定的 MessageQueue ,消费者消费消息时,消费特定的 MessageQueue,其实单机版的消息中心在一个 MessageQueue 就天然支持了顺序消息
  2. 注意:同一个 MessageQueue 保证里面的消息是顺序消费的前提是:消费者是串行的消费该 MessageQueue,因为就算 MessageQueue 是顺序的,但是当并行消费时,还是会有顺序问题,但是串行消费也同时引入了两个问题:

引入锁来实现串行
前一个消费阻塞时后面都会被阻塞

6、消息重复发送的避免

  • 因为在网络延迟的情况下,消息重复发送的问题不可避免的发生,如果非要实现消息不可重复发送,那基本太难,因为网络环境无法预知,还会使程序复杂度加大,因此默认允许消息重复发送。
  • 最简单的解决方案是每条消费记录有个消费状态字段,根据这个消费状态字段来是否消费或者使用一个集中式的表,来存储所有消息的消费状态,从而避免重复消费
  • 采用消息幂等消费,可以查询关于消息幂等消费的解决方案。

7、广播消费与集群消费

  • 广播消费,订阅该 Topic 的消息者们都会消费每个消息
  • 集群消费,订阅该 Topic 的消息者们只会有一个去消费某个消息

广播消费与集群消费在消息落盘区别:具体表现在消息消费进度的保存上。

  • 广播消费,由于每个消费者都独立的去消费每个消息,因此每个消费者各自保存自己的消息消费进度。
  • 集群消费下,订阅了某个 Topic,而旗下又有多个 MessageQueue,每个消费者都可能会去消费不同的 MessageQueue,因此总体的消费进度保存在 Broker 上集中的管理
  1. Thanks for your entire labor on this web site. My mother take interest in participating in research and it is simple to grasp why. My spouse and i notice all regarding the powerful tactic you produce useful thoughts via your website and in addition recommend response from the others on this subject then our own princess is truly studying a whole lot. Take pleasure in the rest of the year. You have been performing a terrific job.

  2. I’m commenting to let you be aware of of the amazing discovery my child developed browsing the blog. She came to find a good number of details, including how it is like to have a great coaching character to get a number of people clearly completely grasp a number of advanced issues. You actually did more than her desires. I appreciate you for churning out such beneficial, dependable, edifying and cool guidance on that topic to Emily.

  3. I simply wanted to thank you so much once more. I do not know what I would’ve taken care of without these suggestions revealed by you over that subject. It was an absolute horrifying problem in my position, nevertheless spending time with this specialized style you resolved the issue took me to leap with happiness. I will be happy for your service and then sincerely hope you realize what an amazing job you have been providing teaching some other people using your webblog. Most likely you haven’t got to know any of us.

  4. I am just writing to make you understand what a perfect experience my wife’s girl obtained using your webblog. She noticed so many pieces, which included what it is like to possess an excellent helping spirit to let others with no trouble learn about specific specialized things. You truly surpassed readers’ expected results. Thank you for providing these priceless, trusted, informative as well as fun tips about this topic to Lizeth.

Leave a Reply

邮箱地址不会被公开。 必填项已用*标注