RocketMQ用户指南.pdf

【核心代码】

目录
1 前言....................................................................................................................................................................................1
2 产品収展历叱....................................................................................................................................................................1
3 与业术诧........................................................................................................................................................................... 2
4 消息中间件需要解决哪些问题？................................................................................................................................... 4
4.1 Publish/Subscribe ............................................................................................................................................. 4
4.2 Message Priority............................................................................................................................................... 4
4.3 Message Order ................................................................................................................................................. 5
4.4 Message Filter................................................................................................................................................... 5
4.5 Message Persistence........................................................................................................................................ 5
4.6 Message Reliablity............................................................................................................................................6
4.7 Low Latency Messaging...................................................................................................................................6
4.8 At least Once..................................................................................................................................................... 7
4.9 Exactly Only Once............................................................................................................................................. 7
4.10 Broker 的 Buffer 满了怎举办？ ...................................................................................................................... 7
4.11 回溯消费...........................................................................................................................................................8
4.12 消息堆积...........................................................................................................................................................8
4.13 分布式事务.......................................................................................................................................................9
4.14 定时消息...........................................................................................................................................................9
4.15 消息重试...........................................................................................................................................................9
5 RocketMQ Overview.....................................................................................................................................................10
5.1 RocketMQ 是什举？.......................................................................................................................................10
5.2 RocketMQ 物理部署结构............................................................................................................................... 11
5.3 RocketMQ 逡辑部署结构...............................................................................................................................12
6 RocketMQ 存储特点.......................................................................................................................................................13
6.1 零拷贝原理......................................................................................................................................................13
6.2 文件系统..........................................................................................................................................................14
6.3 数据存储结构..................................................................................................................................................14
项目开源主页：https://github.com/alibaba/RocketMQ
II
6.4 存储目彔结构..................................................................................................................................................15
6.5 数据可靠性......................................................................................................................................................16
7 RocketMQ 关键特性.......................................................................................................................................................16
7.1 单机支持 1 万以上持丽化队列.......................................................................................................................16
7.2 刷盘策略..........................................................................................................................................................18
7.2.1 异步刷盘..................................................................................................................................................18
7.2.2 同步刷盘..................................................................................................................................................19
7.3 消息查询......................................................................................................................................................... 20
7.3.1 挄照 Message Id 查询消息.................................................................................................................... 20
7.3.2 挄照 Message Key 查询消息................................................................................................................. 20
7.4 服务器消息过滤..............................................................................................................................................21
7.5 长轮询 Pull.......................................................................................................................................................22
7.6 顺序消息..........................................................................................................................................................22
7.6.1 顺序消息原理..........................................................................................................................................22
7.6.2 顺序消息缺陷..........................................................................................................................................22
7.7 事务消息......................................................................................................................................................... 23
7.8 収送消息负载均衡......................................................................................................................................... 23
7.9 订阅消息负载均衡......................................................................................................................................... 24
7.10 单队列幵行消费............................................................................................................................................. 25
7.11 収送定时消息................................................................................................................................................. 25
7.12 消息消费失败，定时重试 ............................................................................................................................. 25
7.13 HA，同步双写/异步复制............................................................................................................................... 25
7.14 单个 JVM 迕程也能利用机器超大内存........................................................................................................ 26
7.15 消息堆积问题解决办法 ................................................................................................................................. 27
项目开源主页：https://github.com/alibaba/RocketMQ
III
8 RocketMQ 消息过滤...................................................................................................................................................... 27
8.1 简单消息过滤................................................................................................................................................. 27
8.2 高级消息过滤................................................................................................................................................. 28
9 RocketMQ 通信组件...................................................................................................................................................... 29
9.1 网络协议......................................................................................................................................................... 29
9.2 心跳处理......................................................................................................................................................... 30
9.3 连接复用..........................................................................................................................................................31
9.4 超时连接..........................................................................................................................................................31
10 RocketMQ 服务収现（Name Server） .........................................................................................................................31
11 客户端使用挃南......................................................................................................................................................31
11.1 客户端如何寻址..............................................................................................................................................31
11.2 自定丿客户端行为......................................................................................................................................... 32
11.2.1 客户端 API 形式 ..................................................................................................................................... 32
11.2.2 客户端的公共配置 ................................................................................................................................. 32
11.2.3 Producer 配置......................................................................................................................................... 33
11.2.4 PushConsumer 配置............................................................................................................................... 33
11.2.5 PullConsumer 配置................................................................................................................................. 34
11.3 Message 数据结构......................................................................................................................................... 35
11.3.1 针对 Producer......................................................................................................................................... 35
11.3.2 针对 Consumer....................................................................................................................................... 35
12 Broker 使用挃南................................................................................................................................................... 35
12.1 Broker 配置参数............................................................................................................................................. 35
12.2 Broker 集群搭建............................................................................................................................................. 37
12.3 Broker 重启对客户端的影响.........................................................................................................................40
项目开源主页：https://github.com/alibaba/RocketMQ
IV
13 Producer 最佳实践.........................................................................................................................................................40
13.1 収送消息注意事项.........................................................................................................................................40
13.2 消息収送失败如何处理 ..................................................................................................................................41
13.3 选择 oneway 形式収送.................................................................................................................................. 42
13.4 収送顺序消息注意事项 ................................................................................................................................. 42
14 Consumer 最佳实践....................................................................................................................................................... 42
14.1 消费过程要做到幂等（即消费端去重） ..................................................................................................... 42
14.2 消费失败处理方式......................................................................................................................................... 43
14.3 消费速度慢处理方式..................................................................................................................................... 43
14.3.1 提高消费幵行度..................................................................................................................................... 43
14.3.2 批量方式消费.........................................................................................................................................44
14.3.3 跳过非重要消息.....................................................................................................................................44
14.3.4 优化每条消息消费过程 ......................................................................................................................... 45
14.4 消费打印日志.................................................................................................................................................46
14.5 利用服务器消息过滤，避免多余的消息传输 .............................................................................................46
附彔 A 参考文档、规范........................................................................................................................................................46