郭昊,MSR56因ipsec抗重放功能导致不通问题经验案例
问题描述
总部MSR56与分支MSR36建立ipsec，能够协商ike/ipsec sa，私网业务不通。
过程分析
此类问题一般排查思路为：确认两端ipsec sa是否一致、确认ipsec sa计数是否正常、确认丢包位置。
1. 确认sa表项是否一致，一般关注以下几点：
a. sa对应接口与配置预期是否一致；
b. 感兴趣流(flow)是否完全对称；
c. 分支in spi与总部out spi是否一致，分支out spi与总部in spi是否一致。
本案例中以上参数均正确。
分支
总部
<branch>dis ipsec sa remote 10.0.0.1
Interface: Vlan-interface200
IPsec policy:
ADWAN-Ipsec-Vlan-interface200
Sequence number: 300
Mode: ISAKMP
-----------------------------
Tunnel id: 9
Encapsulation mode: tunnel
Perfect Forward Secrecy:
Inside VPN:
Extended Sequence Numbers enable: N
Traffic Flow Confidentiality enable: N
Transmitting entity: Initiator
Path MTU: 1420
IPsec over tcp: Disabled
IPsec over tcp mode: --
Tunnel:
local address/port: 10.0.0.2/15254
remote address/port: 10.0.0.1/4500
Flow:
sour addr: 2.2.2.2/255.255.255.255 port: 0 protocol: ip
dest addr: 1.1.1.1/255.255.255.255 port: 0 protocol: ip
[Inbound ESP SAs]
SPI: 532480371 (0x1fbd0173)
Connection ID: 43430709297181
Transform set: ESP-ENCRYPT-SM4-CBC ESP-AUTH-SM3
SA duration (kilobytes/sec): 1843200/3600
SA remaining duration (kilobytes/sec): 1788163/2646
Max received sequence-number: 0
Anti-replay check enable: N
Anti-replay window size:
Encapsulation used for NAT traversal: Y
Status: Active
[Outbound ESP SAs]
SPI: 1776146501 (0x69ddd845)
Connection ID: 42511586295840
Transform set: ESP-ENCRYPT-SM4-CBC ESP-AUTH-SM3
SA duration (kilobytes/sec): 1843200/3600
SA remaining duration (kilobytes/sec): 1806139/2646
Max sent sequence-number: 104667
Encapsulation used for NAT traversal: Y
Status: Active
<center>dis ipsec sa remote 10.0.0.2
Interface: Ten-GigabitEthernet3/0/0.4090
IPsec policy:
ADWAN-Ipsec-Ten-GigabitEthernet3/0/0.4090
Sequence number: 65535
Mode: Template
-----------------------------
Tunnel id: 112
Encapsulation mode: tunnel
Perfect Forward Secrecy:
Inside VPN:
Extended Sequence Numbers enable: N
Traffic Flow Confidentiality enable: N
Transmitting entity: Responder
Path MTU: 1420
IPsec over tcp: Disabled
IPsec over tcp mode: --
Tunnel:
local address/port: 10.0.0.1/4500
remote address/port: 10.0.0.2/15254
Flow:
sour addr: 1.1.1.1/255.255.255.255 port: 0 protocol: ip
dest addr: 2.2.2.2/255.255.255.255 port: 0 protocol: ip
[Inbound ESP SAs]
SPI: 1776146501 (0x69ddd845)
Connection ID: 20336670146704
Transform set: ESP-ENCRYPT-SM4-CBC ESP-AUTH-SM3
SA duration (kilobytes/sec): 1843200/3600
SA remaining duration (kilobytes/sec): 1810388/1553
Max received sequence-number: 73260
Anti-replay check enable: Y
Anti-replay window size: 64
Encapsulation used for NAT traversal: Y
Status: Active
[Outbound ESP SAs]
SPI: 532480371 (0x1fbd0173)
Connection ID: 20697447399774
Transform set: ESP-ENCRYPT-SM4-CBC ESP-AUTH-SM3
SA duration (kilobytes/sec): 1843200/3600
SA remaining duration (kilobytes/sec): 1788074/1553
Max sent sequence-number: 72805
Encapsulation used for NAT traversal: Y
Status: Active
2. 确认 ipsec sa正确后，需检查两端收发包情况。常用命令为dis ipsec statistics。在本案例中以分支设备查看情况为例。
<branch>dis ipsec statistics tunnel-id 9 //tunnel id 9为上述异常业务sa的id
IPsec packet statistics:
Received/sent packets: 34902/38706 //38706为发包计数
Received/sent bytes: 4711760/5152336
Dropped packets (received/sent): 0/0
Dropped packets statistics
No available SA: 0
Wrong SA: 0
Invalid length: 0
Authentication failure: 0
Encapsulation failure: 0
Decapsulation failure: 0
Replayed packets: 0
ACL check failure: 0
MTU check failure: 0
Loopback limit exceeded: 0
Crypto speed limit exceeded: 0
<branch>repeat 1 delay 5
<branch>dis ipsec statistics tunnel-id 9
IPsec packet statistics:
Received/sent packets: 34907/38713
Received/sent bytes: 4712480/5153280
Dropped packets (received/sent): 0/0
Dropped packets statistics
No available SA: 0
Wrong SA: 0
Invalid length: 0
Authentication failure: 0
Encapsulation failure: 0
Decapsulation failure: 0
Replayed packets: 0
ACL check failure: 0
MTU check failure: 0
Loopback limit exceeded: 0
Crypto speed limit exceeded: 0
<branch>dis ipsec statistics tunnel-id 9
IPsec packet statistics:
Received/sent packets: 34923/38728 //无人为操作时，间隔5s发包15个
Received/sent bytes: 4714480/5155168
Dropped packets (received/sent): 0/0
Dropped packets statistics
No available SA: 0
Wrong SA: 0
Invalid length: 0
Authentication failure: 0
Encapsulation failure: 0
Decapsulation failure: 0
Replayed packets: 0
ACL check failure: 0
MTU check failure: 0
Loopback limit exceeded: 0
Crypto speed limit exceeded: 0
<branch>ping -c 100000 -m 10 -t 10 -a 2.2.2.2 1.1.1.1
//通过ping构造流量，因通过上述命令判断该sa存在一定背景流量，此处ping测试需通过-m和-t参数将发包间隔和等待超时间隔缩短。上述参数发包频率大约50pps。
Ping 1.1.1.1 (1.1.1.1): 56 data bytes, press CTRL_C to break
Request time out
Request time out
Request time out
Request time out
Request time out
……
--- Ping statistics for 1.1.1.1 ---
236 packet(s) transmitted, 0 packet(s) received, 100.0% packet loss
<branch>dis ipsec statistics tunnel-id 9
IPsec packet statistics:
Received/sent packets: 34923/39008 //ping执行约5s，发包计数增长约300，远大于背景流量速率，说明测试流量匹配到此处计数，ipsec发包正常。如此处没有计数，说明流量走到其他功能或进程，需检查nat acl、packet filter、qos policy等配置。
Received/sent bytes: 4714480/5166642
Dropped packets (received/sent): 0/0
Dropped packets statistics
No available SA: 0
Wrong SA: 0
Invalid length: 0
Authentication failure: 0
Encapsulation failure: 0
Decapsulation failure: 0
Replayed packets: 0
ACL check failure: 0
MTU check failure: 0
Loopback limit exceeded: 0
Crypto speed limit exceeded: 0
同样，在总部侧检查ipsec sa计数。
[center]dis ipsec statistics tunnel-id 112
IPsec packet statistics:
Received/sent packets: 0/195 //此处收计数为0，说明流量没有正常被sa处理
Received/sent bytes: 0/21840
Dropped packets (received/sent): 1028/0
Dropped packets statistics
No available SA: 0
Wrong SA: 0
Invalid length: 0
Authentication failure: 0
Encapsulation failure: 0
Decapsulation failure: 0
Replayed packets: 1028 //该计数异常，表示总部设备判断报文为重放包，会丢弃
ACL check failure: 0
MTU check failure: 0
Loopback limit exceeded: 0
Crypto speed limit exceeded: 0
[center]dis ipsec statistics tunnel-id 112
IPsec packet statistics:
Received/sent packets: 0/196
Received/sent bytes: 0/21952
Dropped packets (received/sent): 1123/0
Dropped packets statistics
No available SA: 0
Wrong SA: 0
Invalid length: 0
Authentication failure: 0
Encapsulation failure: 0
Decapsulation failure: 0
Replayed packets: 1123 //连续两次查看，重放报文统计有增长，增长数量与分支ping出的报文数量基本一致，可以判断是该功能导致不通。
ACL check failure: 0
MTU check failure: 0
Loopback limit exceeded: 0
Crypto speed limit exceeded: 0
MSR设备的ipsec抗重放功能默认开启。一般出现此类计数是因为设备收到ipsec报文后，根据报文ID判断报文是重放包，设备认为解析此类报文无实际意义且占用性能，因此丢弃。出现此类报文可能与发包设备封装ID行为、中间设备NAT修改报文头行为、报文因线路拥塞等原因收发乱序等相关。可以undo ipsec anti-replay check手动关闭抗重放检测。
3. 如第二步双向检查发现某方向无收包，且没有错误计数，一般需要按照丢包问题继续分析。常用丢包问题定位手段为流量统计和抓包。对 MSR设备可以匹配 esp协议号 50进行流量统计，存在背景流量的情况下流统比较困难。端口镜像无法根据加密后报文判断原始报文特征，可以尝试 ping特定长度报文作为特征辅助筛选。本案例不对流统和抓包具体条件做描述。
解决方法
关闭ipsec抗重放检测解决。 undo ipsec anti-replay check
内容来源：知了社区，基于知识共享署名-相同方式共享3.0中国大陆许可协议CRM论坛（CRMbbs.com）——一个让用户更懂CRM的垂直性行业内容平台,CRM论坛致力于互联网、客户管理、销售管理、SCRM私域流量内容输出5年。 如果您有好的内容，欢迎向我们投稿，共建CRM多元化生态体系，创建CRM客户管理一体化生态解决方案。,MSR56因ipsec抗重放功能导致不通问题经验案例
