我的开源
k8s中的负载均衡的实现
背景
k8s中通过Service将多个Pods的服务，通过统一的端口暴露出去，如下：
# kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-bootcamp NodePort 10.108.63.220 <none> 8080:31098/TCP 21h
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kubernetes-bootcamp-7dc9765bf6-2hbn5 1/1 Running 0 21h 10.244.1.3 172-19-120-201 <none> <none>
kubernetes-bootcamp-7dc9765bf6-4vs55 1/1 Running 0 21h 10.244.3.3 172-19-120-203 <none> <none>
kubernetes-bootcamp-7dc9765bf6-dg2xl 1/1 Running 0 21h 10.244.2.4 172-19-120-202 <none> <none>
kubernetes-bootcamp-7dc9765bf6-hlbv8 1/1 Running 0 21h 10.244.3.2 172-19-120-203 <none> <none>
kubernetes-bootcamp-7dc9765bf6-ljrbc 1/1 Running 0 21h 10.244.1.4 172-19-120-201 <none> <none>
kubernetes-bootcamp-7dc9765bf6-qkmx6 1/1 Running 0 21h 10.244.2.3 172-19-120-202 <none> <none>
用户对master上的31098端口进行请求，最终由pod上的服务进行处理，其中的网络链路是什么样子的呢？
实验
本地发布，在pod所在的宿主机上抓包。
本地抓包
##
T 192.168.31.12:58647 -> 47.xx.xx.xx:31098 [AP] #41
GET / HTTP/1.1.
Host: 47.xx.xx.xx:31098.
Connection: Keep-Alive.
User-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_151).
Accept-Encoding: gzip,deflate.
.
#
T 47.xx.xx.xx:31098 -> 192.168.31.12:58647 [AP] #42
HTTP/1.1 200 OK.
Content-Type: text/plain.
Date: Sun, 11 Aug 2019 15:11:56 GMT.
Connection: keep-alive.
Transfer-Encoding: chunked.
.
29.
Hello Kubernetes bootcamp! | Running on: .
master抓包
23:14:09.766800 IP 172.19.120.198.53192 > 172.19.120.201.8472: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1078:1232, ack 1681, win 2048, options [nop,nop,TS val 1202891188 ecr 121506908], length 154: HTTP: GET / HTTP/1.1
23:14:09.767226 IP 172.19.120.201.37779 > 172.19.120.198.8472: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.1.4.8080 > 10.244.0.0.58647: Flags [P.], seq 1681:1862, ack 1232, win 1400, options [nop,nop,TS val 121507945 ecr 1202891188], length 181: HTTP: HTTP/1.1 200 OK
再抓一个包，看看长连接的样子，端口不变。
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 154:308, ack 241, win 2048, options [nop,nop,TS val 1203161074 ecr 121778628], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 308:462, ack 481, win 2048, options [nop,nop,TS val 1203162141 ecr 121779706], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 462:616, ack 721, win 2048, options [nop,nop,TS val 1203163209 ecr 121780781], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 616:770, ack 961, win 2048, options [nop,nop,TS val 1203164279 ecr 121781857], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 770:924, ack 1201, win 2048, options [nop,nop,TS val 1203165348 ecr 121782934], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 924:1078, ack 1441, win 2048, options [nop,nop,TS val 1203166418 ecr 121784009], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1078:1232, ack 1681, win 2048, options [nop,nop,TS val 1203167484 ecr 121785082], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1232:1386, ack 1921, win 2048, options [nop,nop,TS val 1203168554 ecr 121786157], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1386:1540, ack 2161, win 2048, options [nop,nop,TS val 1203169623 ecr 121787232], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1540:1694, ack 2401, win 2048, options [nop,nop,TS val 1203170690 ecr 121788308], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1694:1848, ack 2641, win 2048, options [nop,nop,TS val 1203171761 ecr 121789382], length 154: HTTP: GET / HTTP/1.1
node抓包
23:13:33.200110 IP 172.19.120.198.53192 > 172.19.120.201.8472: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1694:1848, ack 2641, win 2048, options [nop,nop,TS val 1202854902 ecr 121470339], length 154: HTTP: GET / HTTP/1.1
23:13:33.200540 IP 172.19.120.201.37779 > 172.19.120.198.8472: OTV, flags [I] (0x08), overlay 0, instance 1
IP 10.244.1.4.8080 > 10.244.0.0.58647: Flags [P.], seq 2641:2822, ack 1848, win 1400, options [nop,nop,TS val 121471379 ecr 1202854902], length 181: HTTP: HTTP/1.1 200 OK
再抓一个包，表明是长连接，端口不变。
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 154:308, ack 241, win 2048, options [nop,nop,TS val 1203075562 ecr 121692482], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 308:462, ack 481, win 2048, options [nop,nop,TS val 1203076635 ecr 121693557], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 462:616, ack 721, win 2048, options [nop,nop,TS val 1203077709 ecr 121694635], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 616:770, ack 961, win 2048, options [nop,nop,TS val 1203078772 ecr 121695711], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 770:924, ack 1201, win 2048, options [nop,nop,TS val 1203079842 ecr 121696783], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 924:1078, ack 1441, win 2048, options [nop,nop,TS val 1203080909 ecr 121697858], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1078:1232, ack 1681, win 2048, options [nop,nop,TS val 1203081979 ecr 121698931], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1232:1386, ack 1921, win 2048, options [nop,nop,TS val 1203083052 ecr 121700010], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1386:1540, ack 2161, win 2048, options [nop,nop,TS val 1203084123 ecr 121701086], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1540:1694, ack 2401, win 2048, options [nop,nop,TS val 1203085193 ecr 121702160], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1694:1848, ack 2641, win 2048, options [nop,nop,TS val 1203086266 ecr 121703236], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 1848:2002, ack 2881, win 2048, options [nop,nop,TS val 1203087331 ecr 121704312], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 2002:2156, ack 3121, win 2048, options [nop,nop,TS val 1203088396 ecr 121705385], length 154: HTTP: GET / HTTP/1.1
IP 10.244.0.0.58647 > 10.244.1.4.8080: Flags [P.], seq 2156:2310, ack 3361, win 2048, options [nop,nop,TS val 1203089464 ecr 121706460], length 154: HTTP: GET / HTTP/1.1