iptables/nftables(nft)

1. netfilter才是防火墙真正的安全框架(framework),netfilter位于kernal space; iptables其实是一个命令行工具，位于user space,我们用这个工具操作真正的框架. netfilter/iptables组成Linux平台下的packet filtering firewall,完成packet filtering,packet redirection,NAT(网络地址转换)等功能
   

2. Below are the default chains that iptables uses:

   1. prerouting - packets are checked against this chain before any decisions are made as to where they are going

   2. input - packets that are to be delivered to the server go through this chain

   3. forward - packets to be forwarded on to another system go through this chain

   4. output - packets leaving the server go through this chain

   5. postrouting - packets leaving the server go through this chain after the forward and output chains

3. 把具有相同功能的规则的集合叫做"table" (所有规则都保存在这4张表中)，these tables classify rules according to the type of decisions they are used to make. Within each iptables table, rules are further organized within separate "chains". While tables are defined by the general aim of the rules they hold, the built-in chains represent the netfilter hooks which trigger them. Chains basically determine when rules will be evaluated.

   1. filter table: 负责过滤功能，防火墙，内核模块:iptables_filter
   2. NAT table:网络地址转换功能；内核功能: iptable_nat
   3. mangle table: 拆解报文，做出修改，并重新封装的功能:iptable_mangle

   4.raw table: 关闭nat表上启用的连接追踪机制:iptable_raw

4. iptables为我们定义了4张表，当他们处于同一条"Chain",执行的优先级如下; 优先级次序(由高到低)raw -->mangle -->nat --->filter

5. iptables Command Examples:

   1. Getting a list of the currently used firewall rules : iptables -L/iptables -L -v -n

   2. iptables --table nat --list

   3. Show the details of a specific chain : iptables -L INPUT -n --line-numbers

   4. Get a dump of all the commands that would be required to be typed to implement the current firewall ruleset: iptables-save

6. Rules are placed within a specific chain of a specific table. As each chain is called, the packet in question will be checked against each rule within the chain in order. Each rule has a matching component and an action component.

7. Another thing to keep in mind is that the order of the rules in each chain matters. A packet must not come across a more general rule that it matches if it is meant to match a more specific rule.

   Because of this, rules near the top of a chain should have a higher level of specificity than rules at the bottom. You should match specific cases first, and then provide more general rules to match broader patterns. If a packet falls through the entire chain (doesn't match any rules), it will hit the_most_general rule, the default policy (the default policy is also a target).

8. NAT hairpin功能用于满足位于内网侧的用户之间或内网侧的用户与服务器之间通过NAT地址进行访问的需求。开启NAT hairpin的内网侧接口上会对报文同时进行源地址和目的地址的转换。hairpin-masquerade tricks（地址伪装）

9. -A KUBE-MARK-MASQ -j MARK --set-xmark 0x00004000/0x00004000 : 表示对进入KUBE-MARK-MASQ 的包在内核中设置关联的标签：0x00004000/0x00004000 。mark值不是包本身的一部分，

   而是在包穿越计算机的过程中由内核分配的和它相关联的一个字段。它可能被用来改变包的传输路径或过滤。mark值只在本机有意义！ This table can also place an internal kernel "mark" on the packet for further processing in other tables and by other networking tools. This mark does not touch the actual packet, but adds the mark to the kernel's representation of the packet.

10. Rule -A KUBE-SEP-UH5EYFQKYB24RWKN -s 10.244.1.69/32 -j KUBE-MARK-MASQ and -A KUBE-SEP-5MXPM55VLN7O52FQ -s 10.244.1.70/32 -j KUBE-MARK-MASQ are used for hairpin NAT.

    下面三条规则的意思是：当源IP不是-s 172.16.128.0/17，进入-j KUBE-MARK-MASQ，打上标记0x4000/0x4000，在随后的POSTROUTING阶段，带有0x4000/0x4000标记的报文会被SNAT（-j MASQUERADE）。

    -A KUBE-SERVICES ! -s 172.16.128.0/17 -m comment --comment "Kubernetes service cluster ip + port for masquerade purpose" -m set --match-set KUBE-CLUSTER-IP dst,dst -j KUBE-MARK-MASQ
    -A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
    -A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE
    

11. conntrack - command line interface for netfilter connection tracking

ipset command(IPVS)/ipvsadm

1. IPVS is for load balancing and it can’t handle other workarounds in kube-proxy, e.g. packet filtering, hairpin-masquerade tricks, SNAT -- IPVS用于负载平衡，它无法处理Kube代理中的其他解决方案，例如数据包过滤、发夹伪装技巧、snat

2. List Currently Loaded Modules : lsmod | less
   Install New modules into Linux Kernel: modprobe 'module name'
   ipvs是需要使用ipvs内核模块，先保证有这些内核模块
   1>ip_vs_sh
   2>ip_vs_wrr
   3>ip_vs_rr
   4>ip_vs
   5>nf_conntrack

3. Linux Virtual Server administration : ipvsadm / administration tool for IP sets: ipset
   ipset是什么？

   ipset是iptables的扩展，它可以创建一个集合，这个集合内容可以是ip地址，ip网段，端口等，然后iptables可以直接添加规则对这个集合进行操作。这样的好处在于不用针对每个ip或每个端口添加单独的规则，可以减少大量iptables规则添加，减少性能损耗。比如我们要禁止上万个IP访问我们的服务器，用iptables的话，你需要添加一条条规则，这样会在iptables中生成大量规则造成性能损耗，但通过ipset，可以将地址直接加入到ipset集合中，然后iptables可以添加规则对这个ipset进行操作。

   为什么用ipset？

   因为单独操作iptables就回到iptables模式的问题了，一但Kubernetes集群中service过多，会产生大量iptables规则，造成性能损耗，但用ipset可以配置集合将对象添加进去，这样可以保证即使我有在多的service和pod，但iptables规则是固定不变的。

4. ipvsadm -ln

5. When creating a ClusterIP type Service, IPVS proxier will do the following three things:

   1. Make sure a dummy interface exists in the node, defaults to kube-ipvs0
   2. Bind Service IP addresses to the dummy interface
   3. Create IPVS virtual servers for each Service IP address respectively

6. ipvs 会使用 iptables 进行包过滤、SNAT、masquared(伪装)。具体来说，ipvs 将使用ipset来存储需要DROP或masquared的流量的源或目标地址，以确保 iptables 规则的数量是恒定的，这样我们就不需要关心我们有多少服务了

Blogs

1. iptables系列文章(Great)
2. iptables --Linux man page
3. iptables-extensions (netfilter)
4. Hairpin NAT
5. A Deep Dive into Iptables and Netfilter Architecture
6. An In-Depth Guide to iptables, the Linux Firewall
7. An Introduction To Iptables: The Linux Firewall--Part 1
8. Advanced Firewall Configurations with ipset
9. calico iptables详解
10. calico的iptables规则

******************************************************

IPVS

1. LVS原理详解
2. LVS: Running a firewall on the director: Interaction between LVS and netfilter (iptables)
3. ipvs 基本介绍 (阳明的博客)
4. IPVS-Based In-Cluster Load Balancing Deep Dive

IPTables

1. Layman’s iptables 101

2. IPTables packet traverse map Great