首页 > 代码库 > CNI portmap插件实现源码分析

CNI portmap插件实现源码分析

DNAT创建的iptables规则如下:(重写目的IP和端口)

PREROUTING, OUTPUT: --dst-type local -j CNI-HOSTPORT_DNAT    // PREROUTING和OUTPUT链中目的地址类型为local的跳转至CNI-HOSTPORT-DNAT进行处理

CNI-HOSTPORT-DNAT: -j CNI-DN-abcd123    // 进入相应容器的chain进行处理

CNI-DN-abcd123: -p tcp --dport 8080 -j DNAT --to-destination 192.0.2.33:80

CNI-DN-abcd123: -p tcp --dport 8081 -j DNAT ....

 

// plugins/plugins/meta/portmap/main.go

1、func cmdAdd(args *skel.CmdArgs) error

1、首先调用netConf, err := parseConfig(args.StdinData, args.IfName)对配置进行解析

2、因为portmap只能作为chained plugin存在,因此当netConf.PrevResult为nil时,报错

3、当len(netConf.RuntimeConfig.PortMaps)为0时,说明没有配置portmap,直接返回PrevResult

4、当netConf.ConfIPv4不为nil时,调用forwardPorts(netConf, netConf.ContIPv4)

5、当netConf.ConfIPv6不为nil时,调用forwardPorts(netConf, netConf.ConfIPv6)

 

PortMapConf的结构如下所示:

type PortMapConf struct {
  types.NetConf
  SNAT  *bool
  ConditionsV4 *[]string
  Conditions V6 *[]string
  RuntimeConfig struct {
    PortMaps  [ ]PortMapEntry
  }

  RawPrevResult map[string]interface{}
  PrevResult  *current.Result
  

  // These are fields parsed out of the config or the environment;
  // included here for convenience
  ContainerID  string
  ContIPv4    net.IP
  ContIPv6     net.IP
}

 

PortMapEntry的结构如下所示:

type PortMapEntry struct {
  HostPort     int
  ContainerPort  int
  Protocol      string
  HostIP      string
}

  

// plugins/plugins/meta/portmap/main.go

2、func parseConfig(stdin []byte, ifName string) (*PortMapConf, error)

1、首先创建conf := PortMapConf{},并调用json.Unmarshal(stdin, &conf)进行解析

2、如果conf.RawPrevResult不为nil,则先将其转换为conf.PrevResult

3、如果解析之后conf.SNAT为nil,则设置conf.SNAT赋值为true

4、遍历conf.RuntimeConfig.PortMaps,对每个pm进行检测,如果pm.ContainerPort <= 0或者pm.HostPort<=0,则报错

5、如果conf.PrevResult不为nil,则对conf.PrevResult.IPs进行遍历,将conf.ContIPv4和conf.ContIPv6分别设置为第一个遇到的container interface的地址

 

// plugins/plugins/meta/portmap/portmap.go

3、func forwardPorts(config *PortMapConf, containerIP net.IP) error

1、当containerIP为IPv4时,调用ipt, err = iptables.NewWithProtocol(iptables.ProtocolIPv4)和conditions = config.ConditionsV4获取iptable

2、调用toplevelDnatChain := genToplevelDnatChain()和toplevelDnatChain.setup(ipt, nil)创建top level chain,

3、调用dnatChain := genDnatChain(config.Name, config.ContainerID, conditions)创建dnat chain,并调用_ = dnatChain.teardown(ipt)进行清理,如果有冲突的话

4、调用dnatRules := dnatRules(config.RuntimeConfig.PortMaps, containerIP)和dnatChain.setup(ipt, dnatRules)在dnat chain中创建规则

5、如果config.SNAT为真,且不是地址不是IPv6,则首先调用toplevelSnatChain := genToplevelSnatChain(isV6)和toplevelSnatChain.setup(ipt, nil)创建top level snat chain

6、调用snatChain := genSnatChain(config.Name, config.ContainerID)和_ = snatChain.teardown(ipt)获取相应的snat chain,如果chain已存在,则进行清理

7、调用snatRules := snatRules(config.RuntimeConfig.PortMaps, containerIP)和snatChain.setup(ipt, snatRules)创建规则

8、如果isV6为false,则设置host interface的route_localnet bit,从而让127/8能cross a routing boundary,先调用hostIfName := getRoutableHostIF(containerIP)

如果hostIfName不为"",则调用enableLocalnetRouting(hostIfName)

 

// plugins/plugins/meta/portmap/portmap.go

4、func genToplevelDnatChain() chain

该函数仅仅返回一个chain{}结构,该chain为top-levle summary chain,其他所有的dnat chain都通过它进行调用,具体代码如下:

return chain{
  table:  "nat",
  name:  TopLevelDNATChainName,
  entryRule:  [ ]string{
    "-m", "addrtype",
    "--dst-type", "LOCAL",
  },
  entryChains: []string{"PREROUTING", "OUTPUT"},

}

  

// plugins/plugins/meta/portmap/chain.go

5、func (c *chain) setup(ipt *iptables.IPTables, rules [][]string) error

1、调用exists, err := chainExists(ipt, c.table, c.name)判断chain是否存在,不在则调用ipt.NewChain(c.table, c.name)进行创建

2、倒序遍历rules,调用prependUnique(ipt, c.table, c.name, rules[i])将rule添加至chain中

3、调用entryRule := append(c.entryRule, "-j", c.name),并遍历c.entryChains,调用prependUnique(ipt, c.table, entryChain, entryRule)将rule加入其他chain中

 

// plugins/plugins/meta/portmap/chain.go

6、func genDnatChain(netName, containerID string, conditions *[]string) chain

1、首先调用name := formatChainName("DN-", netName, containerID)创建新的dnat chain名字

2、调用comment := fmt.Sprintf(...)创建comment

3、创建的chain如下所示:

ch := chain{
  table:  "nat",
  name:  name,
  entryRule:  []string{
    "-m", "comment",
    "--comment", comment
  },
  entryChains:  []string{TopLevelDNATChainName},
}

4、如果conditions不为nil,且len(*conditions)不为0,则调用ch.entryRule = append(ch.entryRule, *conditions...)

 

CNI portmap插件实现源码分析