Evaluation of other CNIs (e.g., flannel)

tom commented

2022-11-23 16:54:47 +01:00

Owner

Copy link

It should have the following features:

IPv4 and IPv6 support
custom pod subnet support

==> Flannel

IPv4 only test

IPv6 compatibility
Flannel needs all default routes on the same interface for dual stuck functions.

~~create virtual interface~~ (not necessary)
adjust default routes
split & reroute traffic (v4 & v6)
~~change firewall rules~~ (not necessary)
reroute v6 traffic with second routing table
(change the default v6 route to wg interface instead of v4 route to main interface to avoid "wrong" packets on internet)

=x> Flannel is not optimal for multiple "default" interfaces

==> Calico

adjust configuration for our cluster
~~~solve service reachability problems~~ (not solved)
~~define network policies?~~ (removed for testing purposes...)
~~pod to pod and service communication~~ (works for existing pods)

=x> Calico triggers some problems with new pod creation

==> Cilium

configuration for our cluster
pod to pod and service communication

=x> Cilium triggers reboot problems like earlier the IPTables rules from k8s

However, I am the stupidest person... Flannel works fine if you configure it correctly (force start on wireguard interface).

==> Flannel Test 2

IPv4 only test
Dual-Stack test
IP assignment

==> Flannel Test 2 now works smoothly 🙌

==> Final Tests

IPv6 egress test
Dual-Stack egress test
Drop private IPs before forward to internet

It should have the following features: - [x] IPv4 and IPv6 support - [x] custom pod subnet support ==> **Flannel** - [x] IPv4 only test IPv6 compatibility Flannel needs all default routes on the same interface for dual stuck functions. - [x] ~~create virtual interface~~ (not necessary) - [x] adjust default routes - [x] split & reroute traffic (v4 & v6) - [x] ~~change firewall rules~~ (not necessary) - [x] reroute v6 traffic with second routing table (change the default v6 route to wg interface instead of v4 route to main interface to avoid "wrong" packets on internet) =x> **Flannel** is not optimal for multiple "default" interfaces ==> **Calico** - [x] adjust configuration for our cluster - [x] ~~~solve service reachability problems~~ (not solved) - [x] ~~define network policies?~~ (removed for testing purposes...) - [x] ~~pod to pod and service communication~~ (works for existing pods) =x> **Calico** triggers some problems with new pod creation ==> **Cilium** - [x] configuration for our cluster - [x] pod to pod and service communication =x> **Cilium** triggers reboot problems like earlier the IPTables rules from k8s However, I am the stupidest person... Flannel works fine if you configure it correctly (force start on wireguard interface). ==> **Flannel Test 2** - [x] IPv4 only test - [x] Dual-Stack test - [x] IP assignment ==> **Flannel Test 2** now works smoothly 🙌 ==> **Final Tests** - [x] IPv6 egress test - [x] Dual-Stack egress test - [x] Drop private IPs before forward to internet