endless port+floating-ip allocation (LB for kube-api)

Created by: garloff

The loadbalancer in front of the kubernetes API server(s) does get a floating IP address assigned. Occasionally, k8s does not seem to consider this assignment successful. When that happens, it allocates a new FIP, tries to assign it to the LB VIP port and fails (as we already have one FIP from the same external net assigned to that port). It does repeat that exercise, without cleaning up the allocated new FIP, so this goes until we run against the FIP quota limit. I see at least two bugs here:

1. The first FIP assignment actually has been successful -- it should not be considered failed
2. The retry loop should clean up created and failed FIPs again ... (and maybe notice here finally that the first FIP assignment exists)

It's unclear to me at this point whether the issue is with capo or occm.