TLSCertificateDelegation does not work with networking.k8s.io/v1 Ingress

[ghouscht]

What steps did you take and what happened:
Technically this is not a contour issue, but in practice it breaks TLSCertificateDelegation with networking.k8s.io/v1 Ingress resource. As of today we can use a networking.k8s.io/v1beta1 Ingress like this:

---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: kuard
  annotations:
    kubernetes.io/ingress.class: contour
    ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  rules:
  - host: kuard.somedomain.foo
    http:
      paths:
      - backend:
          serviceName: kuard
          servicePort: http
  tls:
  - secretName: kube-contour/tls-certs
    hosts:
    - kuard.somedomain.foo

Please note; The scretName with the Namespace kube-contour in the TLS section.

Applying this definition does work, but issues a warning about the upcoming deprecation of networking.k8s.io/v1beta1 Ingress:

Warning: networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress

When trying to migrate such an Ingress definition to networking.k8s.io/v1 one would rewrite it to:

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/force-ssl-redirect: "true"
  name: kuard
spec:
  ingressClassName: contour
  rules:
  - host: kuard.somedomain.foo
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: kuard
            port:
              number: 80
  tls:
  - secretName: kube-contour/tls-certs
    hosts:
    - kuard.somedomain.foo

Applying this definition results in:

The Ingress "kuard" is invalid: spec.tls[0].secretName: Invalid value: "kube-contour/tls-certs": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

What did you expect to happen:
I think there should be a way for users to use TLSCertificateDelegation also with networking.k8s.io/v1 Ingress. Maybe there is already a way how we could solve this, but I was not able to find something. Currently this is not a serious issue, since k8s 1.22 is not yet around the corner, but this will be a serious issue for us.

Anything else you would like to add:

Environment:

• Contour version: v1.14.0
• Kubernetes version: (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:12:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:03:00Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes installer & version:

kubeadm version: &version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.4", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-02-18T16:09:38Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}

• Cloud provider or hardware configuration: on-prem
• OS (e.g. from /etc/os-release):

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

[ghouscht]

In the meantime I found https://projectcontour.io/resources/philosophy/#we-meet-users-where-they-are - looks like Ingress v1 is not yet supported. Was not aware of that and also the Compatibility Matrix nor the Support Policy mention this. Can someone please clarify which k8s Ingress resources are supported? Probably it would also be good to document this (can help here as soon as it's clear which Ingress versions are supported by contour)

[sunjayBhatia]

I’ve got a document in the works for detailing Ingress support, right now planning on describing the main features/fields and how they differ between Ingress versions (if there is a difference) and will include this detail

[stevesloka]

To answer your original question @ghouscht, Ingress/v1 make some additional restrictions on the API validation to not allow forward slashes (/) in the secretName which breaks the functionality that Contour previously would allow with TLSCertificateDelegation.

In addition to the v1 support doc that @sunjayBhatia is working on, I think we should also document this limitation with IngressV1 & TLSCertDelegation.

[sunjayBhatia]

#3469 should hopefully cover the documentation bits and should also cover #3546

[ghouscht]

Thank you for your answers. Are there any plans to support TLSCertificateDelegation also with v1 Ingress (maybe through an annotation, or something similar)? This is a bit of a ticking time bomb since we don't want to use the custom httpproxy resource mainly because of portability reasons and since we use wildcard certs we rely on the availability of TLSCertificateDelegation.

[sunjayBhatia]

I'm not sure if we've had asks for this yet, but yeah I could pretty much only think of this being supportable in Ingress v1 in an annotation, though we would have to be pretty exact how we would expose that, since Ingress TLS config has the "Hosts" field, we would need to replicate that in an annotation which could get messy

[youngnick]

Edit: I should really check the PR before commenting.

Given the changes to Ingress v1 seem to specifically stop TLSCertificateDelegation from being used, I'm not sure how best to bring it along. I seem to recall some Gateway API discussions about bringing TLS certificates across namespaces, for the wildcard use case, so we may be able to take an idea from there as well.

It does seem like an annotation may be the only way to go, but I agree it will need careful design if we do that. In general, we're trying to keep away from annotations on Ingress, but maybe this is an exception.

[skriss]

cc @xaleeks

[barbaluc]

Any news about this problem ?
We are in a situation where we don't know how to start using ingress with contour.
For the moment we have the intention to use ingress v1beta1 because the certificate secret is deployed in one namespace and used by ingresses from other namespaces.
But we want visibility because v1beta1 will be remove in kubernetes 1.22, it's nearly tomorrow.
Thanks in advance for your response

For information, we are using contour in vsphere with tanzu and we have contour support in our wmare licence.