Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: add full push when higress-https configmap updated and fix certmagic storage #1105

Merged
merged 8 commits into from
Jul 24, 2024
Merged

fix: add full push when higress-https configmap updated and fix certmagic storage #1105

merged 8 commits into from
Jul 24, 2024

Conversation

2456868764
Copy link
Collaborator

@2456868764 2456868764 commented Jul 9, 2024 •
edited
Loading

Ⅰ. Describe what this PR did

  1. 当 higress-https 配置映射更新时，ingress 将全量推送。
  2. 修复 configMgr.GetConfig 的错误。
  3. 修复 certmagic 配置映射存储分片不能生效问题。
  4. 修复 RenewalWindowRatio 配置不能生效的问题。

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

阿里云 新加波 搭了 higress cluster

  1. higress-https configmap && ingress
apiVersion: v1
kind: ConfigMap
metadata:
  name: higress-https
  namespace: higress-system
data:
  cert: |
    automaticHttps: true
    renewBeforeDays: 14999  # RenewMaxDays = 15000,为了测试Renew
    fallbackForInvalidSecret: false
    acmeIssuer:
    - ak: test
      sk: test
      email: [email protected]
      name: letsencrypt
    credentialConfig:
    - cacertSecret: foo-com-ca-secret
      domains:
      - 8.222.156.101.sslip.io
      tlsSecret: foo-com-secret
      tlsIssuer: letsencrypt
    version: test
---
apiVersion: v1
kind: Namespace
metadata:
  name: higress-course
---
apiVersion: v1
kind: Service
metadata:
  name: echo-server
  namespace: higress-course
spec:
  selector:
    app: echo-server
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server
  namespace: higress-course
  labels:
    app: echo-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-server
  template:
    metadata:
      labels:
        app: echo-server
    spec:
      containers:
        - name: echo-server
          image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/echo-server:1.3.0
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          resources:
            requests:
              cpu: 10m

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-foo
  namespace: higress-course
spec:
  ingressClassName: higress
  tls:
    - hosts:
        - "8.222.156.101.sslip.io"
  rules:
    - host: "8.222.156.101.sslip.io"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: echo-server
                port:
                  number: 8080
  1. 生成 configmap sharding
  root@iZt4ndl8svou8s8c3akegmZ:~# kubectl get configmap -n higress-system | grep higress-cert-store
  higress-cert-store-certificates-92abe106   3      160m
  higress-cert-store-default                 9      160m

higress-cert-store-certificates-92abe106 yaml 如下：

apiVersion: v1
data:
  9b706ce0: '{"k":"certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io/8.222.156.101.sslip.io.json","v":"ewoJInNhbnMiOiBbCgkJIjguMjIyLjE1Ni4xMDEuc3NsaXAuaW8iCgldLAoJImlzc3Vlcl9kYXRhIjogewoJCSJ1cmwiOiAiaHR0cHM6Ly9hY21lLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvY2VydC8wNGQyY2FkYWFhZGQyZjY0YWI3YzA2OWI2ZDU5ODQ0ZjgxZWQiLAoJCSJjYSI6ICJodHRwczovL2FjbWUtdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvZGlyZWN0b3J5IgoJfQp9"}'
  15282d9d: '{"k":"certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io/8.222.156.101.sslip.io.key","v":"LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUNwcXdweGNwbG1UQzgvTnVpSW1DSk5yb3VqV251Wmx1TXJOckRXejdONG5vQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFNEtBWkUyMnFXOXBWeks5b1pYWHdtZlVmZGY1cDlCSVZvYy9hTU56Qy8zUUw2dHdzYm1QVwpEWCtvU0gxVDFLdlhadmlFY3NXWm41dUJIV0pYLy9iYzlnPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo="}'
  a0f678b9: '{"k":"certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io/8.222.156.101.sslip.io.crt","v":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURqVENDQXhPZ0F3SUJBZ0lTQk5MSzJxcmRMMlNyZkFhYmJWbUVUNEh0TUFvR0NDcUdTTTQ5QkFNRE1ESXgKQ3pBSkJnTlZCQVlUQWxWVE1SWXdGQVlEVlFRS0V3MU1aWFFuY3lCRmJtTnllWEIwTVFzd0NRWURWUVFERXdKRgpOakFlRncweU5EQTNNVEV3TXpFME5ERmFGdzB5TkRFd01Ea3dNekUwTkRCYU1DRXhIekFkQmdOVkJBTVRGamd1Ck1qSXlMakUxTmk0eE1ERXVjM05zYVhBdWFXOHdXVEFUQmdjcWhrak9QUUlCQmdncWhrak9QUU1CQndOQ0FBVGcKb0JrVGJhcGIybFhNcjJobGRmQ1o5UjkxL21uMEVoV2h6OW93M01ML2RBdnEzQ3h1WTlZTmY2aElmVlBVcTlkbQorSVJ5eFptZm00RWRZbGYvOXR6Mm80SUNHRENDQWhRd0RnWURWUjBQQVFIL0JBUURBZ2VBTUIwR0ExVWRKUVFXCk1CUUdDQ3NHQVFVRkJ3TUJCZ2dyQmdFRkJRY0RBakFNQmdOVkhSTUJBZjhFQWpBQU1CMEdBMVVkRGdRV0JCU3gKditvM2lqZldkZmYxSml1SnZNb2d0V2J6bERBZkJnTlZIU01FR0RBV2dCU1RKMGFZQTZsUmFJNlkxc1JDU05zagp2MWlVMGpCVkJnZ3JCZ0VGQlFjQkFRUkpNRWN3SVFZSUt3WUJCUVVITUFHR0ZXaDBkSEE2THk5bE5pNXZMbXhsCmJtTnlMbTl5WnpBaUJnZ3JCZ0VGQlFjd0FvWVdhSFIwY0RvdkwyVTJMbWt1YkdWdVkzSXViM0puTHpBaEJnTlYKSFJFRUdqQVlnaFk0TGpJeU1pNHhOVFl1TVRBeExuTnpiR2x3TG1sdk1CTUdBMVVkSUFRTU1Bb3dDQVlHWjRFTQpBUUlCTUlJQkJBWUtLd1lCQkFIV2VRSUVBZ1NCOVFTQjhnRHdBSGNBUHhkTFQ5Y2lSMWlVSFdVY2hMNE5FdTJRCk4zOGZoV3Jyd2I4b2hlejRaRzRBQUFHUW9BQWlCUUFBQkFNQVNEQkdBaUVBbUl2a3JMTGZVQkFzS1V5TWxLWU4KRU5qcEFBb29NNUgweEtJM0x5U2lHNVlDSVFDSFJkNDRKdTh6NHBVWjRWMXBtMEtlMDhWQUZ0aERBc2s0N3BwVgp3NlFydVFCMUFIYi9pRDhLdHZ1VlVjSmh6UFdIdWpTMHBNMjdLZHhvUWdxZjVtZE1XanAwQUFBQmtLQUFJaklBCkFBUURBRVl3UkFJZ2ZDYytmckF3aUgxVS8yNng3VjFlaVQxTExNSFV4ZDRkdHFNQ1VPVXJXdk1DSUJwTnFnT3gKbmhTVkt1Yk0vd0NlcnZoWmxQWmFwemlWUWxyNG9sL2hTTUFITUFvR0NDcUdTTTQ5QkFNREEyZ0FNR1VDTUdmMQpvREIyTW81TWxIbFdTTnptbEhYcUdZblV6QXFsUkxHVmhQTmFBc3NkZUJib0k5MzNJcnhOSjBFOU1mL0RXZ0l4CkFQSFNUOG4zQWh4MXBBYkF2aFpMUFZTV3NNUkdxbUVMMk0vekE3TnM3TmVQbTVQc05OT2pXS0h4TjZIZ2Y3Q1kKYlE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCgotLS0tLUJFR0lOIENFUlRJRklDQVRFLS0tLS0KTUlJRVZ6Q0NBaitnQXdJQkFnSVJBTEJYUHBGemx5ZHcyN1NIeXpwRkt6Z3dEUVlKS29aSWh2Y05BUUVMQlFBdwpUekVMTUFrR0ExVUVCaE1DVlZNeEtUQW5CZ05WQkFvVElFbHVkR1Z5Ym1WMElGTmxZM1Z5YVhSNUlGSmxjMlZoCmNtTm9JRWR5YjNWd01SVXdFd1lEVlFRREV3eEpVMUpISUZKdmIzUWdXREV3SGhjTk1qUXdNekV6TURBd01EQXcKV2hjTk1qY3dNekV5TWpNMU9UVTVXakF5TVFzd0NRWURWUVFHRXdKVlV6RVdNQlFHQTFVRUNoTU5UR1YwSjNNZwpSVzVqY25sd2RERUxNQWtHQTFVRUF4TUNSVFl3ZGpBUUJnY3Foa2pPUFFJQkJnVXJnUVFBSWdOaUFBVFo4WjVHCmgvZ2hjV0NvSnV1aitybnEyaDI1RXFmVUp0bFJGTEZoZkhXV3Z5SUxPUi9WdnRFS1Jxb3RQRW9KaEM2K1FKVlYKNlJsQU4yWjE3VEpPZHdSSitIQjd3eGpuenZkeEVQNnNkTmdBMU8xdEhITVdNeENjT3JMcWJHTDB2YmlqZ2ZndwpnZlV3RGdZRFZSMFBBUUgvQkFRREFnR0dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01DQmdnckJnRUZCUWNECkFUQVNCZ05WSFJNQkFmOEVDREFHQVFIL0FnRUFNQjBHQTFVZERnUVdCQlNUSjBhWUE2bFJhSTZZMXNSQ1NOc2oKdjFpVTBqQWZCZ05WSFNNRUdEQVdnQlI1dEZubWU3Ymw1QUZ6Z0FpSXlCcFk5dW1iYmpBeUJnZ3JCZ0VGQlFjQgpBUVFtTUNRd0lnWUlLd1lCQlFVSE1BS0dGbWgwZEhBNkx5OTRNUzVwTG14bGJtTnlMbTl5Wnk4d0V3WURWUjBnCkJBd3dDakFJQmdabmdRd0JBZ0V3SndZRFZSMGZCQ0F3SGpBY29CcWdHSVlXYUhSMGNEb3ZMM2d4TG1NdWJHVnUKWTNJdWIzSm5MekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBZ0VBZll0N1NpQTFzZ1dHQ0lwdW5rNDZyNEFFeElSYwpNeGtLZ1VoTmxycnYxQjIxaE9hWE4vNW1pRStMT1RicmNtVS9NOXl2QzZNVlk3MzBHTkZvTDhJaEo4ajh2ck9MCnBNWTIyT1A2YmFTMWs5WU1ydERUbHdKSG9HYnkwNFRoVFVlQkRrc1M5Uml1SHZpY1pxQmVkUWRJRjY1cFp1aHAKZURjR0JjTGlZYXNRci9FTzVneHh0THlUbWdzSFNPVlNCY0ZPbjlsZ3Y3TEVDUHE5aTdtZkgzbXB4Z3JSS1N4SApwT29aMEtYTWNCK2hIdXZsa2xIbnR2Y0kwbU1NUTBtaFlqNnF0TUZTdGtGMVJwQ0czSVBkSXdwVkNRcXU4R1Y3CnM4dWJrblJ6cyszQy9CbTE5UkZPb2lQcERrd3Z5TmZ2bVExNFhreXFxS0s1b1o4emhEMzJrRlJRa3hhOHVaU3UKaDRhVEltRnhrbnUzOXdhQnhJUlhFNGpLeGxBbVFjNFFqRlpvcTFLbVFxUWcwSi8xSkY4UmxGdkphczFWY2pMdgpZbHZVQjJ0Nm5wTzZvUWpCM2wrUE5mMERwUUg3aVV4M1d6NUFqUUNpNkwyNUZqeUUwNnE2QlovUWxtdFlkbC84ClpZYW80U1JxUEVzLzZjQWlGK1FmNXpnMlVrYVd0RHBobDFMS011VE5Mb3R2c1g5OUhQNjlWMmZhTnllZ29kUTAKTHlUQXByL3ZUMDFZUEU0NnZOc0RMZ0srNGNMNlRyekMvYTRXY21GNVNSSjkzOHpydi9kdUpITFhRSWt1NXYwKwpFd095NTlIZG0wUFQvRXIvODRkRFYwQ1NqZFIvMlh1Wk0za3B5c1NLTGdEMWNLaURBK0lSZ3VPREN4Zk85Y3lZCklnNDZ2OW1GbUJ2eUgwND0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="}'
kind: ConfigMap
metadata:
  annotations:
    higress.io/cert-https: "true"
  creationTimestamp: "2024-07-11T01:52:00Z"
  name: higress-cert-store-certificates-92abe106
  namespace: higress-system
  resourceVersion: "15408"
  uid: 9da8d7d3-b1b5-4d80-87e8-b9d33d943f22
  1. https://8.222.156.101.sslip.io/
截屏2024-07-11 09 54 26
  1. renew 测试

renew 日志如下：

024-07-11T03:44:43.233612Z     info    cert    certMgr manageSync domains done
1.720670080033656e+09   info    maintenance     start to renew ManagedCertificates
1.7206700800337026e+09  info    maintenance     cache certKey   {"cert_key": "00ad3244c475b7dc411c030285ee8e820dd13109f5e5809b6cbe154cb2108125"}
1.7206700800337105e+09  info    maintenance     cert name       {"cert_name": "8.222.156.101.sslip.io"}
2024-07-11T03:54:40.033718Z     info    cert    certmgr cache GetConfigForCert
2024-07-11T03:54:40.034000Z     info    cert    certmgr config: &{0.9999333333333333 0x1eac500   <nil> false [0xc0009ac140]  false {p256} <nil> {false map[]} ConfigmapStorage false 0xc0009aec40 0xc000c29ef0}
1.7206700800340085e+09  info    maintenance     need to renew cert name {"cert_name": "8.222.156.101.sslip.io"}
1.7206700800433915e+09  info    maintenance     add cert to renewQueue  {"cert_name": "8.222.156.101.sslip.io"}
1.7206700800434122e+09  info    maintenance     certificate expires soon; queuing for renewal   {"identifiers": ["8.222.156.101.sslip.io"], "remaining": 7771800.9565886}
1.7206700800435774e+09  info    maintenance     attempting certificate renewal  {"identifiers": ["8.222.156.101.sslip.io"], "remaining": 7771800.95642431}
1.720670080064369e+09   info    renew   acquiring lock  {"identifier": "8.222.156.101.sslip.io"}
1.720670080065242e+09   info    renew   lock acquired   {"identifier": "8.222.156.101.sslip.io"}
1.7206700800709496e+09  info    renew   renewing certificate    {"identifier": "8.222.156.101.sslip.io", "remaining": 7771800.929052114}
2024-07-11T03:54:40.071066Z     info    cert    certmgr receive event:%!d(string=cert_obtaining)ata:map[forced:false identifier:8.222.156.101.sslip.io issuer:acme-v02.api.letsencrypt.org-directory remaining:2158h50m0.929052114s renewal:true]
1.7206700800783591e+09  info    waiting on internal rate limiter        {"identifiers": ["8.222.156.101.sslip.io"], "ca": "https://acme-v02.api.letsencrypt.org/directory", "account": "[email protected]"}
1.7206700800792503e+09  info    done waiting on internal rate limiter   {"identifiers": ["8.222.156.101.sslip.io"], "ca": "https://acme-v02.api.letsencrypt.org/directory", "account": "[email protected]"}
1.7206700812960255e+09  info    acme_client     authorization finalized {"identifier": "8.222.156.101.sslip.io", "authz_status": "valid"}
1.7206700812960458e+09  info    acme_client     validations succeeded; finalizing order {"order": "https://acme-v02.api.letsencrypt.org/acme/order/1830530677/286215397237"}
1.7206700826325202e+09  info    acme_client     successfully downloaded available certificate chains    {"count": 2, "first_url": "https://acme-v02.api.letsencrypt.org/acme/cert/04d4b7ab2866579e13c0b7bd807fd447807e"}
1.7206700826571453e+09  info    renew   certificate renewed successfully        {"identifier": "8.222.156.101.sslip.io"}
2024-07-11T03:54:42.659062Z     info    cert    certmgr receive event:%!d(string=cert_obtained)ata:map[certificate_path:certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io/8.222.156.101.sslip.io.crt identifier:8.222.156.101.sslip.io issuer:acme-v02.api.letsencrypt.org-directory metadata_path:certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io/8.222.156.101.sslip.io.json private_key_path:certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io/8.222.156.101.sslip.io.key remaining:2158h50m0.929052114s renewal:true storage_path:certificates/acme-v02.api.letsencrypt.org-directory/8.222.156.101.sslip.io]
2024-07-11T03:54:42.663277Z     info    cert    update secret, domain:8.222.156.101.sslip.io, secretName:foo-com-secret, privateKey:-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIMCa8oIQRoMtdWDKd1pzz4oU5dHPgPJDugAGYpE9h2A/oAoGCCqGSM49
AwEHoUQDQgAE2BEF3zoVzL2Pt+DwK2bvH7SPuiYlhf6wfpSvMz9yzaGVAqQ0MmO+
1HcAtOb+56jpNeooas+AtxD5d9kt8H1rDg==
-----END EC PRIVATE KEY-----
, certificate:-----BEGIN CERTIFICATE-----
MIIDizCCAxGgAwIBAgISBNS3qyhmV54TwLe9gH/UR4B+MAoGCCqGSM49BAMDMDIx
CzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQDEwJF
NjAeFw0yNDA3MTEwMjU0NDFaFw0yNDEwMDkwMjU0NDBaMCExHzAdBgNVBAMTFjgu
MjIyLjE1Ni4xMDEuc3NsaXAuaW8wWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAATY
EQXfOhXMvY+34PArZu8ftI+6JiWF/rB+lK8zP3LNoZUCpDQyY77UdwC05v7nqOk1
6ihqz4C3EPl32S3wfWsOo4ICFjCCAhIwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQW
MBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBR0
7hHQq7w1qQ3QdkIjunQgq3tOBTAfBgNVHSMEGDAWgBSTJ0aYA6lRaI6Y1sRCSNsj
v1iU0jBVBggrBgEFBQcBAQRJMEcwIQYIKwYBBQUHMAGGFWh0dHA6Ly9lNi5vLmxl
bmNyLm9yZzAiBggrBgEFBQcwAoYWaHR0cDovL2U2LmkubGVuY3Iub3JnLzAhBgNV
HREEGjAYghY4LjIyMi4xNTYuMTAxLnNzbGlwLmlvMBMGA1UdIAQMMAowCAYGZ4EM
AQIBMIIBAgYKKwYBBAHWeQIEAgSB8wSB8ADuAHUAPxdLT9ciR1iUHWUchL4NEu2Q
N38fhWrrwb8ohez4ZG4AAAGQn+3S0QAABAMARjBEAiAQmBQu+f08Rl/QB7saLene
hC6lHC6YgTmpcjQAlfkfaAIgHDvUzMKScC7kU4Eg9NzwLHAcs2OdUIgYLe5mQPEC
fiEAdQBIsONr2qZHNA/lagL6nTDrHFIBy1bdLIHZu7+rOdiEcwAAAZCf7dMOAAAE
AwBGMEQCIGsa3I3rj+qSC6FmrRgEjN7AnUg1jqo+y9Xdg6qzlviyAiA6/L/o9b6u
454pZzIlGXYPBnI9wxrg6oCJzn+bfszaXjAKBggqhkjOPQQDAwNoADBlAjEAqQXW
pd1Beq28WQ+pHu/03HxR6s0vzL/t1TWq/z6zB8UfdAAe2DSVIjlRWprDvwuCAjAg
1hNDw4PGW9cVOO6w6rYLCq6x5fVxDGtj2QJwql1PIgc8AMXp5kBDTb5x2M5J1i4=
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
MIIEVzCCAj+gAwIBAgIRALBXPpFzlydw27SHyzpFKzgwDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjQwMzEzMDAwMDAw
WhcNMjcwMzEyMjM1OTU5WjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCRTYwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAATZ8Z5G
h/ghcWCoJuuj+rnq2h25EqfUJtlRFLFhfHWWvyILOR/VvtEKRqotPEoJhC6+QJVV
6RlAN2Z17TJOdwRJ+HB7wxjnzvdxEP6sdNgA1O1tHHMWMxCcOrLqbGL0vbijgfgw
gfUwDgYDVR0PAQH/BAQDAgGGMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEFBQcD
ATASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdDgQWBBSTJ0aYA6lRaI6Y1sRCSNsj
v1iU0jAfBgNVHSMEGDAWgBR5tFnme7bl5AFzgAiIyBpY9umbbjAyBggrBgEFBQcB
AQQmMCQwIgYIKwYBBQUHMAKGFmh0dHA6Ly94MS5pLmxlbmNyLm9yZy8wEwYDVR0g
BAwwCjAIBgZngQwBAgEwJwYDVR0fBCAwHjAcoBqgGIYWaHR0cDovL3gxLmMubGVu
Y3Iub3JnLzANBgkqhkiG9w0BAQsFAAOCAgEAfYt7SiA1sgWGCIpunk46r4AExIRc
MxkKgUhNlrrv1B21hOaXN/5miE+LOTbrcmU/M9yvC6MVY730GNFoL8IhJ8j8vrOL
pMY22OP6baS1k9YMrtDTlwJHoGby04ThTUeBDksS9RiuHvicZqBedQdIF65pZuhp
eDcGBcLiYasQr/EO5gxxtLyTmgsHSOVSBcFOn9lgv7LECPq9i7mfH3mpxgrRKSxH
pOoZ0KXMcB+hHuvlklHntvcI0mMMQ0mhYj6qtMFStkF1RpCG3IPdIwpVCQqu8GV7
s8ubknRzs+3C/Bm19RFOoiPpDkwvyNfvmQ14XkyqqKK5oZ8zhD32kFRQkxa8uZSu
h4aTImFxknu39waBxIRXE4jKxlAmQc4QjFZoq1KmQqQg0J/1JF8RlFvJas1VcjLv
YlvUB2t6npO6oQjB3l+PNf0DpQH7iUx3Wz5AjQCi6L25FjyE06q6BZ/QlmtYdl/8
ZYao4SRqPEs/6cAiF+Qf5zg2UkaWtDphl1LKMuTNLotvsX99HP69V2faNyegodQ0
LyTApr/vT01YPE46vNsDLgK+4cL6TrzC/a4WcmF5SRJ938zrv/duJHLXQIku5v0+
EwOy59Hdm0PT/Er/84dDV0CSjdR/2XuZM3kpysSKLgD1cKiDA+IRguODCxfO9cyY
Ig46v9mFmBvyH04=
-----END CERTIFICATE-----
, notBefore:2024-07-11 02:54:42 +0000 UTC, notAfter:2024-10-09 02:54:41 +0000 UTC, isRenew:true
1.7206700826769848e+09  info    renew   releasing lock  {"identifier": "8.222.156.101.sslip.io"}
1.7206700826770115e+09  info    reloading managed certificate   {"identifiers": ["8.222.156.101.sslip.io"]}
1.7206700829618974e+09  debug   removed certificate from cache  {"subjects": ["8.222.156.101.sslip.io"], "expiration": 1728441881, "managed": true, "issuer_key": "acme-v02.api.letsencrypt.org-directory", "hash": "00ad3244c475b7dc411c030285ee8e820dd13109f5e5809b6cbe154cb2108125", "cache_size": 0, "cache_capacity": 0}
1.7206700829619248e+09  debug   added certificate to cache      {"subjects": ["8.222.156.101.sslip.io"], "expiration": 1728442481, "managed": true, "issuer_key": "acme-v02.api.letsencrypt.org-directory", "hash": "a45d839a3aaa736cc1c978fa465ae3ba605c1b97fef1c4581cea03d01ab273e7", "cache_size": 1, "cache_capacity": 0}
1.720670082961932e+09   info    replaced certificate in cache   {"subjects": ["8.222.156.101.sslip.io"], "new_expiration": 1728442481}

foo-com-secretsecret yaml info 如下：


  apiVersion: v1
  data:
    tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURqVENDQXhPZ0F3SUJBZ0lTQk5MSzJxcmRMMlNyZkFhYmJWbUVUNEh0TUFvR0NDcUdTTTQ5QkFNRE1ESXgKQ3pBSkJnTlZCQVlUQWxWVE1SWXdGQVlEVlFRS0V3MU1aWFFuY3lCRmJtTnllWEIwTVFzd0NRWURWUVFERXdKRgpOakFlRncweU5EQTNNVEV3TXpFME5ERmFGdzB5TkRFd01Ea3dNekUwTkRCYU1DRXhIekFkQmdOVkJBTVRGamd1Ck1qSXlMakUxTmk0eE1ERXVjM05zYVhBdWFXOHdXVEFUQmdjcWhrak9QUUlCQmdncWhrak9QUU1CQndOQ0FBVGcKb0JrVGJhcGIybFhNcjJobGRmQ1o5UjkxL21uMEVoV2h6OW93M01ML2RBdnEzQ3h1WTlZTmY2aElmVlBVcTlkbQorSVJ5eFptZm00RWRZbGYvOXR6Mm80SUNHRENDQWhRd0RnWURWUjBQQVFIL0JBUURBZ2VBTUIwR0ExVWRKUVFXCk1CUUdDQ3NHQVFVRkJ3TUJCZ2dyQmdFRkJRY0RBakFNQmdOVkhSTUJBZjhFQWpBQU1CMEdBMVVkRGdRV0JCU3gKditvM2lqZldkZmYxSml1SnZNb2d0V2J6bERBZkJnTlZIU01FR0RBV2dCU1RKMGFZQTZsUmFJNlkxc1JDU05zagp2MWlVMGpCVkJnZ3JCZ0VGQlFjQkFRUkpNRWN3SVFZSUt3WUJCUVVITUFHR0ZXaDBkSEE2THk5bE5pNXZMbXhsCmJtTnlMbTl5WnpBaUJnZ3JCZ0VGQlFjd0FvWVdhSFIwY0RvdkwyVTJMbWt1YkdWdVkzSXViM0puTHpBaEJnTlYKSFJFRUdqQVlnaFk0TGpJeU1pNHhOVFl1TVRBeExuTnpiR2x3TG1sdk1CTUdBMVVkSUFRTU1Bb3dDQVlHWjRFTQpBUUlCTUlJQkJBWUtLd1lCQkFIV2VRSUVBZ1NCOVFTQjhnRHdBSGNBUHhkTFQ5Y2lSMWlVSFdVY2hMNE5FdTJRCk4zOGZoV3Jyd2I4b2hlejRaRzRBQUFHUW9BQWlCUUFBQkFNQVNEQkdBaUVBbUl2a3JMTGZVQkFzS1V5TWxLWU4KRU5qcEFBb29NNUgweEtJM0x5U2lHNVlDSVFDSFJkNDRKdTh6NHBVWjRWMXBtMEtlMDhWQUZ0aERBc2s0N3BwVgp3NlFydVFCMUFIYi9pRDhLdHZ1VlVjSmh6UFdIdWpTMHBNMjdLZHhvUWdxZjVtZE1XanAwQUFBQmtLQUFJaklBCkFBUURBRVl3UkFJZ2ZDYytmckF3aUgxVS8yNng3VjFlaVQxTExNSFV4ZDRkdHFNQ1VPVXJXdk1DSUJwTnFnT3gKbmhTVkt1Yk0vd0NlcnZoWmxQWmFwemlWUWxyNG9sL2hTTUFITUFvR0NDcUdTTTQ5QkFNREEyZ0FNR1VDTUdmMQpvREIyTW81TWxIbFdTTnptbEhYcUdZblV6QXFsUkxHVmhQTmFBc3NkZUJib0k5MzNJcnhOSjBFOU1mL0RXZ0l4CkFQSFNUOG4zQWh4MXBBYkF2aFpMUFZTV3NNUkdxbUVMMk0vekE3TnM3TmVQbTVQc05OT2pXS0h4TjZIZ2Y3Q1kKYlE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCgotLS0tLUJFR0lOIENFUlRJRklDQVRFLS0tLS0KTUlJRVZ6Q0NBaitnQXdJQkFnSVJBTEJYUHBGemx5ZHcyN1NIeXpwRkt6Z3dEUVlKS29aSWh2Y05BUUVMQlFBdwpUekVMTUFrR0ExVUVCaE1DVlZNeEtUQW5CZ05WQkFvVElFbHVkR1Z5Ym1WMElGTmxZM1Z5YVhSNUlGSmxjMlZoCmNtTm9JRWR5YjNWd01SVXdFd1lEVlFRREV3eEpVMUpISUZKdmIzUWdXREV3SGhjTk1qUXdNekV6TURBd01EQXcKV2hjTk1qY3dNekV5TWpNMU9UVTVXakF5TVFzd0NRWURWUVFHRXdKVlV6RVdNQlFHQTFVRUNoTU5UR1YwSjNNZwpSVzVqY25sd2RERUxNQWtHQTFVRUF4TUNSVFl3ZGpBUUJnY3Foa2pPUFFJQkJnVXJnUVFBSWdOaUFBVFo4WjVHCmgvZ2hjV0NvSnV1aitybnEyaDI1RXFmVUp0bFJGTEZoZkhXV3Z5SUxPUi9WdnRFS1Jxb3RQRW9KaEM2K1FKVlYKNlJsQU4yWjE3VEpPZHdSSitIQjd3eGpuenZkeEVQNnNkTmdBMU8xdEhITVdNeENjT3JMcWJHTDB2YmlqZ2ZndwpnZlV3RGdZRFZSMFBBUUgvQkFRREFnR0dNQjBHQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01DQmdnckJnRUZCUWNECkFUQVNCZ05WSFJNQkFmOEVDREFHQVFIL0FnRUFNQjBHQTFVZERnUVdCQlNUSjBhWUE2bFJhSTZZMXNSQ1NOc2oKdjFpVTBqQWZCZ05WSFNNRUdEQVdnQlI1dEZubWU3Ymw1QUZ6Z0FpSXlCcFk5dW1iYmpBeUJnZ3JCZ0VGQlFjQgpBUVFtTUNRd0lnWUlLd1lCQlFVSE1BS0dGbWgwZEhBNkx5OTRNUzVwTG14bGJtTnlMbTl5Wnk4d0V3WURWUjBnCkJBd3dDakFJQmdabmdRd0JBZ0V3SndZRFZSMGZCQ0F3SGpBY29CcWdHSVlXYUhSMGNEb3ZMM2d4TG1NdWJHVnUKWTNJdWIzSm5MekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBZ0VBZll0N1NpQTFzZ1dHQ0lwdW5rNDZyNEFFeElSYwpNeGtLZ1VoTmxycnYxQjIxaE9hWE4vNW1pRStMT1RicmNtVS9NOXl2QzZNVlk3MzBHTkZvTDhJaEo4ajh2ck9MCnBNWTIyT1A2YmFTMWs5WU1ydERUbHdKSG9HYnkwNFRoVFVlQkRrc1M5Uml1SHZpY1pxQmVkUWRJRjY1cFp1aHAKZURjR0JjTGlZYXNRci9FTzVneHh0THlUbWdzSFNPVlNCY0ZPbjlsZ3Y3TEVDUHE5aTdtZkgzbXB4Z3JSS1N4SApwT29aMEtYTWNCK2hIdXZsa2xIbnR2Y0kwbU1NUTBtaFlqNnF0TUZTdGtGMVJwQ0czSVBkSXdwVkNRcXU4R1Y3CnM4dWJrblJ6cyszQy9CbTE5UkZPb2lQcERrd3Z5TmZ2bVExNFhreXFxS0s1b1o4emhEMzJrRlJRa3hhOHVaU3UKaDRhVEltRnhrbnUzOXdhQnhJUlhFNGpLeGxBbVFjNFFqRlpvcTFLbVFxUWcwSi8xSkY4UmxGdkphczFWY2pMdgpZbHZVQjJ0Nm5wTzZvUWpCM2wrUE5mMERwUUg3aVV4M1d6NUFqUUNpNkwyNUZqeUUwNnE2QlovUWxtdFlkbC84ClpZYW80U1JxUEVzLzZjQWlGK1FmNXpnMlVrYVd0RHBobDFMS011VE5Mb3R2c1g5OUhQNjlWMmZhTnllZ29kUTAKTHlUQXByL3ZUMDFZUEU0NnZOc0RMZ0srNGNMNlRyekMvYTRXY21GNVNSSjkzOHpydi9kdUpITFhRSWt1NXYwKwpFd095NTlIZG0wUFQvRXIvODRkRFYwQ1NqZFIvMlh1Wk0za3B5c1NLTGdEMWNLaURBK0lSZ3VPREN4Zk85Y3lZCklnNDZ2OW1GbUJ2eUgwND0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    tls.key: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUNwcXdweGNwbG1UQzgvTnVpSW1DSk5yb3VqV251Wmx1TXJOckRXejdONG5vQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFNEtBWkUyMnFXOXBWeks5b1pYWHdtZlVmZGY1cDlCSVZvYy9hTU56Qy8zUUw2dHdzYm1QVwpEWCtvU0gxVDFLdlhadmlFY3NXWm41dUJIV0pYLy9iYzlnPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=
  kind: Secret
  metadata:
    annotations:
      higress.io/cert-domain: 8.222.156.101.sslip.io
      higress.io/cert-notAfter: "2024-10-09 03:14:41"
      higress.io/cert-notBefore: "2024-07-11 03:14:42"
      higress.io/cert-renew: "true"
      higress.io/cert-renew-time: "2024-07-11 04:14:42"
      higress.io/cert-source: letsencrypt
    creationTimestamp: "2024-07-11T01:52:10Z"
    name: foo-com-secret
    namespace: higress-system
    resourceVersion: "15409"
    uid: abde9c14-3cd0-4279-abe4-de900ed2e896
  type: kubernetes.io/tls

这里看到 secret已经Renew
https://8.222.156.101.sslip.io/
截屏2024-07-11 12 30 20

证书签发日期变更成最新日期。

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

@2456868764 2456868764 changed the title add full push when higress-https configmap updated and fix getConfig bug fix: add full push when higress-https configmap updated and fix getConfig bug Jul 9, 2024
johnlanni
johnlanni requested changes Jul 9, 2024
View reviewed changes
pkg/cert/config.go Outdated Show resolved Hide resolved
@2456868764 2456868764 changed the title fix: add full push when higress-https configmap updated and fix getConfig bug fix: add full push when higress-https configmap updated and fix certmagic storage Jul 9, 2024
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 21.95122% with 32 lines in your changes missing coverage. Please review.

Project coverage is 35.84%. Comparing base (e0159f5) to head (a9ec873).
Report is 10 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1105      +/-   ##
==========================================
- Coverage   35.89%   35.84%   -0.06%     
==========================================
  Files          69       69              
  Lines       11549    11587      +38     
==========================================
+ Hits         4146     4153       +7     
- Misses       7087     7118      +31     
  Partials      316      316
Files Coverage Δ
pkg/bootstrap/server.go 57.45% <100.00%> (ø)
pkg/cert/storage.go 55.45% <100.00%> (-0.21%) ⬇️
pkg/cert/secret.go 0.00% <0.00%> (ø)
pkg/cert/server.go 0.00% <0.00%> (ø)
pkg/cert/certmgr.go 0.00% <0.00%> (ø)
pkg/cert/config.go 8.15% <0.00%> (+4.05%) ⬆️

... and 2 files with indirect coverage changes

@2456868764
Copy link
Collaborator Author

@johnlanni 麻烦看一下

johnlanni
johnlanni requested changes Jul 11, 2024
View reviewed changes
Copy link
Collaborator

@johnlanni johnlanni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

需要代码注释说明一下certmagic的证书存储原理，按目前代码理解，证书在configmap和secret里似乎冗余存储了？

pkg/cert/storage.go Outdated Show resolved Hide resolved
johnlanni
johnlanni requested changes Jul 11, 2024
View reviewed changes
pkg/cert/secret.go Outdated Show resolved Hide resolved
johnlanni
johnlanni approved these changes Jul 24, 2024
View reviewed changes
Copy link
Collaborator

@johnlanni johnlanni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@johnlanni johnlanni merged commit 2a588c9 into alibaba:main Jul 24, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@johnlanni johnlanni johnlanni approved these changes

@SpecialYang SpecialYang Awaiting requested review from SpecialYang SpecialYang is a code owner

@CH3CHO CH3CHO Awaiting requested review from CH3CHO CH3CHO is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@2456868764 @codecov-commenter @johnlanni