Log file created at: 2020/08/26 20:07:41 Running on machine: k8s-node-worker-1 Binary: Built with gc go1.13.14 for linux/amd64 Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg I0826 20:07:41.969833 1 log_file.go:99] Set log file max size to 104857600 I0826 20:07:41.970115 1 controller.go:72] Starting Antrea Controller (version v0.10.0-dev-23d5c33.dirty) I0826 20:07:41.970131 1 client.go:34] No kubeconfig file was specified. Falling back to in-cluster config I0826 20:07:43.884550 1 serving.go:307] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:07:43.893982 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:10:43.893977455 +0000 UTC m=+181.969454278 I0826 20:07:45.041503 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:07:45.042811 1 controller.go:60] Starting Antrea Controller Monitor I0826 20:07:45.042827 1 controller.go:62] Waiting for node synced for Controller Monitor I0826 20:07:45.042853 1 networkpolicy_controller.go:962] Starting NetworkPolicy controller I0826 20:07:45.042858 1 networkpolicy_controller.go:965] Waiting for caches to sync for NetworkPolicy controller I0826 20:07:45.042956 1 log_file.go:127] Starting log file monitoring. Maximum log file number is 4 I0826 20:07:45.052379 1 log_file.go:184] Deleted log file antrea-controller.k8s-node-worker-1.root.log.INFO.20200825-210430.1 I0826 20:07:45.143493 1 controller.go:67] Caches are synced for Controller Monitor I0826 20:07:45.144798 1 networkpolicy_controller.go:977] Caches are synced for NetworkPolicy controller I0826 20:07:45.196066 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:07:46.550342 1 cacert_controller.go:183] Starting CACertController I0826 20:07:46.552464 1 dynamic_cafile_content.go:167] Starting self-signed cert::/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt I0826 20:07:46.599075 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0826 20:07:46.599096 1 shared_informer.go:223] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0826 20:07:46.599116 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0826 20:07:46.599121 1 shared_informer.go:223] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0826 20:07:46.599986 1 dynamic_serving_content.go:130] Starting serving-cert::/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt::/var/run/antrea/antrea-controller-self-signed/antrea-controller.key I0826 20:07:46.600067 1 secure_serving.go:178] Serving securely on [::]:10349 I0826 20:07:46.600104 1 tlsconfig.go:240] Starting DynamicServingCertificateController I0826 20:07:46.699552 1 shared_informer.go:230] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0826 20:07:46.699775 1 shared_informer.go:230] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0826 20:10:43.897927 1 certificate.go:182] Rotating self signed certificate I0826 20:10:44.307350 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:10:44.307677 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:10:44.307837 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:10:44.313192 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:10:44.315541 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:10:44.321488 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:10:44.327980 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:10:44.343997 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:13:44.343991855 +0000 UTC m=+362.419468676 I0826 20:13:44.344444 1 certificate.go:182] Rotating self signed certificate I0826 20:13:44.697798 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:13:44.697958 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:13:44.698012 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:13:44.704207 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:13:44.706997 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:13:44.712526 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:13:44.717983 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:13:44.729229 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating antrea CA cert of APIService v1beta1.networking.antrea.tanzu.vmware.com: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.networking.antrea.tanzu.vmware.com": the object has been modified; please apply your changes to the latest version and try again I0826 20:13:44.739677 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:13:44.740473 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:16:44.740469917 +0000 UTC m=+542.815946747 I0826 20:13:44.744559 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:16:44.745203 1 certificate.go:182] Rotating self signed certificate I0826 20:16:45.510127 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:16:45.510397 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:16:45.510474 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:16:45.516519 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:16:45.517253 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:16:45.523508 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:16:45.525217 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:16:45.533279 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating antrea CA cert of APIService v1beta1.networking.antrea.tanzu.vmware.com: Operation cannot be fulfilled on apiservices.apiregistration.k8s.io "v1beta1.networking.antrea.tanzu.vmware.com": the object has been modified; please apply your changes to the latest version and try again I0826 20:16:45.545534 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:16:45.549820 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:19:45.549805544 +0000 UTC m=+723.625282438 I0826 20:16:45.555408 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:17:45.055455 1 log_file.go:184] Deleted log file antrea-controller.k8s-node-worker-1.root.log.WARNING.20200813-103522.1 I0826 20:17:45.055614 1 log_file.go:184] Deleted log file antrea-controller.k8s-node-worker-1.root.log.ERROR.20200813-103507.1 I0826 20:19:45.550224 1 certificate.go:182] Rotating self signed certificate I0826 20:19:45.887826 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:19:45.888028 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:19:45.888325 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:19:45.894073 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:19:45.914646 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:19:45.920075 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:19:45.922357 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:19:45.925417 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:22:45.925412465 +0000 UTC m=+904.000889280 I0826 20:22:45.926011 1 certificate.go:182] Rotating self signed certificate I0826 20:22:46.350501 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:22:46.350775 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:22:46.350948 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:22:46.358590 1 cacert_controller.go:124] Syncing CA certificate with APIServices E0826 20:22:46.359807 1 cacert_controller.go:213] Error syncing CA cert, requeuing it: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:22:46.365850 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:22:46.375635 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:22:46.380659 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:25:46.380655077 +0000 UTC m=+1084.456131892 I0826 20:25:46.387973 1 certificate.go:182] Rotating self signed certificate I0826 20:25:46.683487 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:25:46.683954 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:25:46.683994 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:25:46.691364 1 cacert_controller.go:124] Syncing CA certificate with APIServices W0826 20:25:46.691982 1 cacert_controller.go:172] Initial sync of CA content failed: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:25:46.692229 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:28:46.692223237 +0000 UTC m=+1264.767700053 I0826 20:25:46.712892 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:25:46.716204 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:25:56.410962 1 log.go:172] http: TLS handshake error from 192.168.77.100:30032: remote error: tls: bad certificate I0826 20:25:56.443256 1 log.go:172] http: TLS handshake error from 192.168.77.100:56406: remote error: tls: bad certificate I0826 20:25:56.703147 1 log.go:172] http: TLS handshake error from 192.168.77.100:53749: remote error: tls: bad certificate I0826 20:25:56.965977 1 log.go:172] http: TLS handshake error from 192.168.77.100:38195: remote error: tls: bad certificate I0826 20:25:57.162528 1 log.go:172] http: TLS handshake error from 192.168.77.100:46925: remote error: tls: bad certificate I0826 20:25:57.306558 1 log.go:172] http: TLS handshake error from 192.168.77.100:36242: remote error: tls: bad certificate I0826 20:25:57.321349 1 log.go:172] http: TLS handshake error from 192.168.77.100:26617: remote error: tls: bad certificate I0826 20:25:58.910439 1 log.go:172] http: TLS handshake error from 192.168.77.100:39400: remote error: tls: bad certificate I0826 20:26:26.912409 1 log.go:172] http: TLS handshake error from 192.168.77.100:9630: remote error: tls: bad certificate I0826 20:26:27.414400 1 log.go:172] http: TLS handshake error from 192.168.77.100:25273: remote error: tls: bad certificate I0826 20:26:28.955083 1 log.go:172] http: TLS handshake error from 192.168.77.100:10532: remote error: tls: bad certificate I0826 20:26:29.453083 1 log.go:172] http: TLS handshake error from 192.168.77.100:3634: remote error: tls: bad certificate I0826 20:26:29.611231 1 log.go:172] http: TLS handshake error from 192.168.77.100:1393: remote error: tls: bad certificate I0826 20:26:31.459674 1 log.go:172] http: TLS handshake error from 192.168.77.100:58635: remote error: tls: bad certificate I0826 20:28:46.692836 1 certificate.go:182] Rotating self signed certificate I0826 20:28:47.048923 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:28:47.049167 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:28:47.049289 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:28:47.057196 1 cacert_controller.go:124] Syncing CA certificate with APIServices W0826 20:28:47.058750 1 cacert_controller.go:172] Initial sync of CA content failed: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:28:47.058898 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:31:47.058895875 +0000 UTC m=+1445.134372693 I0826 20:28:47.102824 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:28:47.106344 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:28:57.349976 1 log.go:172] http: TLS handshake error from 192.168.77.100:40992: remote error: tls: bad certificate I0826 20:28:57.366168 1 log.go:172] http: TLS handshake error from 192.168.77.100:60613: remote error: tls: bad certificate I0826 20:28:59.481567 1 log.go:172] http: TLS handshake error from 192.168.77.100:49061: remote error: tls: bad certificate I0826 20:28:59.973240 1 log.go:172] http: TLS handshake error from 192.168.77.100:15268: remote error: tls: bad certificate I0826 20:29:03.531977 1 log.go:172] http: TLS handshake error from 192.168.77.100:17343: remote error: tls: bad certificate I0826 20:29:03.973060 1 log.go:172] http: TLS handshake error from 192.168.77.100:4430: remote error: tls: bad certificate I0826 20:29:04.168424 1 log.go:172] http: TLS handshake error from 192.168.77.100:26086: remote error: tls: bad certificate I0826 20:29:05.817212 1 log.go:172] http: TLS handshake error from 192.168.77.100:52856: remote error: tls: bad certificate I0826 20:29:29.993128 1 log.go:172] http: TLS handshake error from 192.168.77.100:63894: remote error: tls: bad certificate I0826 20:29:30.148417 1 log.go:172] http: TLS handshake error from 192.168.77.100:29754: remote error: tls: bad certificate I0826 20:29:36.044189 1 log.go:172] http: TLS handshake error from 192.168.77.100:12931: remote error: tls: bad certificate I0826 20:29:36.429146 1 log.go:172] http: TLS handshake error from 192.168.77.100:64462: remote error: tls: bad certificate I0826 20:29:36.718160 1 log.go:172] http: TLS handshake error from 192.168.77.100:53866: remote error: tls: bad certificate I0826 20:29:38.520145 1 log.go:172] http: TLS handshake error from 192.168.77.100:27347: remote error: tls: bad certificate I0826 20:31:47.059316 1 certificate.go:182] Rotating self signed certificate I0826 20:31:47.343679 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:31:47.343808 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:31:47.343892 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:31:47.350592 1 cacert_controller.go:124] Syncing CA certificate with APIServices W0826 20:31:47.351559 1 cacert_controller.go:172] Initial sync of CA content failed: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:31:47.351775 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:34:47.351773113 +0000 UTC m=+1625.427249930 I0826 20:31:47.387187 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:31:47.390864 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:31:57.397815 1 log.go:172] http: TLS handshake error from 192.168.77.100:31399: remote error: tls: bad certificate I0826 20:31:57.413441 1 log.go:172] http: TLS handshake error from 192.168.77.100:20116: remote error: tls: bad certificate I0826 20:32:02.535495 1 log.go:172] http: TLS handshake error from 192.168.77.100:11829: remote error: tls: bad certificate I0826 20:32:03.026449 1 log.go:172] http: TLS handshake error from 192.168.77.100:38959: remote error: tls: bad certificate I0826 20:32:10.575763 1 log.go:172] http: TLS handshake error from 192.168.77.100:45346: remote error: tls: bad certificate I0826 20:32:11.060157 1 log.go:172] http: TLS handshake error from 192.168.77.100:54271: remote error: tls: bad certificate I0826 20:32:11.216781 1 log.go:172] http: TLS handshake error from 192.168.77.100:3078: remote error: tls: bad certificate I0826 20:32:13.056051 1 log.go:172] http: TLS handshake error from 192.168.77.100:63043: remote error: tls: bad certificate I0826 20:32:33.058411 1 log.go:172] http: TLS handshake error from 192.168.77.100:8276: remote error: tls: bad certificate I0826 20:32:33.548821 1 log.go:172] http: TLS handshake error from 192.168.77.100:11765: remote error: tls: bad certificate I0826 20:32:43.063658 1 log.go:172] http: TLS handshake error from 192.168.77.100:2310: remote error: tls: bad certificate I0826 20:32:43.574731 1 log.go:172] http: TLS handshake error from 192.168.77.100:25715: remote error: tls: bad certificate I0826 20:32:43.723291 1 log.go:172] http: TLS handshake error from 192.168.77.100:15190: remote error: tls: bad certificate I0826 20:32:45.411034 1 log.go:172] http: TLS handshake error from 192.168.77.100:59246: remote error: tls: bad certificate I0826 20:34:47.352255 1 certificate.go:182] Rotating self signed certificate I0826 20:34:47.831760 1 certificate.go:205] Generated self-signed cert (/var/run/antrea/antrea-controller-self-signed/antrea-controller.crt, /var/run/antrea/antrea-controller-self-signed/antrea-controller.key) I0826 20:34:47.831927 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:34:47.832058 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:34:47.839407 1 cacert_controller.go:124] Syncing CA certificate with APIServices W0826 20:34:47.839702 1 cacert_controller.go:172] Initial sync of CA content failed: error updating ConfigMap antrea-ca: Operation cannot be fulfilled on configmaps "antrea-ca": the object has been modified; please apply your changes to the latest version and try again I0826 20:34:47.839912 1 certificate.go:178] Certificate will be rotated at 2020-08-26 20:37:47.8399098 +0000 UTC m=+1805.915386615 I0826 20:34:47.868392 1 cacert_controller.go:143] Syncing CA certificate with ConfigMap I0826 20:34:47.872097 1 cacert_controller.go:124] Syncing CA certificate with APIServices I0826 20:34:57.392377 1 log.go:172] http: TLS handshake error from 192.168.77.100:22978: remote error: tls: bad certificate I0826 20:34:57.463302 1 log.go:172] http: TLS handshake error from 192.168.77.100:56192: remote error: tls: bad certificate I0826 20:34:57.476430 1 log.go:172] http: TLS handshake error from 192.168.77.100:45066: remote error: tls: bad certificate I0826 20:35:05.602815 1 log.go:172] http: TLS handshake error from 192.168.77.100:20567: remote error: tls: bad certificate I0826 20:35:06.096243 1 log.go:172] http: TLS handshake error from 192.168.77.100:26364: remote error: tls: bad certificate I0826 20:35:06.478006 1 log.go:172] http: TLS handshake error from 192.168.77.100:52038: remote error: tls: bad certificate I0826 20:35:06.492189 1 log.go:172] http: TLS handshake error from 192.168.77.100:61621: remote error: tls: bad certificate I0826 20:35:17.619679 1 log.go:172] http: TLS handshake error from 192.168.77.100:40796: remote error: tls: bad certificate I0826 20:35:18.130319 1 log.go:172] http: TLS handshake error from 192.168.77.100:16355: remote error: tls: bad certificate I0826 20:35:18.277044 1 log.go:172] http: TLS handshake error from 192.168.77.100:23276: remote error: tls: bad certificate I0826 20:35:20.080562 1 log.go:172] http: TLS handshake error from 192.168.77.100:48126: remote error: tls: bad certificate I0826 20:35:36.106176 1 log.go:172] http: TLS handshake error from 192.168.77.100:26924: remote error: tls: bad certificate I0826 20:35:36.271404 1 log.go:172] http: TLS handshake error from 192.168.77.100:12776: remote error: tls: bad certificate