2022-06-10T14:51:54.371Z [shell] [INFO ] [] [installerCommon.sh:1597 ] [main] - Checking open files and processes limits 2022-06-10T14:51:54.401Z [shell] [INFO ] [] [installerCommon.sh:1600 ] [main] - Current max open files is 1048576 2022-06-10T14:51:54.430Z [shell] [INFO ] [] [installerCommon.sh:1611 ] [main] - Current max open processes is unlimited [WARN ] Error while initializing File resolver : Config file does not exists : /opt/jfrog/artifactory/var/etc/system.yaml Database connection check failed Could not determine database type 2022-06-10T14:51:54.536Z [shell] [INFO ] [] [installerCommon.sh:1680 ] [main] - Testing directory /opt/jfrog/artifactory/var has read/write permissions for user id 0 2022-06-10T14:51:54.565Z [shell] [INFO ] [] [installerCommon.sh:1695 ] [main] - Permissions for /opt/jfrog/artifactory/var are good 2022-06-10T14:51:55.165Z [shell] [INFO ] [] [installerCommon.sh:3464 ] [main] - Setting JF_SHARED_NODE_ID to 7969ad763bf2 2022-06-10T14:51:55.329Z [shell] [INFO ] [] [installerCommon.sh:3464 ] [main] - Setting JF_SHARED_NODE_IP to 172.17.0.2 2022-06-10T14:51:55.499Z [shell] [INFO ] [] [installerCommon.sh:3464 ] [main] - Setting JF_SHARED_NODE_NAME to 7969ad763bf2 2022-06-10T14:51:56.868Z [shell] [INFO ] [] [artifactoryCommon.sh:384 ] [main] - Removing old custom drivers : /opt/jfrog/artifactory/app/artifactory/tomcat/lib/jf_* 2022-06-10T14:51:56.900Z [shell] [INFO ] [] [artifactoryCommon.sh:389 ] [main] - Copying /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib/README.md to /opt/jfrog/artifactory/app/artifactory/tomcat/lib/jf_README.md 2022-06-10T14:51:57.464Z [shell] [INFO ] [] [artifactoryCommon.sh:185 ] [main] - Saving /opt/jfrog/artifactory/app/artifactory/tomcat/conf/server.xml as /opt/jfrog/artifactory/app/artifactory/tomcat/conf/server.xml.orig 2022-06-10T14:51:57.492Z [shell] [INFO ] [] [artifactoryCommon.sh:193 ] [main] - Using Tomcat template to generate : /opt/jfrog/artifactory/app/artifactory/tomcat/conf/server.xml 2022-06-10T14:51:57.736Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.port||8081} to default value : 8081 2022-06-10T14:51:57.890Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.connector.sendReasonPhrase||false} to default value : false 2022-06-10T14:51:58.051Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.connector.relaxedPathChars||_SQUARE_BRACKETS_} to default value : _SQUARE_BRACKETS_ 2022-06-10T14:51:58.228Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.connector.relaxedQueryChars||_SQUARE_BRACKETS_} to default value : _SQUARE_BRACKETS_ 2022-06-10T14:51:58.409Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.connector.maxThreads||200} to default value : 200 2022-06-10T14:51:58.695Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.maintenanceConnector.port||8091} to default value : 8091 2022-06-10T14:51:58.842Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.maintenanceConnector.maxThreads||5} to default value : 5 2022-06-10T14:51:58.990Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${artifactory.tomcat.maintenanceConnector.acceptCount||5} to default value : 5 2022-06-10T14:51:59.267Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${access.http.port||8040} to default value : 8040 2022-06-10T14:51:59.424Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${access.tomcat.connector.sendReasonPhrase||false} to default value : false 2022-06-10T14:51:59.591Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${access.tomcat.connector.maxThreads||50} to default value : 50 2022-06-10T14:51:59.954Z [shell] [INFO ] [] [systemYamlHelper.sh:605 ] [main] - Resolved JF_PRODUCT_HOME (/opt/jfrog/artifactory) from environment variable 2022-06-10T14:52:00.311Z [shell] [INFO ] [] [systemYamlHelper.sh:956 ] [main] - Resolved ${shared.tomcat.workDir||/opt/jfrog/artifactory/var/work/artifactory/tomcat} to default value : /opt/jfrog/artifactory/var/work/artifactory/tomcat ======================== JF Environment variables ======================== JF_ARTIFACTORY_PID : /opt/jfrog/artifactory/app/run/artifactory.pid JF_PRODUCT_HOME : /opt/jfrog/artifactory JF_ROUTER_TOPOLOGY_LOCAL_REQUIREDSERVICETYPES : jfrt,jfac,jfmd,jffe,jfob,jfint,jfcon,jfevt JF_SHARED_NODE_NAME : 7969ad763bf2 JF_SHARED_NODE_IP : 172.17.0.2 JF_SYSTEM_YAML : /opt/jfrog/artifactory/var/etc/system.yaml JF_SHARED_NODE_ID : 7969ad763bf2 Using default router's certificate and private key Starting router... router not running. Proceed to start it up. router started. PID: 9614 2022-06-10T14:52:02.276Z [jfrou] [INFO ] [63376afd9572709f] [bootstrap.go:88 ] [main ] - Router (jfrou) service initialization started. Version: 7.38.0-1 Revision: 89f07fc2b65bad0638539945b1ea3ba9017b068b PID: 9614 Home: /opt/jfrog/artifactory 2022-06-10T14:52:02.277Z [jfrou] [INFO ] [63376afd9572709f] [bootstrap.go:91 ] [main ] - JFrog Router IP: 172.17.0.2 Starting metadata... JF_METADATA_ACCESSCLIENT_URL: http://localhost:8081/access metadata started. PID: 9801 [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [TRACE] Config key not set for aws secret (metadata.database.secretsManagerAlias) Starting event... event not running. Proceed to start it up. event started. PID: 9945 [INFO ] JFrog Event (jfevt) service initialization started. Version: 7.40.0 (revision: 363eeefbaf, build date: 2022-04-04T11:56:16Z) PID: 9945 Home: /opt/jfrog/artifactory [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml Starting frontend... frontend not running. Proceed to start it up. frontend started. PID: 10094 Starting observability... observability not running. Proceed to start it up. observability started. PID: 10237 [INFO ] JFrog Observability (jfob) service initialization started. Version: 1.5.1 (revision: 1d373c15fa, build date: 2022-03-31T10:36:20Z) PID: 10237 Home: /opt/jfrog/artifactory [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml Starting integration... integration not running. Proceed to start it up. integration started. PID: 10384 [INFO ] JFrog Integration (jfint) service initialization started. Version: 7.40.0 (revision: 4b7db20662, build date: 2022-04-04T12:27:12Z) PID: 10384 Home: /opt/jfrog/artifactory [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml Starting jfconnect jfconnect not running. Proceed to start it up. 2022-06-10T14:52:06.369Z [jffe ] [INFO ] [ ] [ ] [main ] - frontend (jffe) service initialization started. Version: 1.38.8 Revision: 10000008 PID: 10094 Home: /opt/jfrog/artifactory 2022-06-10T14:52:06.374Z [jffe ] [INFO ] [ ] [ ] [main ] - attempting pinging artifactory for 180 retires and 1.0s interval for total of 3 minutes jfconnect started. PID: 10526 [INFO ] JFConnect (jfcon) service initialization started. Version: 7.38.3 (revision: 5ef5ceb391, build date: 2022-05-02T13:56:55Z) PID: 10526 Home: /opt/jfrog/artifactory [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml 2022-06-10T14:52:07.066Z [shell] [INFO ] [] [installerCommon.sh:1273 ] [main] - Redirection is set to false. Skipping catalina log redirection Using CATALINA_BASE: /opt/jfrog/artifactory/app/artifactory/tomcat Using CATALINA_HOME: /opt/jfrog/artifactory/app/artifactory/tomcat Using CATALINA_TMPDIR: /opt/jfrog/artifactory/var/work/artifactory/tomcat/temp Using JRE_HOME: /opt/jfrog/artifactory/app/third-party/java Using CLASSPATH: /opt/jfrog/artifactory/app/artifactory/tomcat/bin/bootstrap.jar:/opt/jfrog/artifactory/app/artifactory/tomcat/bin/tomcat-juli.jar Using CATALINA_OPTS: -server -Xms512m -Xmx2g -XX:+UseG1GC -XX:OnOutOfMemoryError="kill -9 %p" --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED -Dfile.encoding=UTF8 -Djruby.compile.invokedynamic=false -Djruby.bytecode.version=1.8 -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Djava.security.egd=file:/dev/./urandom -Dartdist=zip -Djf.product.home=/opt/jfrog/artifactory -Djruby.bytecode.version=1.8 Using CATALINA_PID: /opt/jfrog/artifactory/app/run/artifactory.pid 2022-06-10T14:52:07.281Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 5 seconds with 2m0s timeout 2022-06-10T14:52:08.060Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 5 seconds with 2m0s timeout [init] 2022-06-10T14:52:08.665Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 5 seconds with 2m0s timeout [startup] 2022-06-10T14:52:10.039Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 5 seconds with 2m0s timeout [startup] 2022-06-10T14:52:10.904Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 5 seconds with 2m0s timeout [startup] 2022-06-10T14:52:11.668Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 5 seconds with 2m0s timeout [startup] 2022-06-10T14:52:12.301Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 10 seconds with 2m0s timeout 2022-06-10T14:52:13.076Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 10 seconds with 2m0s timeout [init] 2022-06-10T14:52:13.694Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 10 seconds with 2m0s timeout [startup] 2022-06-10T14:52:15.051Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 10 seconds with 2m0s timeout [startup] 2022-06-10T14:52:15.523Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 10 2022-06-10T14:52:15.533Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 10 failed with code : ECONNREFUSED 2022-06-10T14:52:15.931Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 10 seconds with 2m0s timeout [startup] 2022-06-10T14:52:16.682Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 10 seconds with 2m0s timeout [startup] 2022-06-10T14:52:16.932Z [jfrt ] [INFO ] [c5d1ec30098a5752] [o.a.c.h.HaNodeProperties:65 ] [Catalina-utility-1 ] - Artifactory is running in non-clustered mode. 2022-06-10T14:52:17.018Z [jfrt ] [INFO ] [c5d1ec30098a5752] [tifactoryHomeConfigListener:85] [Catalina-utility-1 ] - Resolved Home: '/opt/jfrog/artifactory 2022-06-10T14:52:17.324Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 15 seconds with 2m0s timeout 2022-06-10T14:52:18.100Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 15 seconds with 2m0s timeout [init] 2022-06-10T14:52:18.612Z [jfrt ] [INFO ] [ ] [o.j.c.w.FileWatcher:146 ] [file-watcher-poller ] - Starting watch of folder configurations 2022-06-10T14:52:18.740Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 15 seconds with 2m0s timeout [startup] 2022-06-10T14:52:18.951Z [jfrt ] [WARN ] [c5d1ec30098a5752] [c.z.h.u.DriverDataSource:70 ] [Catalina-utility-1 ] - Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation. 2022-06-10T14:52:19.896Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [licationContextInitializer:162] [Catalina-utility-2 ] - Access (jfac) service initialization started. Version: 7.43.6 Revision: 74306900 PID: 10694 Home: /opt/jfrog/artifactory 2022-06-10T14:52:20.023Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [o.j.a.AccessApplication:55 ] [Catalina-utility-2 ] - Starting AccessApplication v7.43.6 using Java 11.0.13 on 7969ad763bf2 with PID 10694 (/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/access/WEB-INF/lib/access-application-7.43.6.jar started by root in /opt/jfrog/artifactory/var) 2022-06-10T14:52:20.025Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [o.j.a.AccessApplication:681 ] [Catalina-utility-2 ] - The following 1 profile is active: "production" 2022-06-10T14:52:20.069Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 15 seconds with 2m0s timeout [startup] 2022-06-10T14:52:20.935Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 15 seconds with 2m0s timeout [startup] 2022-06-10T14:52:21.698Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 15 seconds with 2m0s timeout [startup] 2022-06-10T14:52:22.335Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 20 seconds with 2m0s timeout 2022-06-10T14:52:23.117Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 20 seconds with 2m0s timeout [init] 2022-06-10T14:52:23.753Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 20 seconds with 2m0s timeout [startup] 2022-06-10T14:52:25.089Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 20 seconds with 2m0s timeout [startup] 2022-06-10T14:52:25.729Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 20 2022-06-10T14:52:25.754Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 20 failed with code : ECONNREFUSED 2022-06-10T14:52:25.940Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 20 seconds with 2m0s timeout [startup] 2022-06-10T14:52:26.725Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 20 seconds with 2m0s timeout [startup] 2022-06-10T14:52:27.352Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 25 seconds with 2m0s timeout 2022-06-10T14:52:28.128Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 25 seconds with 2m0s timeout [init] 2022-06-10T14:52:28.771Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 25 seconds with 2m0s timeout [startup] 2022-06-10T14:52:30.110Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 25 seconds with 2m0s timeout [startup] 2022-06-10T14:52:30.720Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [c.z.h.u.DriverDataSource:70 ] [Catalina-utility-2 ] - Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation. 2022-06-10T14:52:30.946Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 25 seconds with 2m0s timeout [startup] 2022-06-10T14:52:31.747Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 25 seconds with 2m0s timeout [startup] 2022-06-10T14:52:32.358Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 30 seconds with 2m0s timeout 2022-06-10T14:52:33.134Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 30 seconds with 2m0s timeout [init] 2022-06-10T14:52:33.794Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 30 seconds with 2m0s timeout [startup] 2022-06-10T14:52:35.122Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 30 seconds with 2m0s timeout [startup] 2022-06-10T14:52:35.889Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 30 2022-06-10T14:52:35.901Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 30 failed with code : ECONNREFUSED 2022-06-10T14:52:35.950Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 30 seconds with 2m0s timeout [startup] 2022-06-10T14:52:36.754Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 30 seconds with 2m0s timeout [startup] 2022-06-10T14:52:37.363Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 35 seconds with 2m0s timeout 2022-06-10T14:52:38.138Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 35 seconds with 2m0s timeout [init] 2022-06-10T14:52:38.797Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 35 seconds with 2m0s timeout [startup] 2022-06-10T14:52:40.102Z [jfrt ] [ERROR] [c5d1ec30098a5752] [c.z.h.p.HikariPool:593 ] [Catalina-utility-1 ] - HikariCP Main - Exception during pool initialization. java.sql.SQLTimeoutException: Login timeout exceeded. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source) at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source) at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:121) at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358) at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477) at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:560) at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115) at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) at org.jfrog.storage.wrapper.JFrogDataSourceWrapper.getConnection(JFrogDataSourceWrapper.java:102) at org.jfrog.storage.wrapper.JFrogDataSourceWrapper.invoke(JFrogDataSourceWrapper.java:71) at com.sun.proxy.$Proxy20.getConnection(Unknown Source) at org.jfrog.storage.util.DbUtils.withMetadata(DbUtils.java:691) at org.jfrog.storage.util.DbUtils.tableExists(DbUtils.java:273) at org.artifactory.common.config.db.DbVersionDao.isDbPropertiesTableExists(DbVersionDao.java:62) at org.artifactory.converter.VersionProviderImpl.tryToResolveFromDb(VersionProviderImpl.java:197) at org.artifactory.converter.VersionProviderImpl.getCompoundVersionDetailsFromDbIfExists(VersionProviderImpl.java:122) at org.artifactory.converter.VersionProviderImpl.loadOriginalDbVersion(VersionProviderImpl.java:143) at org.artifactory.converter.VersionProviderImpl.init(VersionProviderImpl.java:104) at org.artifactory.lifecycle.webapp.servlet.BasicConfigurationManager.initVersion(BasicConfigurationManager.java:160) at org.artifactory.lifecycle.webapp.servlet.BasicConfigurationManager.initArtifactoryInstallation(BasicConfigurationManager.java:139) at org.artifactory.lifecycle.webapp.servlet.BasicConfigurationManager.initialize(BasicConfigurationManager.java:126) at org.artifactory.lifecycle.webapp.servlet.ArtifactoryHomeConfigListener.initBasicConfigManager(ArtifactoryHomeConfigListener.java:61) at org.artifactory.lifecycle.webapp.servlet.ArtifactoryHomeConfigListener.contextInitialized(ArtifactoryHomeConfigListener.java:53) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4768) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5230) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:690) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1889) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: org.apache.derby.iapi.error.StandardException: Login timeout exceeded. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source) ... 45 common frames omitted 2022-06-10T14:52:40.106Z [jfrt ] [ERROR] [c5d1ec30098a5752] [o.a.c.VersionProviderImpl:108 ] [Catalina-utility-1 ] - Failed to resolve version information: Could not find database table: db_properties 2022-06-10T14:52:40.107Z [jfrt ] [ERROR] [c5d1ec30098a5752] [tifactoryHomeConfigListener:55] [Catalina-utility-1 ] - Failed initializing Home. Caught exception: java.lang.IllegalStateException: Could not find database table: db_properties at org.artifactory.converter.VersionProviderImpl.init(VersionProviderImpl.java:109) at org.artifactory.lifecycle.webapp.servlet.BasicConfigurationManager.initVersion(BasicConfigurationManager.java:160) at org.artifactory.lifecycle.webapp.servlet.BasicConfigurationManager.initArtifactoryInstallation(BasicConfigurationManager.java:139) at org.artifactory.lifecycle.webapp.servlet.BasicConfigurationManager.initialize(BasicConfigurationManager.java:126) at org.artifactory.lifecycle.webapp.servlet.ArtifactoryHomeConfigListener.initBasicConfigManager(ArtifactoryHomeConfigListener.java:61) at org.artifactory.lifecycle.webapp.servlet.ArtifactoryHomeConfigListener.contextInitialized(ArtifactoryHomeConfigListener.java:53) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4768) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5230) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:690) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1889) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.IllegalStateException: Could not find database table: db_properties at org.artifactory.common.config.db.DbVersionDao.isDbPropertiesTableExists(DbVersionDao.java:65) at org.artifactory.converter.VersionProviderImpl.tryToResolveFromDb(VersionProviderImpl.java:197) at org.artifactory.converter.VersionProviderImpl.getCompoundVersionDetailsFromDbIfExists(VersionProviderImpl.java:122) at org.artifactory.converter.VersionProviderImpl.loadOriginalDbVersion(VersionProviderImpl.java:143) at org.artifactory.converter.VersionProviderImpl.init(VersionProviderImpl.java:104) ... 20 common frames omitted Caused by: java.sql.SQLTimeoutException: Login timeout exceeded. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.jdbc.InternalDriver.timeLogin(Unknown Source) at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) at org.apache.derby.jdbc.EmbeddedDriver.connect(Unknown Source) at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:121) at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358) at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477) at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:560) at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:115) at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) at org.jfrog.storage.wrapper.JFrogDataSourceWrapper.getConnection(JFrogDataSourceWrapper.java:102) at org.jfrog.storage.wrapper.JFrogDataSourceWrapper.invoke(JFrogDataSourceWrapper.java:71) at com.sun.proxy.$Proxy20.getConnection(Unknown Source) at org.jfrog.storage.util.DbUtils.withMetadata(DbUtils.java:691) at org.jfrog.storage.util.DbUtils.tableExists(DbUtils.java:273) at org.artifactory.common.config.db.DbVersionDao.isDbPropertiesTableExists(DbVersionDao.java:62) ... 24 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: Login timeout exceeded. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown Source) ... 45 common frames omitted 2022-06-10T14:52:40.126Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 35 seconds with 2m0s timeout [startup] 2022-06-10T14:52:40.126Z [jfrt ] [ERROR] [c5d1ec30098a5752] [actoryContextConfigListener:90] [Catalina-utility-1 ] - Failed initializing Artifactory context: Artifactory home not initialized. 2022-06-10T14:52:40.954Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 35 seconds with 2m0s timeout [startup] 2022-06-10T14:52:41.759Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 35 seconds with 2m0s timeout [startup] 2022-06-10T14:52:42.367Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 40 seconds with 2m0s timeout 2022-06-10T14:52:43.143Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 40 seconds with 2m0s timeout [init] 2022-06-10T14:52:43.800Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 40 seconds with 2m0s timeout [startup] 2022-06-10T14:52:45.132Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 40 seconds with 2m0s timeout [startup] 2022-06-10T14:52:45.960Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 40 seconds with 2m0s timeout [startup] 2022-06-10T14:52:45.985Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 40 2022-06-10T14:52:45.989Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 40 failed with code : ECONNREFUSED 2022-06-10T14:52:46.762Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 40 seconds with 2m0s timeout [startup] 2022-06-10T14:52:47.370Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 45 seconds with 2m0s timeout 2022-06-10T14:52:48.146Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 45 seconds with 2m0s timeout [init] 2022-06-10T14:52:48.807Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 45 seconds with 2m0s timeout [startup] 2022-06-10T14:52:50.136Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 45 seconds with 2m0s timeout [startup] 2022-06-10T14:52:50.975Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 45 seconds with 2m0s timeout [startup] 2022-06-10T14:52:51.765Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 45 seconds with 2m0s timeout [startup] 2022-06-10T14:52:52.391Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 50 seconds with 2m0s timeout 2022-06-10T14:52:53.168Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 50 seconds with 2m0s timeout [init] 2022-06-10T14:52:53.814Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 50 seconds with 2m0s timeout [startup] 2022-06-10T14:52:55.143Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 50 seconds with 2m0s timeout [startup] 2022-06-10T14:52:55.980Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 50 seconds with 2m0s timeout [startup] 2022-06-10T14:52:56.096Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 50 2022-06-10T14:52:56.102Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 50 failed with code : ECONNREFUSED 2022-06-10T14:52:56.770Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 50 seconds with 2m0s timeout [startup] 2022-06-10T14:52:57.394Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 55 seconds with 2m0s timeout 2022-06-10T14:52:58.172Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 55 seconds with 2m0s timeout [init] 2022-06-10T14:52:58.818Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 55 seconds with 2m0s timeout [startup] 2022-06-10T14:53:00.147Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 55 seconds with 2m0s timeout [startup] 2022-06-10T14:53:00.985Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 55 seconds with 2m0s timeout [startup] 2022-06-10T14:53:01.774Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 55 seconds with 2m0s timeout [startup] 2022-06-10T14:53:02.399Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 60 seconds with 2m0s timeout 2022-06-10T14:53:03.176Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 60 seconds with 2m0s timeout [init] 2022-06-10T14:53:03.418Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [s.d.u.AccessJdbcHelperImpl:157] [Catalina-utility-2 ] - Database: Apache Derby 10.14.2.0 - (1828579). Driver: Apache Derby Embedded JDBC Driver 10.14.2.0 - (1828579) 2022-06-10T14:53:03.823Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 60 seconds with 2m0s timeout [startup] 2022-06-10T14:53:04.165Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [s.d.u.AccessJdbcHelperImpl:168] [Catalina-utility-2 ] - ***Creating database schema*** 2022-06-10T14:53:05.156Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 60 seconds with 2m0s timeout [startup] 2022-06-10T14:53:06.000Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 60 seconds with 2m0s timeout [startup] 2022-06-10T14:53:06.192Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 60 2022-06-10T14:53:06.197Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 60 failed with code : ECONNREFUSED 2022-06-10T14:53:06.778Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 60 seconds with 2m0s timeout [startup] 2022-06-10T14:53:07.412Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 65 seconds with 2m0s timeout 2022-06-10T14:53:08.181Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 65 seconds with 2m0s timeout [init] 2022-06-10T14:53:08.828Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 65 seconds with 2m0s timeout [startup] 2022-06-10T14:53:10.160Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 65 seconds with 2m0s timeout [startup] 2022-06-10T14:53:11.005Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 65 seconds with 2m0s timeout [startup] 2022-06-10T14:53:11.783Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 65 seconds with 2m0s timeout [startup] 2022-06-10T14:53:12.416Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 70 seconds with 2m0s timeout 2022-06-10T14:53:13.184Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 70 seconds with 2m0s timeout [init] 2022-06-10T14:53:13.832Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 70 seconds with 2m0s timeout [startup] 2022-06-10T14:53:15.164Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 70 seconds with 2m0s timeout [startup] 2022-06-10T14:53:16.009Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 70 seconds with 2m0s timeout [startup] 2022-06-10T14:53:16.273Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 70 2022-06-10T14:53:16.288Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 70 failed with code : ECONNREFUSED 2022-06-10T14:53:16.788Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 70 seconds with 2m0s timeout [startup] 2022-06-10T14:53:17.420Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 75 seconds with 2m0s timeout 2022-06-10T14:53:18.189Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 75 seconds with 2m0s timeout [init] 2022-06-10T14:53:18.835Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 75 seconds with 2m0s timeout [startup] 2022-06-10T14:53:20.168Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 75 seconds with 2m0s timeout [startup] 2022-06-10T14:53:21.013Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 75 seconds with 2m0s timeout [startup] 2022-06-10T14:53:21.794Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 75 seconds with 2m0s timeout [startup] 2022-06-10T14:53:22.423Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 80 seconds with 2m0s timeout 2022-06-10T14:53:23.192Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 80 seconds with 2m0s timeout [init] 2022-06-10T14:53:23.838Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 80 seconds with 2m0s timeout [startup] 2022-06-10T14:53:25.172Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 80 seconds with 2m0s timeout [startup] 2022-06-10T14:53:26.018Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 80 seconds with 2m0s timeout [startup] 2022-06-10T14:53:26.358Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 80 2022-06-10T14:53:26.363Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 80 failed with code : ECONNREFUSED 2022-06-10T14:53:26.802Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 80 seconds with 2m0s timeout [startup] 2022-06-10T14:53:27.426Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 85 seconds with 2m0s timeout 2022-06-10T14:53:28.196Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 85 seconds with 2m0s timeout [init] 2022-06-10T14:53:28.842Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 85 seconds with 2m0s timeout [startup] 2022-06-10T14:53:30.177Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 85 seconds with 2m0s timeout [startup] 2022-06-10T14:53:31.027Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 85 seconds with 2m0s timeout [startup] 2022-06-10T14:53:31.805Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 85 seconds with 2m0s timeout [startup] 2022-06-10T14:53:32.430Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 90 seconds with 2m0s timeout 2022-06-10T14:53:33.203Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 90 seconds with 2m0s timeout [init] 2022-06-10T14:53:33.845Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 90 seconds with 2m0s timeout [startup] 2022-06-10T14:53:35.184Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 90 seconds with 2m0s timeout [startup] 2022-06-10T14:53:36.031Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 90 seconds with 2m0s timeout [startup] 2022-06-10T14:53:36.444Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 90 2022-06-10T14:53:36.450Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 90 failed with code : ECONNREFUSED 2022-06-10T14:53:36.810Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 90 seconds with 2m0s timeout [startup] 2022-06-10T14:53:37.393Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [s.d.u.AccessJdbcHelperImpl:184] [Catalina-utility-2 ] - Setting new schema baseline to latest version: 7.40.1.0 2022-06-10T14:53:37.438Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 95 seconds with 2m0s timeout 2022-06-10T14:53:38.207Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 95 seconds with 2m0s timeout [init] 2022-06-10T14:53:38.848Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 95 seconds with 2m0s timeout [startup] 2022-06-10T14:53:40.188Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 95 seconds with 2m0s timeout [startup] 2022-06-10T14:53:40.406Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [alConfigurationServiceBase:185] [Catalina-utility-2 ] - Configuration data wasn't found in db. Loading defaults 2022-06-10T14:53:41.042Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 95 seconds with 2m0s timeout [startup] 2022-06-10T14:53:41.359Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [alConfigurationServiceBase:125] [Catalina-utility-2 ] - Current configurations are the same as the new configurations, no need for an update. No action was taken. 2022-06-10T14:53:41.819Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 95 seconds with 2m0s timeout [startup] 2022-06-10T14:53:42.458Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 100 seconds with 2m0s timeout 2022-06-10T14:53:42.548Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.h.AccessNodeIdProviderImpl:73] [Catalina-utility-2 ] - Service id initialized: jfac@01g571pmkscx5c0q2wmcqy0gez 2022-06-10T14:53:43.002Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [a.k.m.MasterKeyBootstrapper:77] [Catalina-utility-2 ] - No master.key supplied. Generating master.key 2022-06-10T14:53:43.224Z [jfmd ] [INFO ] [ ] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 100 seconds with 2m0s timeout [init] 2022-06-10T14:53:43.471Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.k.m.MasterKeyBootstrapper:151] [Catalina-utility-2 ] - Saving master.key to /opt/jfrog/artifactory/var/etc/security/master.key 2022-06-10T14:53:43.862Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:163 ] [main ] - Master key is missing. Pending for 100 seconds with 2m0s timeout [startup] 2022-06-10T14:53:44.151Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.s.s.s.ServiceConfigFactory:42] [Catalina-utility-2 ] - Initializing task scheduler with 9 threads 2022-06-10T14:53:44.230Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [j.a.s.s.t.TokenServiceImpl:165] [Catalina-utility-2 ] - Scheduling task for revoking expired tokens using cron expression: 59 30 0/1 * * ? 2022-06-10T14:53:44.538Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [b.AccessServerBootstrapImpl:54] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Starting JFrog Access bootstrap... 2022-06-10T14:53:45.011Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.a.s.d.s.c.v.DerbyValidator:26] [Catalina-utility-2 ] - Validating connection collation for derby database 2022-06-10T14:53:45.013Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [tCertificateFileHandlerBase:68] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Initializing root certificate. 2022-06-10T14:53:45.018Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [CertificateFileHandlerBase:182] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Generating root private key and certificate. 2022-06-10T14:53:45.200Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 100 seconds with 2m0s timeout [startup] 2022-06-10T14:53:46.058Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 100 seconds with 2m0s timeout [startup] 2022-06-10T14:53:46.573Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 100 2022-06-10T14:53:46.587Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 100 failed with code : ECONNREFUSED 2022-06-10T14:53:46.831Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 100 seconds with 2m0s timeout [startup] 2022-06-10T14:53:47.473Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 105 seconds with 2m0s timeout 2022-06-10T14:53:47.927Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [CertificateFileHandlerBase:160] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Saved new root certificate at: /opt/jfrog/artifactory/var/etc/access/keys/root.crt 2022-06-10T14:53:47.940Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [tCertificateFileHandlerBase:85] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Finished initializing root certificate. Certificate source: GENERATED 2022-06-10T14:53:47.942Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [tCertificateFileHandlerBase:68] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Initializing ca certificate. 2022-06-10T14:53:47.946Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [CertificateFileHandlerBase:182] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Generating ca private key and certificate. 2022-06-10T14:53:48.236Z [jfmd ] [INFO ] [ ] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 105 seconds with 2m0s timeout [init] 2022-06-10T14:53:48.520Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [CertificateFileHandlerBase:160] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Saved new ca certificate at: /opt/jfrog/artifactory/var/etc/access/keys/ca.crt 2022-06-10T14:53:48.521Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [tCertificateFileHandlerBase:85] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Finished initializing ca certificate. Certificate source: GENERATED 2022-06-10T14:53:48.868Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 105 seconds with 2m0s timeout [startup] 2022-06-10T14:53:49.008Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [c.z.h.u.DriverDataSource:70 ] [Catalina-utility-2 ] - Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation. 2022-06-10T14:53:49.903Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.s.b.AccessProjectBootstrap:75] [Catalina-utility-2 ] - Finished initializing Projects Entities in 75.44 millis 2022-06-10T14:53:49.911Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.s.b.AccessProjectBootstrap:68] [Catalina-utility-2 ] - Finished initializing Projects Shared Entities in 6.49 millis 2022-06-10T14:53:50.100Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.s.b.AccessProjectBootstrap:88] [Catalina-utility-2 ] - Finished initializing Projects permissions in 186.01 millis 2022-06-10T14:53:50.103Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.s.b.AccessProjectBootstrap:60] [Catalina-utility-2 ] - Finished initializing Projects in 279.99 millis 2022-06-10T14:53:50.103Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.a.s.s.m.MetricsServiceImpl:55] [Catalina-utility-2 ] - Metrics Framework Service is disabled 2022-06-10T14:53:50.106Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [o.j.m.MetricsHelper:56 ] [Catalina-utility-2 ] - Initializing Metrics collector and logger for feature adoption... 2022-06-10T14:53:50.107Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.FeatureAdoptionServiceImpl:47] [Catalina-utility-2 ] - FeatureAdoption Framework service is started and is running in background... 2022-06-10T14:53:50.108Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [b.AccessServerBootstrapImpl:66] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] JFrog Access bootstrap finished. 2022-06-10T14:53:50.216Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 105 seconds with 2m0s timeout [startup] 2022-06-10T14:53:50.342Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.a.s.JfConnectConfiguration:40] [Catalina-utility-2 ] - JFCON entitlements are disabled. jfconnect.enabled is false or undefined 2022-06-10T14:53:51.065Z [jfint] [INFO ] [605b2e97386125fc] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 105 seconds with 2m0s timeout [startup] 2022-06-10T14:53:51.842Z [jfcon] [INFO ] [40e550fc80ab73a0] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 105 seconds with 2m0s timeout [startup] 2022-06-10T14:53:52.480Z [jfrou] [INFO ] [63376afd9572709f] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 110 seconds with 2m0s timeout 2022-06-10T14:53:53.240Z [jfmd ] [INFO ] [ ] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 110 seconds with 2m0s timeout [init] 2022-06-10T14:53:53.883Z [jfevt] [INFO ] [5461784fd2183141] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 110 seconds with 2m0s timeout [startup] 2022-06-10T14:53:54.615Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [o.j.a.s.r.s.GrpcServerImpl:65 ] [Catalina-utility-2 ] - Starting gRPC Server on port 8045 2022-06-10T14:53:55.230Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [security_keys.go:177 ] [main ] - Cluster join: Join key is missing. Pending for 110 seconds with 2m0s timeout [startup] 2022-06-10T14:53:55.632Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [o.j.a.s.r.s.GrpcServerImpl:84 ] [Catalina-utility-2 ] - gRPC Server started, listening on 8045 2022-06-10T14:53:55.703Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [o.j.a.s.s.JoinKeyAccess:172 ] [Catalina-utility-2 ] - Cluster join: Join key loaded successfully (from: generated) 2022-06-10T14:53:55.823Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [a.s.b.AccessServerRegistrar:70] [Catalina-utility-2 ] - [ACCESS BOOTSTRAP] Starting JFrog Access registrar... 2022-06-10T14:53:55.910Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [a.c.RefreshableScheduledJob:53] [Catalina-utility-2 ] - Scheduling EmailsLogCleanup task to run every 86400 seconds 2022-06-10T14:53:55.984Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [a.c.RefreshableScheduledJob:53] [Catalina-utility-2 ] - Scheduling staleTokenCleanup task to run every 3600 seconds 2022-06-10T14:53:56.031Z [jfac ] [INFO ] [3de37130bd294397] [a.c.RefreshableScheduledJob:53] [Catalina-utility-2 ] - Scheduling loadCertificates task to run every 30 seconds 2022-06-10T14:53:56.104Z [jfac ] [INFO ] [ ] [.j.a.s.s.SystemScheduledJob:37] [Catalina-utility-2 ] - Scheduling Readiness task to run with 15000 milliseconds fixed delay 2022-06-10T14:53:56.215Z [jfac ] [INFO ] [d61d26e2713c6a42] [a.c.RefreshableScheduledJob:53] [Catalina-utility-2 ] - Scheduling heartbeat task to run every 5 seconds 2022-06-10T14:53:56.245Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:125 ] [main ] - Resolving service Id... [startup] 2022-06-10T14:53:56.261Z [jfmd ] [INFO ] [ ] [database_bearer.go:126 ] [main ] - Connecting to (db config: {sqlite /opt/jfrog/artifactory/var/data/metadata/sqlite.db?_fk=yes&_busy_timeout=5000&_txlock=immediate&cache=shared}) [database] 2022-06-10T14:53:56.264Z [jfac ] [INFO ] [ ] [.j.a.s.s.SystemScheduledJob:37] [Catalina-utility-2 ] - Scheduling KeepAliveFederationServers task to run with 86400000 milliseconds fixed delay 2022-06-10T14:53:56.271Z [jfac ] [INFO ] [ ] [a.c.RefreshableScheduledJob:53] [Catalina-utility-2 ] - Scheduling federationCleanupService task to run every 1209600 seconds 2022-06-10T14:53:56.288Z [jfmd ] [INFO ] [ ] [migrator.go:63 ] [main ] - Applying migration files [database] 2022-06-10T14:53:56.504Z [jfrou] [INFO ] [63376afd9572709f] [bootstrap.go:204 ] [main ] - System configuration encryption report: shared.newrelic.licenseKey: does not exist in the config file shared.security.joinKeyFile: file '/opt/jfrog/artifactory/var/etc/security/join.key' - already encrypted 2022-06-10T14:53:56.505Z [jfrou] [INFO ] [63376afd9572709f] [bootstrap.go:96 ] [main ] - JFrog Router Service ID: jfrou@01g571q2csz4h63exja1rs9rvm 2022-06-10T14:53:56.506Z [jfrou] [INFO ] [63376afd9572709f] [bootstrap.go:97 ] [main ] - JFrog Router Node ID: 7969ad763bf2 2022-06-10T14:53:56.591Z [jfrou] [INFO ] [63376afd9572709f] [trusted.go:36 ] [main ] - System cert pool contents were loaded as trusted CAs for TLS communication 2022-06-10T14:53:56.591Z [jfrou] [INFO ] [63376afd9572709f] [trusted.go:58 ] [main ] - No certificate to be loaded as trusted CAs found in [] 2022-06-10T14:53:56.592Z [jfrou] [INFO ] [63376afd9572709f] [trusted.go:36 ] [main ] - System cert pool contents were loaded as trusted CAs for TLS communication 2022-06-10T14:53:56.592Z [jfrou] [INFO ] [63376afd9572709f] [trusted.go:58 ] [main ] - No certificate to be loaded as trusted CAs found in [] [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Config file parameter jfconnect.logging.application.level is updated [INFO ] Configuration update detected [DEBUG] Running config update listener 'Configure Application Logger' [DEBUG] Setting logger with parameters: {Application:{Console:true FilePath:jfconnect-service.log Level:info Format:jftext Caller:false Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10} Packages:map[]} Metrics:{Console:false FilePath:jfconnect-metrics.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}} Request:{Console:false FilePath:jfconnect-request.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}}} [DEBUG] Successfully ran config update listener 'Configure Application Logger' 2022-06-10T14:53:56.736Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 110 2022-06-10T14:53:56.741Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 110 failed with code : ECONNREFUSED 2022-06-10T14:53:56.820Z [jfmd ] [INFO ] [ ] [migrator.go:164 ] [main ] - Migration lock table created or is already present [database] 2022-06-10T14:53:56.820Z [jfmd ] [INFO ] [ ] [migration_lock_manager.go:49 ] [main ] - Trying to acquire migration lock for 7969ad763bf2 [database] [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Config file parameter observability.logging.application.level is updated [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Config file parameter metadata.logging.application.level is updated [INFO ] Configuration update detected [DEBUG] Running config update listener 'Configure Application Logger' [DEBUG] Setting logger with parameters: {Application:{Console:true FilePath:observability-service.log Level:info Format:jftext Caller:false Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10} Packages:map[]} Metrics:{Console:false FilePath:observability-metrics.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}} Request:{Console:false FilePath:observability-request.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}}} [DEBUG] Successfully ran config update listener 'Configure Application Logger' 2022-06-10T14:53:57.195Z [jfmd ] [INFO ] [ ] [migration_lock_manager.go:70 ] [main ] - Migration lock acquired for 7969ad763bf2 [database] [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Config file parameter event.logging.application.level is updated [INFO ] Configuration update detected [DEBUG] Running config update listener 'Configure Application Logger' [DEBUG] Setting logger with parameters: {Application:{Console:true FilePath:event-service.log Level:info Format:jftext Caller:false Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10} Packages:map[]} Metrics:{Console:false FilePath:event-metrics.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}} Request:{Console:false FilePath:event-request.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}} ConsoleLog:{Format:}} [DEBUG] Successfully ran config update listener 'Configure Application Logger' [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml [DEBUG] Config file parameter integration.logging.application.level is updated [INFO ] Configuration update detected [DEBUG] Running config update listener 'Configure Application Logger' [DEBUG] Setting logger with parameters: {Application:{Console:true FilePath:integration-service.log Level:info Format:jftext Caller:false Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10} Packages:map[]} Metrics:{Console:false FilePath:integration-metrics.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}} Request:{Console:false FilePath:integration-request.log Rotation:{Compress:true KeepLastDecompressed:1 MaxSizeMb:25 MaxAgeDays:365 MaxFiles:10}} ConsoleLog:{Format:}} [DEBUG] Successfully ran config update listener 'Configure Application Logger' 2022-06-10T14:53:58.244Z [jfmd ] [INFO ] [ ] [migrator.go:130 ] [main ] - Applying 47 migrations files [database] 2022-06-10T14:53:58.245Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v001_packages.sql successfully applied (4 statements) [database] 2022-06-10T14:53:58.279Z [jfrou] [INFO ] [63376afd9572709f] [config_holder.go:110 ] [main ] - Configuration update detected 2022-06-10T14:53:59.012Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v002_versions.sql successfully applied (5 statements) [database] 2022-06-10T14:53:59.868Z [jfcon] [INFO ] [40e550fc80ab73a0] [access_thin_client.go:102 ] [main ] - Cluster join: Retry 5: Service registry ping failed, will retry. Error: failed connecting to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup] 2022-06-10T14:53:59.910Z [jfevt] [INFO ] [5461784fd2183141] [access_thin_client.go:104 ] [main ] - Cluster join: Retry 5: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup] 2022-06-10T14:54:00.095Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v003_package_qualifiers.sql successfully applied (2 statements) [database] 2022-06-10T14:54:00.125Z [jfint] [INFO ] [605b2e97386125fc] [access_thin_client.go:96 ] [main ] - Cluster join: Retry 5: Service registry ping failed, will retry. Error: error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup] 2022-06-10T14:54:00.971Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v004_package_user_properties.sql successfully applied (3 statements) [database] 2022-06-10T14:54:01.060Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-1 ] - Retry 10 Elapsed 5.23 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again 2022-06-10T14:54:01.827Z [jfac ] [INFO ] [ ] [o.j.a.AccessApplication:61 ] [Catalina-utility-2 ] - Started AccessApplication in 107.009 seconds (JVM running for 114.566) 2022-06-10T14:54:01.845Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v005_package_tags.sql successfully applied (2 statements) [database] 2022-06-10T14:54:03.546Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v006_package_stats.sql successfully applied (1 statements) [database] 2022-06-10T14:54:04.452Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v007_sources.sql successfully applied (2 statements) [database] 2022-06-10T14:54:04.959Z [jfevt] [INFO ] [5461784fd2183141] [access_thin_client.go:104 ] [main ] - Cluster join: Retry 10: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup] 2022-06-10T14:54:04.960Z [jfcon] [INFO ] [40e550fc80ab73a0] [access_thin_client.go:102 ] [main ] - Cluster join: Retry 10: Service registry ping failed, will retry. Error: failed connecting to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup] 2022-06-10T14:54:05.139Z [jfint] [INFO ] [605b2e97386125fc] [access_thin_client.go:96 ] [main ] - Cluster join: Retry 10: Service registry ping failed, will retry. Error: error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp 127.0.0.1:8046: connect: connection refused [startup] 2022-06-10T14:54:05.225Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v008_package_info_sources.sql successfully applied (3 statements) [database] 2022-06-10T14:54:05.545Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v009_servers.sql successfully applied (2 statements) [database] 2022-06-10T14:54:05.929Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v010_version_qualifiers.sql successfully applied (3 statements) [database] 2022-06-10T14:54:06.121Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-1 ] - Retry 20 Elapsed 10.29 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again 2022-06-10T14:54:06.288Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v011_version_user_properties.sql successfully applied (3 statements) [database] 2022-06-10T14:54:06.554Z [jfrou] [INFO ] [63376afd9572709f] [join_executor.go:117 ] [main ] - Cluster join: Trying to rejoin the cluster 2022-06-10T14:54:06.716Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v012_version_tags.sql successfully applied (3 statements) [database] 2022-06-10T14:54:06.890Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 120 2022-06-10T14:54:06.900Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 120 failed with code : ECONNREFUSED 2022-06-10T14:54:07.085Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v013_version_stats.sql successfully applied (1 statements) [database] 2022-06-10T14:54:07.514Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v014_licenses.sql successfully applied (2 statements) [database] 2022-06-10T14:54:07.993Z [jfac ] [INFO ] [7e6e97558770dc61] [a.c.RefreshableScheduledJob:53] [27.0.0.1-8040-exec-1] - Scheduling federationCleanupService task to run every 1209600 seconds 2022-06-10T14:54:08.000Z [jfac ] [INFO ] [6bc5f5b1367e2d0c] [.f.FederationCleanupService:59] [jf-access-task9 ] - Running clean up of outdated Federation events 2022-06-10T14:54:08.034Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v015_files.sql successfully applied (5 statements) [database] 2022-06-10T14:54:08.608Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v016_file_qualifiers.sql successfully applied (3 statements) [database] 2022-06-10T14:54:08.920Z [jfac ] [INFO ] [7e6e97558770dc61] [s.r.NodeRegistryServiceImpl:68] [27.0.0.1-8040-exec-1] - Cluster join: Successfully joined jfrou@01g571q2csz4h63exja1rs9rvm with node id 7969ad763bf2 2022-06-10T14:54:09.158Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v017_package_licenses.sql successfully applied (3 statements) [database] 2022-06-10T14:54:09.227Z [jfrou] [INFO ] [63376afd9572709f] [join_executor.go:206 ] [main ] - Cluster join: Successfully joined the cluster 2022-06-10T14:54:09.230Z [jfrou] [INFO ] [63376afd9572709f] [trusted.go:36 ] [main ] - System cert pool contents were loaded as trusted CAs for TLS communication 2022-06-10T14:54:09.230Z [jfrou] [INFO ] [63376afd9572709f] [trusted.go:56 ] [main ] - Following certificates were successfully loaded as trusted CAs for TLS communication: [/opt/jfrog/artifactory/var/data/router/keys/trusted/access-root-ca.crt] 2022-06-10T14:54:09.481Z [jfrou] [INFO ] [63376afd9572709f] [mtls_client.go:221 ] [main ] - Detected mTLS configuration change, resetting users and certificates cache expiry to 1m40s 2022-06-10T14:54:09.487Z [jfrou] [INFO ] [63376afd9572709f] [registry_handler.go:77 ] [main ] - Persisted local services: [] 2022-06-10T14:54:09.490Z [jfrou] [INFO ] [ ] [logger.go:58 ] [main ] - Starting provider aggregator.ProviderAggregator {} 2022-06-10T14:54:09.491Z [jfrou] [INFO ] [ ] [logger.go:58 ] [main ] - Starting provider *file.Provider {"directory":"/opt/jfrog/artifactory/var/data/router/traefik/routes","watch":true} 2022-06-10T14:54:09.494Z [jfrou] [INFO ] [ ] [logger.go:58 ] [main ] - Starting provider *traefik.Provider {} 2022-06-10T14:54:09.587Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v018_version_licenses.sql successfully applied (3 statements) [database] 2022-06-10T14:54:09.975Z [jfcon] [INFO ] [40e550fc80ab73a0] [access_thin_client.go:102 ] [main ] - Cluster join: Retry 15: Service registry ping failed, will retry. Error: failed to ping access, response status: 404 Not Found (returned 404) [startup] 2022-06-10T14:54:09.991Z [jfevt] [INFO ] [5461784fd2183141] [access_thin_client.go:104 ] [main ] - Cluster join: Retry 15: Service registry ping failed, will retry. Error: Failed to ping access, response status: 404 Not Found (returned 404) [startup] 2022-06-10T14:54:09.997Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v019_files_versions.sql successfully applied (3 statements) [database] 2022-06-10T14:54:10.179Z [jfint] [INFO ] [605b2e97386125fc] [access_thin_client.go:96 ] [main ] - Cluster join: Retry 15: Service registry ping failed, will retry. Error: Failed to ping access, response status: 404 Not Found (returned 404) [startup] 2022-06-10T14:54:10.607Z [jfrou] [INFO ] [63376afd9572709f] [local_topology.go:116 ] [main ] - Required service types: [jfrt jfac jfmd jffe jfob jfint jfcon jfevt] 2022-06-10T14:54:10.607Z [jfrou] [INFO ] [63376afd9572709f] [main.go:49 ] [main ] - Router (jfrou) service initialization completed in 128.337 seconds. Listening on port: 8082 2022-06-10T14:54:10.607Z [jfrou] [WARN ] [323eb58c4bbe5574] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfac jfmd jffe jfob jfint jfcon jfevt] 2022-06-10T14:54:10.607Z [jfrou] [WARN ] [323eb58c4bbe5574] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:10.751Z [jfrou] [WARN ] [15a7749481f36028] [external_topology.go:126 ] [main ] - restart needed, the node has no registered services 2022-06-10T14:54:10.917Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v020_repo.sql successfully applied (2 statements) [database] 2022-06-10T14:54:11.208Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-1 ] - Retry 30 Elapsed 15.38 secs failed: Registration with router on URL http://localhost:8046 failed with error: UNAVAILABLE: io exception. Trying again 2022-06-10T14:54:11.280Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfac jfmd jffe jfob jfint jfcon jfevt] 2022-06-10T14:54:11.281Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:11.641Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v021_lead_paths.sql successfully applied (4 statements) [database] 2022-06-10T14:54:12.989Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v022_package_descriptions.sql successfully applied (1 statements) [database] 2022-06-10T14:54:13.563Z [jfac ] [INFO ] [ec20a1b61cc7f41e] [.AccessWithRouterRegistrar:111] [jf-common-pool-1 ] - Access Server successfully registered on Router on URL: http://localhost:8046 2022-06-10T14:54:13.644Z [jfrou] [WARN ] [66502935f945786d] [external_topology.go:126 ] [main ] - restart needed, the node has no registered services 2022-06-10T14:54:13.825Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v023_contributors.sql successfully applied (2 statements) [database] 2022-06-10T14:54:14.247Z [jfint] [INFO ] [605b2e97386125fc] [bootstrap.go:84 ] [main ] - JFrog Integration Service ID: jfint@01g571pmkscx5c0q2wmcqy0gez [startup] 2022-06-10T14:54:14.247Z [jfint] [INFO ] [605b2e97386125fc] [bootstrap.go:85 ] [main ] - JFrog Integration Node ID : 7969ad763bf2 [startup] 2022-06-10T14:54:14.247Z [jfint] [INFO ] [605b2e97386125fc] [bootstrap.go:86 ] [main ] - JFrog Integration Node IP : 172.17.0.2 [startup] 2022-06-10T14:54:14.306Z [jfac ] [INFO ] [750e26cb0e8a2047] [a.c.RefreshableScheduledJob:53] [27.0.0.1-8040-exec-8] - Scheduling federationCleanupService task to run every 1209600 seconds 2022-06-10T14:54:14.310Z [jfac ] [INFO ] [8925b3730350421 ] [.f.FederationCleanupService:59] [jf-access-task6 ] - Running clean up of outdated Federation events 2022-06-10T14:54:14.937Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v024_version_contributors.sql successfully applied (3 statements) [database] 2022-06-10T14:54:15.021Z [jfcon] [INFO ] [40e550fc80ab73a0] [bootstrap.go:84 ] [main ] - JFConnect Service ID: jfcon@01g571pmkscx5c0q2wmcqy0gez [startup] 2022-06-10T14:54:15.021Z [jfcon] [INFO ] [40e550fc80ab73a0] [bootstrap.go:85 ] [main ] - JFConnect Node ID : 7969ad763bf2 [startup] 2022-06-10T14:54:15.022Z [jfcon] [INFO ] [40e550fc80ab73a0] [bootstrap.go:86 ] [main ] - JFConnect Node IP : 172.17.0.2 [startup] 2022-06-10T14:54:15.051Z [jfevt] [INFO ] [5461784fd2183141] [bootstrap.go:84 ] [main ] - JFrog Event Service ID: jfevt@01g571pmkscx5c0q2wmcqy0gez [startup] 2022-06-10T14:54:15.052Z [jfevt] [INFO ] [5461784fd2183141] [bootstrap.go:85 ] [main ] - JFrog Event Node ID : 7969ad763bf2 [startup] 2022-06-10T14:54:15.052Z [jfevt] [INFO ] [5461784fd2183141] [bootstrap.go:86 ] [main ] - JFrog Event Node IP : 172.17.0.2 [startup] 2022-06-10T14:54:15.113Z [jfac ] [INFO ] [7f13253a49965d86] [a.c.RefreshableScheduledJob:53] [27.0.0.1-8040-exec-4] - Scheduling federationCleanupService task to run every 1209600 seconds 2022-06-10T14:54:15.120Z [jfac ] [INFO ] [491641e26635554 ] [.f.FederationCleanupService:59] [jf-access-task9 ] - Running clean up of outdated Federation events 2022-06-10T14:54:15.165Z [jfac ] [INFO ] [5c7be3a5dfa859ac] [a.c.RefreshableScheduledJob:53] [27.0.0.1-8040-exec-5] - Scheduling federationCleanupService task to run every 1209600 seconds 2022-06-10T14:54:15.171Z [jfac ] [INFO ] [484324a70f3b7e3c] [.f.FederationCleanupService:59] [jf-access-task5 ] - Running clean up of outdated Federation events 2022-06-10T14:54:15.613Z [jfac ] [INFO ] [750e26cb0e8a2047] [s.r.NodeRegistryServiceImpl:68] [27.0.0.1-8040-exec-8] - Cluster join: Successfully joined jfint@01g571pmkscx5c0q2wmcqy0gez with node id 7969ad763bf2 2022-06-10T14:54:15.621Z [jfrou] [WARN ] [6c456e87037e4652] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob jfint jfcon jfevt] 2022-06-10T14:54:15.621Z [jfrou] [WARN ] [6c456e87037e4652] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:15.627Z [jfint] [INFO ] [605b2e97386125fc] [access_join.go:50 ] [main ] - Cluster join: Successfully joined the cluster [application] 2022-06-10T14:54:15.627Z [jfint] [INFO ] [605b2e97386125fc] [access_join.go:50 ] [main ] - Cluster join: executing Router register at: localhost:8046 for service id jfint@01g571pmkscx5c0q2wmcqy0gez [application] 2022-06-10T14:54:15.645Z [jfint] [INFO ] [605b2e97386125fc] [application.go:134 ] [main ] - JFrog Integration (jfint) service initialization completed in 129.764 seconds. HTTP server listening on 127.0.0.1:8071. GRPC server listening on 127.0.0.1:8072. [application] 2022-06-10T14:54:15.653Z [jfint] [INFO ] [605b2e97386125fc] [git_subscription_worker_servic] [main ] - Starting GitSubscriptionWorker [subscription_service] 2022-06-10T14:54:15.985Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v025_sharedconfigs.sql successfully applied (2 statements) [database] 2022-06-10T14:54:16.292Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob jfcon jfevt] 2022-06-10T14:54:16.294Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:16.294Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:16.294Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:17.010Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 130 2022-06-10T14:54:17.015Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 130 failed with code : undefined 2022-06-10T14:54:17.359Z [jfac ] [INFO ] [7f13253a49965d86] [s.r.NodeRegistryServiceImpl:68] [27.0.0.1-8040-exec-4] - Cluster join: Successfully joined jfcon@01g571pmkscx5c0q2wmcqy0gez with node id 7969ad763bf2 2022-06-10T14:54:17.367Z [jfcon] [INFO ] [40e550fc80ab73a0] [access_join.go:50 ] [main ] - Cluster join: Successfully joined the cluster [application] 2022-06-10T14:54:17.367Z [jfcon] [INFO ] [40e550fc80ab73a0] [access_join.go:50 ] [main ] - Cluster join: executing Router register at: localhost:8046 for service id jfcon@01g571pmkscx5c0q2wmcqy0gez [application] 2022-06-10T14:54:17.390Z [jfcon] [INFO ] [40e550fc80ab73a0] [entitlements_storage.go:40 ] [main ] - EnforcementOn value is forced to the value 'false' [entitlements_storage] 2022-06-10T14:54:17.391Z [jfcon] [INFO ] [40e550fc80ab73a0] [application.go:134 ] [main ] - JFConnect (jfcon) service initialization completed in 130.734 seconds. HTTP server listening on 127.0.0.1:8030. GRPC server listening on 127.0.0.1:8035. [application] 2022-06-10T14:54:17.395Z [jfcon] [INFO ] [40e550fc80ab73a0] [entitlements_service.go:129 ] [main ] - Starting synced actions loop [entitlement_service] 2022-06-10T14:54:17.439Z [jfac ] [INFO ] [5c7be3a5dfa859ac] [s.r.NodeRegistryServiceImpl:68] [27.0.0.1-8040-exec-5] - Cluster join: Successfully joined jfevt@01g571pmkscx5c0q2wmcqy0gez with node id 7969ad763bf2 2022-06-10T14:54:17.442Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v026_version_vulns.sql successfully applied (1 statements) [database] 2022-06-10T14:54:17.457Z [jfevt] [INFO ] [5461784fd2183141] [access_join.go:50 ] [main ] - Cluster join: Successfully joined the cluster [application] 2022-06-10T14:54:17.459Z [jfevt] [INFO ] [5461784fd2183141] [access_join.go:50 ] [main ] - Cluster join: executing Router register at: localhost:8046 for service id jfevt@01g571pmkscx5c0q2wmcqy0gez [application] 2022-06-10T14:54:17.484Z [jfevt] [INFO ] [5461784fd2183141] [application.go:134 ] [main ] - JFrog Event (jfevt) service initialization completed in 133.833 seconds. HTTP server listening on 127.0.0.1:8061. GRPC server listening on 127.0.0.1:8062. [application] 2022-06-10T14:54:17.488Z [jfevt] [INFO ] [5461784fd2183141] [datastore_migration_manager.go] [main ] - Datastore migration check (version: 2)... [migration] 2022-06-10T14:54:17.539Z [jfevt] [INFO ] [5461784fd2183141] [datastore_migration_manager.go] [main ] - Datastore is up-to-date [migration] 2022-06-10T14:54:17.846Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:54:17.913Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:54:18.119Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:54:18.240Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v027_package_vulns.sql successfully applied (1 statements) [database] 2022-06-10T14:54:20.625Z [jfrou] [WARN ] [640962f03b3f29e1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:20.625Z [jfrou] [WARN ] [640962f03b3f29e1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:21.303Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:21.304Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:21.305Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:21.305Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:21.305Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:21.305Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:25.744Z [jfrou] [WARN ] [651e2f8f5a5b7526] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:25.744Z [jfrou] [WARN ] [651e2f8f5a5b7526] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:26.289Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:54:26.312Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:26.314Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:26.314Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:26.314Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:26.314Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:26.314Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:27.113Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 140 2022-06-10T14:54:27.120Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 140 failed with code : undefined 2022-06-10T14:54:27.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:54:27.932Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:54:28.244Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:54:30.857Z [jfrou] [WARN ] [13b93a6e53ac3810] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:30.857Z [jfrou] [WARN ] [13b93a6e53ac3810] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:31.326Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:31.328Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:31.329Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:31.329Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:31.329Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:31.329Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:35.975Z [jfrou] [WARN ] [12ba9771a7607b2d] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:35.975Z [jfrou] [WARN ] [12ba9771a7607b2d] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:36.290Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:54:36.338Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:36.339Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:36.340Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:36.340Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:36.340Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:36.340Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:37.193Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 150 2022-06-10T14:54:37.201Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 150 failed with code : undefined 2022-06-10T14:54:37.613Z [jfrou] [ERROR] [24a5e5286af5baaf] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:54:37.862Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:54:37.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:54:38.200Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:54:40.978Z [jfrou] [ERROR] [12ba9771a7607b2d] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 4 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:54:41.087Z [jfrou] [WARN ] [162d7eee6d6f8fa9] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:41.088Z [jfrou] [WARN ] [162d7eee6d6f8fa9] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:41.348Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:41.352Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:41.353Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:41.353Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:41.353Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:41.353Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:46.091Z [jfrou] [ERROR] [162d7eee6d6f8fa9] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 5 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:54:46.201Z [jfrou] [WARN ] [08f89ebc49a32117] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:46.202Z [jfrou] [WARN ] [08f89ebc49a32117] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:46.289Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:54:46.360Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:46.362Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:46.362Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:46.362Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:46.362Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:46.362Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:47.303Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 160 2022-06-10T14:54:47.310Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 160 failed with code : undefined 2022-06-10T14:54:47.877Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:54:47.928Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:54:48.210Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:54:51.204Z [jfrou] [ERROR] [08f89ebc49a32117] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 6 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:54:51.321Z [jfrou] [WARN ] [59a18b98b0744122] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:51.322Z [jfrou] [WARN ] [59a18b98b0744122] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:51.373Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:51.374Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:51.375Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:51.375Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:51.375Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:51.376Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:52.715Z [jfrou] [ERROR] [0f4517a8f63eb732] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:54:56.289Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:54:56.325Z [jfrou] [ERROR] [59a18b98b0744122] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 7 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:54:56.385Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:56.387Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:56.388Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:56.388Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:56.389Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:54:56.390Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:54:56.438Z [jfrou] [WARN ] [0f8932aa785e235b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:54:56.439Z [jfrou] [WARN ] [0f8932aa785e235b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:54:57.384Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 170 2022-06-10T14:54:57.390Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 170 failed with code : undefined 2022-06-10T14:54:57.873Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:54:57.938Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:54:58.189Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:55:01.396Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:01.397Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:01.398Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:01.398Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:01.398Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:01.398Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:01.447Z [jfrou] [ERROR] [0f8932aa785e235b] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 8 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:01.566Z [jfrou] [WARN ] [6ea7627bb04aa137] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:01.566Z [jfrou] [WARN ] [6ea7627bb04aa137] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:06.290Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:55:06.413Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:06.415Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:06.416Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:06.416Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:06.416Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:06.416Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:06.572Z [jfrou] [ERROR] [6ea7627bb04aa137] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 9 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:06.684Z [jfrou] [WARN ] [6419935773da00b2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:06.684Z [jfrou] [WARN ] [6419935773da00b2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:07.480Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory, attempt number 180 2022-06-10T14:55:07.490Z [jffe ] [INFO ] [ ] [ ] [main ] - pinging artifactory attempt number 180 failed with code : undefined 2022-06-10T14:55:07.818Z [jfrou] [ERROR] [48e7f87cf9e9d97a] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:55:07.892Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:55:07.952Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:55:08.228Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:55:08.492Z [jffe ] [ERROR] [ ] [ ] [main ] - Error starting application - Error: Failed pinging artifactory for 180Request failed with status code 404 2022-06-10T14:55:08.493Z [jffe ] [INFO ] [ ] [ ] [main ] - exit detected 2022-06-10T14:55:08.493Z [jffe ] [INFO ] [ ] [ ] [main ] - doing clean up 2022-06-10T14:55:08.493Z [jffe ] [INFO ] [ ] [ ] [main ] - unregistering from router 2022-06-10T14:55:08.494Z [jffe ] [INFO ] [ ] [ ] [main ] - client is already unregistered from router 2022-06-10T14:55:08.494Z [jffe ] [INFO ] [ ] [ ] [main ] - exit code : 0 2022-06-10T14:55:08.494Z [jffe ] [INFO ] [ ] [ ] [main ] - closing recurring jobs 2022-06-10T14:55:08.494Z [jffe ] [INFO ] [ ] [ ] [main ] - closing recurring jobs 2022-06-10T14:55:09.103Z [jffe ] [INFO ] [ ] [ ] [main ] - Waiting for Express to close all connections 2022-06-10T14:55:11.436Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:11.439Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:11.439Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:11.439Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:11.439Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:11.440Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:11.686Z [jfrou] [ERROR] [6419935773da00b2] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 10 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:11.800Z [jfrou] [WARN ] [014746e321aeea67] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:11.800Z [jfrou] [WARN ] [014746e321aeea67] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:16.290Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:55:16.446Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:16.447Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:16.448Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:16.448Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:16.448Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:16.448Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:16.803Z [jfrou] [ERROR] [014746e321aeea67] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 11 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:16.914Z [jfrou] [WARN ] [68a44fc2399dd96c] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:16.915Z [jfrou] [WARN ] [68a44fc2399dd96c] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:17.888Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:55:17.950Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:55:18.220Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:55:20.754Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v028_versions_ordinal_index.sql successfully applied (1 statements) [database] 2022-06-10T14:55:20.808Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:55:21.310Z [jfcon] [WARN ] [40e550fc80ab73a0] [artifactory_usage_service.go:2] [main ] - Could not perform logs cleanup in artifactory; response status:`404` [artifactory_client] 2022-06-10T14:55:21.312Z [jfcon] [WARN ] [40e550fc80ab73a0] [artifactory_usage_service.go:2] [main ] - Could not perform logs cleanup in artifactory; response status:`404` [artifactory_client] 2022-06-10T14:55:21.314Z [jfcon] [WARN ] [40e550fc80ab73a0] [artifactory_usage_service.go:2] [main ] - Could not perform logs cleanup in artifactory; response status:`404` [artifactory_client] 2022-06-10T14:55:21.316Z [jfcon] [WARN ] [40e550fc80ab73a0] [artifactory_usage_service.go:2] [main ] - Could not perform logs cleanup in artifactory; response status:`404` [artifactory_client] 2022-06-10T14:55:21.319Z [jfcon] [WARN ] [40e550fc80ab73a0] [artifactory_usage_service.go:2] [main ] - Could not perform logs cleanup in artifactory; response status:`404` [artifactory_client] 2022-06-10T14:55:21.321Z [jfcon] [WARN ] [40e550fc80ab73a0] [artifactory_usage_service.go:2] [main ] - Could not perform logs cleanup in artifactory; response status:`404` [artifactory_client] 2022-06-10T14:55:21.322Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:55:21.323Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 3s) [usage_service] 2022-06-10T14:55:21.461Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:21.462Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:21.462Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:21.462Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:21.462Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:21.462Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:21.532Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v029_qualifier_user_prop_length.sql successfully applied (0 statements) [database] 2022-06-10T14:55:21.921Z [jfrou] [ERROR] [68a44fc2399dd96c] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 12 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:21.924Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v030_version_addcolumn_size.sql successfully applied (1 statements) [database] 2022-06-10T14:55:22.041Z [jfrou] [WARN ] [795a35b1cd74f3b0] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:22.041Z [jfrou] [WARN ] [795a35b1cd74f3b0] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:22.268Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v031_package_add_base_model_complete.sql successfully applied (1 statements) [database] 2022-06-10T14:55:22.825Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v032_rename_stats_columns.sql successfully applied (3 statements) [database] 2022-06-10T14:55:22.920Z [jfrou] [ERROR] [135adb4109bd757b] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:55:23.243Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v033_rename_size_column.sql successfully applied (1 statements) [database] 2022-06-10T14:55:23.656Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v034_ver_stats_fk_ver_repos_fk.sql successfully applied (11 statements) [database] 2022-06-10T14:55:24.041Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 10 Elapsed 1.17 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:55:24 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:55:24.071Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v035_files_unique_name_sha256.sql successfully applied (4 statements) [database] 2022-06-10T14:55:24.913Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:55:24.915Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v036_version_add_modified.sql successfully applied (1 statements) [database] 2022-06-10T14:55:25.033Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:55:25.033Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 6s) [usage_service] 2022-06-10T14:55:25.434Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v037_version_add_base_model_complete.sql successfully applied (1 statements) [database] 2022-06-10T14:55:25.867Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v038_file_targets.sql successfully applied (2 statements) [database] 2022-06-10T14:55:26.198Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v039_file_targets_fk.sql successfully applied (1 statements) [database] 2022-06-10T14:55:26.289Z [jfmd ] [INFO ] [ ] [migrator.go:99 ] [main ] - Currently applying db migrations, do not restart until finished [database] 2022-06-10T14:55:26.472Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:26.473Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:26.473Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:26.473Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:26.473Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:26.474Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:26.530Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v042_async_data_converter.sql successfully applied (5 statements) [database] 2022-06-10T14:55:26.870Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v043_add_mark_for_deletion.sql successfully applied (3 statements) [database] 2022-06-10T14:55:27.044Z [jfrou] [ERROR] [795a35b1cd74f3b0] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 13 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:27.160Z [jfrou] [WARN ] [0b2050a821e22fb9] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:27.160Z [jfrou] [WARN ] [0b2050a821e22fb9] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:27.781Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v044_add_dependant_task.sql successfully applied (1 statements) [database] 2022-06-10T14:55:27.866Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:55:27.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:55:28.202Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:55:28.855Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v045_delete_reorder_prerelease_version_task.sql successfully applied (1 statements) [database] 2022-06-10T14:55:29.618Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v046_add_cvss3.sql successfully applied (2 statements) [database] 2022-06-10T14:55:30.982Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v047_reorder_task_temp_table.sql successfully applied (2 statements) [database] 2022-06-10T14:55:31.465Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:55:31.484Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:31.485Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:31.485Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:31.485Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:31.485Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:31.486Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:31.944Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:55:31.944Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 12s) [usage_service] 2022-06-10T14:55:32.041Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v048_version_operational_risk.sql successfully applied (1 statements) [database] 2022-06-10T14:55:32.163Z [jfrou] [ERROR] [0b2050a821e22fb9] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 14 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:32.280Z [jfrou] [WARN ] [13aee61b69353228] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:32.281Z [jfrou] [WARN ] [13aee61b69353228] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:34.139Z [jfmd ] [INFO ] [ ] [migrator.go:254 ] [main ] - Migration v049_package_operational_risk.sql successfully applied (1 statements) [database] 2022-06-10T14:55:34.919Z [jfmd ] [INFO ] [ ] [migration_lock_manager.go:76 ] [main ] - Trying to release lock owned by: 7969ad763bf2 [database] 2022-06-10T14:55:35.938Z [jfmd ] [INFO ] [ ] [migration_lock_manager.go:90 ] [main ] - Released lock owned by node 7969ad763bf2 [database] 2022-06-10T14:55:35.939Z [jfmd ] [INFO ] [ ] [migrator.go:73 ] [main ] - Applied 112 database migrations statements in 99.649731912 seconds [database] 2022-06-10T14:55:35.940Z [jfmd ] [INFO ] [ ] [application.go:71 ] [main ] - Metadata (jfmd) service initialization started. Version: 7.37.0 Revision: fb74002b6b9 PID: 9801 Home: /opt/jfrog/artifactory/var [app_initializer] 2022-06-10T14:55:36.499Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:36.500Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:36.501Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:36.501Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:36.501Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:36.501Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:36.588Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 20 Elapsed 1.38 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:55:36 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:55:36.656Z [jfmd ] [INFO ] [ ] [server_bearer.go:178 ] [main ] - Created service_id and pushed to store: jfmd@01g571t3g5zn6pajctpnrmh7kd [ServerInit] 2022-06-10T14:55:37.287Z [jfrou] [ERROR] [13aee61b69353228] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 15 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:37.299Z [jfac ] [INFO ] [506a4b5893dad3da] [a.c.RefreshableScheduledJob:53] [27.0.0.1-8040-exec-3] - Scheduling federationCleanupService task to run every 1209600 seconds 2022-06-10T14:55:37.301Z [jfac ] [INFO ] [9b344fa7d2c5f3c9] [.f.FederationCleanupService:59] [jf-access-task8 ] - Running clean up of outdated Federation events 2022-06-10T14:55:37.415Z [jfrou] [WARN ] [417a7dbafb81a8a8] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jfmd jffe jfob] 2022-06-10T14:55:37.415Z [jfrou] [WARN ] [417a7dbafb81a8a8] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:37.860Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:55:37.939Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:55:38.023Z [jfrou] [ERROR] [79dbb385a7e6001d] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:55:38.145Z [jfac ] [INFO ] [506a4b5893dad3da] [s.r.NodeRegistryServiceImpl:68] [27.0.0.1-8040-exec-3] - Cluster join: Successfully joined jfmd@01g571t3g5zn6pajctpnrmh7kd with node id 7969ad763bf2 2022-06-10T14:55:38.151Z [jfmd ] [INFO ] [ ] [accessclient.go:62 ] [main ] - Cluster join: Successfully joined the cluster [access_client] 2022-06-10T14:55:38.152Z [jfmd ] [INFO ] [ ] [accessclient.go:62 ] [main ] - Cluster join: executing Router register at: localhost:8046 for service id jfmd@01g571t3g5zn6pajctpnrmh7kd [access_client] 2022-06-10T14:55:38.361Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:55:40.856Z [jfac ] [WARN ] [162d7eee6d6f8fa9] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-3] - 4 records lost 2022-06-10T14:55:40.878Z [jfac ] [WARN ] [651e2f8f5a5b7526] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-1] - 4 records lost 2022-06-10T14:55:40.950Z [jfac ] [WARN ] [6419935773da00b2] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-7] - 4 records lost 2022-06-10T14:55:40.954Z [jfac ] [WARN ] [014746e321aeea67] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-11] - 4 records lost 2022-06-10T14:55:40.974Z [jfac ] [WARN ] [68a44fc2399dd96c] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-10] - 4 records lost 2022-06-10T14:55:40.989Z [jfac ] [ERROR] [135adb4109bd757b] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-1] - Request GET /access/v1/topology failed: null java.lang.reflect.UndeclaredThrowableException: null at com.sun.proxy.$Proxy120.next(Unknown Source) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:355) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:340) at org.jfrog.storage.SqlDaoHelper.getAll(SqlDaoHelper.java:60) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl.getTopology(AccessTopologyDaoImpl.java:121) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$FastClassBySpringCGLIB$$5dcdde5e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$EnhancerBySpringCGLIB$$643f6ddf.getTopology() at org.jfrog.access.server.service.db.TopologyStorageServiceImpl.getTopology(TopologyStorageServiceImpl.java:217) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$FastClassBySpringCGLIB$$f6f98c55.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$EnhancerBySpringCGLIB$$f58cb2bd.getTopology() at org.jfrog.access.server.service.endpoint.TopologyServiceImpl.getTopology(TopologyServiceImpl.java:107) at org.jfrog.access.server.rest.resource.topology.TopologyResource.buildTopologyResponse(TopologyResource.java:65) at org.jfrog.access.server.rest.resource.topology.TopologyResource.getTopology(TopologyResource.java:51) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.UsernameAttributeFilter.doFilterInternal(UsernameAttributeFilter.java:35) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.RequestLoggingFilter.doFilterInternal(RequestLoggingFilter.java:52) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.TracingFilter.doFilterInternal(TracingFilter.java:47) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.jfrog.access.filter.LoggerConfigAssignerFilter.doFilterInternal(LoggerConfigAssignerFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:359) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1735) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jfrog.storage.wrapper.ResultSetWrapper.invoke(ResultSetWrapper.java:85) ... 147 common frames omitted Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {783, S} , APP, SELECT * FROM access_topology Granted XID : {722, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {722, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {672, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {717, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Granted XID : {689, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {689, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 783. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java) ... 151 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {783, S} , APP, SELECT * FROM access_topology Granted XID : {722, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {722, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {672, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {717, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Granted XID : {689, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {689, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 783. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.services.locks.Deadlock.buildException(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source) at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 154 common frames omitted 2022-06-10T14:55:41.067Z [jfac ] [WARN ] [08f89ebc49a32117] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-5] - 4 records lost 2022-06-10T14:55:41.515Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:41.516Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:41.517Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:41.517Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:41.517Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:41.517Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:41.517Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:42.422Z [jfrou] [ERROR] [417a7dbafb81a8a8] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 16 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:42.545Z [jfrou] [WARN ] [79448693db953e4c] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:42.546Z [jfrou] [WARN ] [79448693db953e4c] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:44.131Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:55:44.274Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:55:44.275Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 24s) [usage_service] 2022-06-10T14:55:45.754Z [jfmd ] [INFO ] [ ] [async_converter.go:196 ] [main ] - Starting async converter routine [async_converter] [workerNumber][1] 2022-06-10T14:55:45.755Z [jfmd ] [INFO ] [ ] [router_middlewares.go:56 ] [main ] - Metadata (jfmd) service initialization completed in 9.814 seconds. Listening on port: 8086 [router] 2022-06-10T14:55:46.527Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:46.528Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:46.528Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:46.529Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:46.529Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:46.529Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:46.529Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:47.549Z [jfrou] [ERROR] [79448693db953e4c] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 17 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:47.664Z [jfrou] [WARN ] [55c7b2b9a8df90c1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:47.665Z [jfrou] [WARN ] [55c7b2b9a8df90c1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:47.867Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:55:47.935Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:55:48.191Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:55:49.056Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 30 Elapsed 1.59 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:55:49 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:55:51.536Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:51.538Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:51.538Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:51.538Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:51.538Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:51.538Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:51.539Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:52.461Z [jfmd ] [INFO ] [ ] [database_bearer.go:368 ] [main ] - Slow statement detected (5.202747085s) - UPDATE md_servers SET last_heartbeat = ? WHERE id = ? [database] 2022-06-10T14:55:52.669Z [jfrou] [ERROR] [55c7b2b9a8df90c1] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 18 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:52.782Z [jfrou] [WARN ] [0a2de52a4ca71dd3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:52.782Z [jfrou] [WARN ] [0a2de52a4ca71dd3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:53.126Z [jfrou] [ERROR] [0c5b02171dfadd78] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:55:53.776Z [jfmd ] [INFO ] [ ] [versions_package_id_ordinal_in] [main ] - Running DB indexes [md_version_pkgid_ordinal_idx,md_versions_package_id_idx,md_versions_ordinal_idx] maintenance in the background, it might cause some slowness in the Metadata service [BatchEndKey][1] [BatchKey][versionsPackageIdOrdinalIndexTask] [BatchStartKey][0] [versions_package_id_ordinal_index_task] [workerNumber][1] 2022-06-10T14:55:53.778Z [jfmd ] [INFO ] [ ] [versions_package_id_ordinal_in] [main ] - Finished with DB indexes [[md_version_pkgid_ordinal_idx md_versions_package_id_idx md_versions_ordinal_idx]] background maintenance, Metadata service is back to normal [BatchEndKey][1] [BatchKey][versionsPackageIdOrdinalIndexTask] [BatchStartKey][0] [versions_package_id_ordinal_index_task] [workerNumber][1] 2022-06-10T14:55:56.547Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:56.548Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:56.548Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:56.548Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:56.548Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:56.549Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:55:56.549Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:55:56.862Z [jfmd ] [INFO ] [ ] [ver_repos_lead_file_path_index] [main ] - Running DB index [md_ver_repos_lead_file_pth_idx] maintenance in the background, it might cause some slowness in the Metadata service [BatchEndKey][1] [BatchKey][verReposLeadFilePathIndexIndexTask] [BatchStartKey][0] [ver_repos_lead_file_path_index_task] [workerNumber][1] 2022-06-10T14:55:56.868Z [jfmd ] [INFO ] [ ] [ver_repos_lead_file_path_index] [main ] - Finished with DB index [md_ver_repos_lead_file_pth_idx] background maintenance, Metadata service is back to normal [BatchEndKey][1] [BatchKey][verReposLeadFilePathIndexIndexTask] [BatchStartKey][0] [ver_repos_lead_file_path_index_task] [workerNumber][1] 2022-06-10T14:55:57.786Z [jfrou] [ERROR] [0a2de52a4ca71dd3] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 19 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:55:57.876Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:55:57.898Z [jfrou] [WARN ] [6a14ac9b9842a792] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:55:57.898Z [jfrou] [WARN ] [6a14ac9b9842a792] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:55:57.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:55:58.177Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:56:00.041Z [jfmd ] [INFO ] [ ] [value_columns_to_nullable_task] [main ] - DB Type sqlite doesn't require valueColumnsToNullableTask migration [BatchEndKey][1] [BatchKey][valueColumnsToNullableTask] [BatchStartKey][0] [value_columns_to_nullable] [workerNumber][1] 2022-06-10T14:56:00.546Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 40 Elapsed 1.78 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:56:00 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:56:01.094Z [jfac ] [ERROR] [0c5b02171dfadd78] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-2] - Request GET /access/v1/topology failed: null java.lang.reflect.UndeclaredThrowableException: null at com.sun.proxy.$Proxy120.next(Unknown Source) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:355) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:340) at org.jfrog.storage.SqlDaoHelper.getAll(SqlDaoHelper.java:60) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl.getTopology(AccessTopologyDaoImpl.java:121) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$FastClassBySpringCGLIB$$5dcdde5e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$EnhancerBySpringCGLIB$$643f6ddf.getTopology() at org.jfrog.access.server.service.db.TopologyStorageServiceImpl.getTopology(TopologyStorageServiceImpl.java:217) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$FastClassBySpringCGLIB$$f6f98c55.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$EnhancerBySpringCGLIB$$f58cb2bd.getTopology() at org.jfrog.access.server.service.endpoint.TopologyServiceImpl.getTopology(TopologyServiceImpl.java:107) at org.jfrog.access.server.rest.resource.topology.TopologyResource.buildTopologyResponse(TopologyResource.java:65) at org.jfrog.access.server.rest.resource.topology.TopologyResource.getTopology(TopologyResource.java:51) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.UsernameAttributeFilter.doFilterInternal(UsernameAttributeFilter.java:35) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.RequestLoggingFilter.doFilterInternal(RequestLoggingFilter.java:52) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.TracingFilter.doFilterInternal(TracingFilter.java:47) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.jfrog.access.filter.LoggerConfigAssignerFilter.doFilterInternal(LoggerConfigAssignerFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:359) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1735) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jfrog.storage.wrapper.ResultSetWrapper.invoke(ResultSetWrapper.java:85) ... 147 common frames omitted Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {928, S} , APP, SELECT * FROM access_topology Granted XID : {849, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {849, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {770, S} , APP, SELECT endpoint_id, service_id, node_id, router_id, created, ip, port, is_secure, paths, state, last_updated FROM access_topology WHERE router_id IN (?) Granted XID : {743, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {743, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 928. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java) ... 151 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {928, S} , APP, SELECT * FROM access_topology Granted XID : {849, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {849, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {770, S} , APP, SELECT endpoint_id, service_id, node_id, router_id, created, ip, port, is_secure, paths, state, last_updated FROM access_topology WHERE router_id IN (?) Granted XID : {743, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {743, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 928. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.services.locks.Deadlock.buildException(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source) at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 154 common frames omitted 2022-06-10T14:56:01.588Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:01.590Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:01.590Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:01.590Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:01.590Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:01.590Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:01.591Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:01.708Z [jfac ] [WARN ] [417a7dbafb81a8a8] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-14] - 4 records lost 2022-06-10T14:56:01.728Z [jfac ] [ERROR] [79dbb385a7e6001d] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-4] - Request GET /access/v1/topology failed: null java.lang.reflect.UndeclaredThrowableException: null at com.sun.proxy.$Proxy120.next(Unknown Source) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:355) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:340) at org.jfrog.storage.SqlDaoHelper.getAll(SqlDaoHelper.java:60) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl.getTopology(AccessTopologyDaoImpl.java:121) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$FastClassBySpringCGLIB$$5dcdde5e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$EnhancerBySpringCGLIB$$643f6ddf.getTopology() at org.jfrog.access.server.service.db.TopologyStorageServiceImpl.getTopology(TopologyStorageServiceImpl.java:217) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$FastClassBySpringCGLIB$$f6f98c55.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$EnhancerBySpringCGLIB$$f58cb2bd.getTopology() at org.jfrog.access.server.service.endpoint.TopologyServiceImpl.getTopology(TopologyServiceImpl.java:107) at org.jfrog.access.server.rest.resource.topology.TopologyResource.buildTopologyResponse(TopologyResource.java:65) at org.jfrog.access.server.rest.resource.topology.TopologyResource.getTopology(TopologyResource.java:51) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.UsernameAttributeFilter.doFilterInternal(UsernameAttributeFilter.java:35) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.RequestLoggingFilter.doFilterInternal(RequestLoggingFilter.java:52) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.TracingFilter.doFilterInternal(TracingFilter.java:47) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.jfrog.access.filter.LoggerConfigAssignerFilter.doFilterInternal(LoggerConfigAssignerFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:359) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1735) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jfrog.storage.wrapper.ResultSetWrapper.invoke(ResultSetWrapper.java:85) ... 147 common frames omitted Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {852, S} , APP, SELECT * FROM access_topology Granted XID : {849, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {849, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {751, S} , APP, SELECT endpoint_id, service_id, node_id, router_id, created, ip, port, is_secure, paths, state, last_updated FROM access_topology WHERE router_id IN (?) Granted XID : {743, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {743, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 852. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java) ... 151 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {852, S} , APP, SELECT * FROM access_topology Granted XID : {849, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {849, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {751, S} , APP, SELECT endpoint_id, service_id, node_id, router_id, created, ip, port, is_secure, paths, state, last_updated FROM access_topology WHERE router_id IN (?) Granted XID : {743, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {743, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 852. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.services.locks.Deadlock.buildException(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source) at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 154 common frames omitted 2022-06-10T14:56:01.746Z [jfac ] [WARN ] [0b2050a821e22fb9] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-2] - 4 records lost 2022-06-10T14:56:01.752Z [jfac ] [WARN ] [6ea7627bb04aa137] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-9] - 4 records lost 2022-06-10T14:56:02.568Z [jfac ] [WARN ] [0f8932aa785e235b] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-6] - 4 records lost 2022-06-10T14:56:02.904Z [jfrou] [ERROR] [6a14ac9b9842a792] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 20 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:03.012Z [jfrou] [WARN ] [0d9988f7ae5df611] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:03.012Z [jfrou] [WARN ] [0d9988f7ae5df611] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:06.606Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:06.609Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:06.610Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:06.611Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:06.611Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:06.611Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:06.612Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:07.286Z [jfmd ] [INFO ] [ ] [database_bearer.go:368 ] [main ] - Slow statement detected (5.019724257s) - UPDATE md_servers SET last_heartbeat = ? WHERE id = ? [database] 2022-06-10T14:56:07.286Z [jfmd ] [INFO ] [ ] [server_bearer.go:143 ] [main ] - RETRY: fail to update heartbeat for server in the database [ServerInit] 2022-06-10T14:56:07.879Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:56:07.925Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:56:08.015Z [jfrou] [ERROR] [0d9988f7ae5df611] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 21 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:08.194Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:56:08.230Z [jfrou] [ERROR] [2fd525da940e7283] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:56:08.307Z [jfac ] [WARN ] [795a35b1cd74f3b0] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-12] - 4 records lost 2022-06-10T14:56:08.666Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:56:08.983Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:56:08.983Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 48s) [usage_service] 2022-06-10T14:56:09.098Z [jfrou] [WARN ] [2fc5db3d14e0a101] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:09.098Z [jfrou] [WARN ] [2fc5db3d14e0a101] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:11.621Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:11.622Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:11.622Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:11.622Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:11.622Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:11.622Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:11.622Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:13.064Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 50 Elapsed 1.99 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:56:13 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:56:14.101Z [jfrou] [ERROR] [2fc5db3d14e0a101] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 22 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:14.234Z [jfrou] [WARN ] [18e426a7138f8768] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:14.235Z [jfrou] [WARN ] [18e426a7138f8768] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:16.650Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:16.654Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:16.654Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:16.654Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:16.655Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:16.655Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:16.655Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:17.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:56:17.941Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:56:18.192Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:56:19.237Z [jfrou] [ERROR] [18e426a7138f8768] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 23 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:19.360Z [jfrou] [WARN ] [45027da1acc1d6cd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:19.363Z [jfrou] [WARN ] [45027da1acc1d6cd] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:21.662Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:21.665Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:21.665Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:21.666Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:21.666Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:21.666Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:21.667Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:23.333Z [jfrou] [ERROR] [0bc6fb754f7be1f5] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:56:24.367Z [jfrou] [ERROR] [45027da1acc1d6cd] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 24 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:24.411Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 60 Elapsed 2.18 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:56:24 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:56:24.488Z [jfrou] [WARN ] [17e83c91fe3bb564] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:24.488Z [jfrou] [WARN ] [17e83c91fe3bb564] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:26.680Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:26.681Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:26.681Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:26.682Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:26.682Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:26.682Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:26.682Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:27.858Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:56:27.935Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:56:28.212Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:56:28.776Z [jfac ] [WARN ] [79448693db953e4c] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-0] - 5 records lost 2022-06-10T14:56:28.868Z [jfac ] [WARN ] [13aee61b69353228] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-15] - 4 records lost 2022-06-10T14:56:29.491Z [jfrou] [ERROR] [17e83c91fe3bb564] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 25 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:29.605Z [jfrou] [WARN ] [04ae266e53506fcf] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:29.605Z [jfrou] [WARN ] [04ae266e53506fcf] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:31.687Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:31.688Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:31.688Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:31.689Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:31.689Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:31.689Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:31.695Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:34.611Z [jfrou] [ERROR] [04ae266e53506fcf] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 26 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:34.729Z [jfrou] [WARN ] [59fb963cc6d6e764] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:34.729Z [jfrou] [WARN ] [59fb963cc6d6e764] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:35.913Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 70 Elapsed 2.37 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:56:35 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:56:36.700Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:36.701Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:36.701Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:36.701Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:36.701Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:36.702Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:36.702Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:37.859Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:56:37.943Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:56:38.225Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:56:39.741Z [jfrou] [ERROR] [59fb963cc6d6e764] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 27 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:39.855Z [jfrou] [WARN ] [476c204938a2b910] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:39.855Z [jfrou] [WARN ] [476c204938a2b910] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:41.711Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:41.712Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:41.712Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:41.713Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:41.713Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:41.713Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:41.713Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:44.222Z [jfrou] [ERROR] [5a200fb47a12e72b] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:56:44.858Z [jfrou] [ERROR] [476c204938a2b910] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 28 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:44.975Z [jfrou] [WARN ] [46491de70cfd1e01] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:44.975Z [jfrou] [WARN ] [46491de70cfd1e01] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:46.723Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:46.725Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:46.725Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:46.725Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:46.725Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:46.725Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:46.726Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:47.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:56:47.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:56:48.183Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:56:49.130Z [jfac ] [WARN ] [45027da1acc1d6cd] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-9] - 5 records lost 2022-06-10T14:56:49.132Z [jfac ] [WARN ] [17e83c91fe3bb564] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-2] - 5 records lost 2022-06-10T14:56:49.979Z [jfrou] [ERROR] [46491de70cfd1e01] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 29 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:50.093Z [jfrou] [WARN ] [2ed27fe391aa966a] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:50.093Z [jfrou] [WARN ] [2ed27fe391aa966a] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:51.735Z [jfrou] [WARN ] [6fb74bd67a5de1dd] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:51.737Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:51.738Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:51.738Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:51.738Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:51.740Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:190 ] [main ] - Obtained instance Id: [startup] 2022-06-10T14:56:51.741Z [[35[jfob ] [INFO ] [6fb74bd67a5de1dd] [bootstrap.go:141 ] [main ] - Unable to find parent service using router health, trying again... [startup] 2022-06-10T14:56:52.432Z [jfmd ] [INFO ] [ ] [database_bearer.go:368 ] [main ] - Slow statement detected (15.172088553s) - UPDATE md_servers SET last_heartbeat = ? WHERE id = ? [database] 2022-06-10T14:56:52.513Z [jfac ] [WARN ] [6a14ac9b9842a792] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-5] - 5 records lost 2022-06-10T14:56:55.096Z [jfrou] [ERROR] [2ed27fe391aa966a] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 30 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:56:55.214Z [jfrou] [WARN ] [7e209058df63d894] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:56:55.215Z [jfrou] [WARN ] [7e209058df63d894] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:56:57.142Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:56:57.253Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:56:57.256Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 1m36s) [usage_service] 2022-06-10T14:56:57.860Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:56:57.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:56:58.184Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:57:00.218Z [jfrou] [ERROR] [7e209058df63d894] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 31 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:00.347Z [jfrou] [WARN ] [40f36880ecc8b43b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:00.347Z [jfrou] [WARN ] [40f36880ecc8b43b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:01.283Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 80 Elapsed 2.8 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:57:01 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:57:05.351Z [jfrou] [ERROR] [40f36880ecc8b43b] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 32 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:05.477Z [jfrou] [WARN ] [400a8be3d83aa2e8] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:05.477Z [jfrou] [WARN ] [400a8be3d83aa2e8] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:07.868Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:57:07.932Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:57:07.941Z [jfrou] [ERROR] [498b32d6c1f279ce] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:57:08.167Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:57:10.484Z [jfrou] [ERROR] [400a8be3d83aa2e8] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 33 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:10.598Z [jfrou] [WARN ] [775510fc66a3049e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:10.598Z [jfrou] [WARN ] [775510fc66a3049e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:12.611Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 90 Elapsed 2.98 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:57:12 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:57:12.940Z [jfac ] [WARN ] [2ed27fe391aa966a] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-2] - 5 records lost 2022-06-10T14:57:12.946Z [jfac ] [WARN ] [46491de70cfd1e01] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-15] - 5 records lost 2022-06-10T14:57:13.037Z [jfac ] [WARN ] [476c204938a2b910] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-10] - 5 records lost 2022-06-10T14:57:15.600Z [jfrou] [ERROR] [775510fc66a3049e] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 34 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:15.724Z [jfrou] [WARN ] [2f2ec7936c2f44fe] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:15.724Z [jfrou] [WARN ] [2f2ec7936c2f44fe] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:17.866Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:57:17.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:57:18.197Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:57:20.732Z [jfrou] [ERROR] [2f2ec7936c2f44fe] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 35 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:20.855Z [jfrou] [WARN ] [32eaa9cd6ab22edc] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:20.856Z [jfrou] [WARN ] [32eaa9cd6ab22edc] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:23.043Z [jfrou] [ERROR] [59064a9bafaa7efe] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:57:23.773Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 100 Elapsed 3.17 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:57:23 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:57:25.870Z [jfrou] [ERROR] [32eaa9cd6ab22edc] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 36 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:25.979Z [jfrou] [WARN ] [7e6e48567e0310c6] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:25.980Z [jfrou] [WARN ] [7e6e48567e0310c6] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:27.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:57:27.930Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:57:28.195Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:57:30.983Z [jfrou] [ERROR] [7e6e48567e0310c6] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 37 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:31.097Z [jfrou] [WARN ] [39ebdbb7a1cf3706] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:31.100Z [jfrou] [WARN ] [39ebdbb7a1cf3706] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:33.175Z [jfac ] [WARN ] [775510fc66a3049e] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-9] - 5 records lost 2022-06-10T14:57:33.682Z [jfac ] [WARN ] [40f36880ecc8b43b] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-6] - 5 records lost 2022-06-10T14:57:35.056Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 110 Elapsed 3.36 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:57:35 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:57:36.146Z [jfrou] [ERROR] [39ebdbb7a1cf3706] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 38 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:36.282Z [jfrou] [WARN ] [21e9b59bd32a11f7] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:36.282Z [jfrou] [WARN ] [21e9b59bd32a11f7] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:37.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:57:37.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:57:38.151Z [jfrou] [ERROR] [5c3974a2d18be976] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:57:38.220Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:57:41.285Z [jfrou] [ERROR] [21e9b59bd32a11f7] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 39 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:41.406Z [jfrou] [WARN ] [0bfa16f739191f77] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:41.406Z [jfrou] [WARN ] [0bfa16f739191f77] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:46.332Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 120 Elapsed 3.55 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:57:46 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:57:46.418Z [jfrou] [ERROR] [0bfa16f739191f77] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 40 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:46.543Z [jfrou] [WARN ] [68ce2b22b8b56a40] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:46.543Z [jfrou] [WARN ] [68ce2b22b8b56a40] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:47.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:57:47.939Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:57:48.186Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:57:51.551Z [jfrou] [ERROR] [68ce2b22b8b56a40] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 41 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:51.659Z [jfrou] [WARN ] [551e80afa74f4d99] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:51.659Z [jfrou] [WARN ] [551e80afa74f4d99] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:53.253Z [jfrou] [ERROR] [73d2eea92edfe3de] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:57:53.727Z [jfac ] [ERROR] [5c3974a2d18be976] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-9] - Request GET /access/v1/topology failed: null java.lang.reflect.UndeclaredThrowableException: null at com.sun.proxy.$Proxy120.next(Unknown Source) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:355) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:340) at org.jfrog.storage.SqlDaoHelper.getAll(SqlDaoHelper.java:60) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl.getTopology(AccessTopologyDaoImpl.java:121) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$FastClassBySpringCGLIB$$5dcdde5e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$EnhancerBySpringCGLIB$$643f6ddf.getTopology() at org.jfrog.access.server.service.db.TopologyStorageServiceImpl.getTopology(TopologyStorageServiceImpl.java:217) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$FastClassBySpringCGLIB$$f6f98c55.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$EnhancerBySpringCGLIB$$f58cb2bd.getTopology() at org.jfrog.access.server.service.endpoint.TopologyServiceImpl.getTopology(TopologyServiceImpl.java:107) at org.jfrog.access.server.rest.resource.topology.TopologyResource.buildTopologyResponse(TopologyResource.java:65) at org.jfrog.access.server.rest.resource.topology.TopologyResource.getTopology(TopologyResource.java:51) at jdk.internal.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.UsernameAttributeFilter.doFilterInternal(UsernameAttributeFilter.java:35) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.RequestLoggingFilter.doFilterInternal(RequestLoggingFilter.java:52) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.TracingFilter.doFilterInternal(TracingFilter.java:47) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.jfrog.access.filter.LoggerConfigAssignerFilter.doFilterInternal(LoggerConfigAssignerFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:359) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1735) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jfrog.storage.wrapper.ResultSetWrapper.invoke(ResultSetWrapper.java:85) ... 146 common frames omitted Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1277, S} , APP, SELECT * FROM access_topology Granted XID : {1217, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {1217, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Granted XID : {1249, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1249, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 1277. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java) ... 150 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1277, S} , APP, SELECT * FROM access_topology Granted XID : {1217, X} Lock : ROW, ACCESS_TOPOLOGY, (1,8) Waiting XID : {1217, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Granted XID : {1249, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1249, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 1277. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.services.locks.Deadlock.buildException(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source) at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 153 common frames omitted 2022-06-10T14:57:53.730Z [jfac ] [WARN ] [7e6e48567e0310c6] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-10] - 5 records lost 2022-06-10T14:57:53.738Z [jfac ] [WARN ] [39ebdbb7a1cf3706] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-5] - 5 records lost 2022-06-10T14:57:53.920Z [jfac ] [WARN ] [400a8be3d83aa2e8] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-8] - 5 records lost 2022-06-10T14:57:56.662Z [jfrou] [ERROR] [551e80afa74f4d99] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 42 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:57:56.780Z [jfrou] [WARN ] [31cf8f89064e82a7] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:57:56.780Z [jfrou] [WARN ] [31cf8f89064e82a7] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:57:57.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:57:57.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:57:58.008Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 130 Elapsed 3.74 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:57:58 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:57:58.183Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:01.784Z [jfrou] [ERROR] [31cf8f89064e82a7] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 43 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:01.920Z [jfrou] [WARN ] [1695b05981fcdd7c] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:01.921Z [jfrou] [WARN ] [1695b05981fcdd7c] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:06.927Z [jfrou] [ERROR] [1695b05981fcdd7c] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 44 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:07.055Z [jfrou] [WARN ] [4a81e8cc2a3dca63] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:07.055Z [jfrou] [WARN ] [4a81e8cc2a3dca63] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:07.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:58:07.932Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:58:08.169Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:08.355Z [jfrou] [ERROR] [1c6ac777bd9b1f7e] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:58:09.413Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 140 Elapsed 3.93 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:58:09 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:58:12.061Z [jfrou] [ERROR] [4a81e8cc2a3dca63] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 45 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:12.174Z [jfrou] [WARN ] [75796cc5059db9ec] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:12.174Z [jfrou] [WARN ] [75796cc5059db9ec] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:14.161Z [jfac ] [WARN ] [32eaa9cd6ab22edc] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-15] - 5 records lost 2022-06-10T14:58:14.168Z [jfac ] [WARN ] [551e80afa74f4d99] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-2] - 5 records lost 2022-06-10T14:58:14.175Z [jfac ] [ERROR] [1c6ac777bd9b1f7e] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-2] - Request GET /access/v1/topology failed: null java.lang.reflect.UndeclaredThrowableException: null at com.sun.proxy.$Proxy120.next(Unknown Source) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:355) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:340) at org.jfrog.storage.SqlDaoHelper.getAll(SqlDaoHelper.java:60) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl.getTopology(AccessTopologyDaoImpl.java:121) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$FastClassBySpringCGLIB$$5dcdde5e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$EnhancerBySpringCGLIB$$643f6ddf.getTopology() at org.jfrog.access.server.service.db.TopologyStorageServiceImpl.getTopology(TopologyStorageServiceImpl.java:217) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$FastClassBySpringCGLIB$$f6f98c55.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$EnhancerBySpringCGLIB$$f58cb2bd.getTopology() at org.jfrog.access.server.service.endpoint.TopologyServiceImpl.getTopology(TopologyServiceImpl.java:107) at org.jfrog.access.server.rest.resource.topology.TopologyResource.buildTopologyResponse(TopologyResource.java:65) at org.jfrog.access.server.rest.resource.topology.TopologyResource.getTopology(TopologyResource.java:51) at jdk.internal.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.UsernameAttributeFilter.doFilterInternal(UsernameAttributeFilter.java:35) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.RequestLoggingFilter.doFilterInternal(RequestLoggingFilter.java:52) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.TracingFilter.doFilterInternal(TracingFilter.java:47) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.jfrog.access.filter.LoggerConfigAssignerFilter.doFilterInternal(LoggerConfigAssignerFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:359) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1735) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jfrog.storage.wrapper.ResultSetWrapper.invoke(ResultSetWrapper.java:85) ... 146 common frames omitted Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,9) Waiting XID : {1370, S} , APP, SELECT * FROM access_topology Granted XID : {1335, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1335, X} , APP, ? Granted XID : {1311, X} Lock : ROW, ACCESS_TOPOLOGY, (1,9) Waiting XID : {1311, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 1370. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java) ... 150 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,9) Waiting XID : {1370, S} , APP, SELECT * FROM access_topology Granted XID : {1335, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1335, X} , APP, ? Granted XID : {1311, X} Lock : ROW, ACCESS_TOPOLOGY, (1,9) Waiting XID : {1311, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 1370. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.services.locks.Deadlock.buildException(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source) at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 153 common frames omitted 2022-06-10T14:58:14.210Z [jfac ] [WARN ] [0bfa16f739191f77] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-6] - 5 records lost 2022-06-10T14:58:17.179Z [jfrou] [ERROR] [75796cc5059db9ec] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 46 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:17.288Z [jfrou] [WARN ] [1316eb1af6e4cc7e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:17.288Z [jfrou] [WARN ] [1316eb1af6e4cc7e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:17.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:58:17.934Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:58:18.183Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:21.194Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 150 Elapsed 4.13 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:58:21 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:58:22.290Z [jfrou] [ERROR] [1316eb1af6e4cc7e] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 47 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:22.403Z [jfrou] [WARN ] [3b4d865a08c0f3ea] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:22.403Z [jfrou] [WARN ] [3b4d865a08c0f3ea] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:27.407Z [jfrou] [ERROR] [3b4d865a08c0f3ea] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 48 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:27.514Z [jfrou] [WARN ] [7bca83c363bdd35d] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:27.514Z [jfrou] [WARN ] [7bca83c363bdd35d] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:27.858Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:58:27.937Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:58:28.206Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:29.631Z [jfrou] [ERROR] [6a353c7c9b4c6b5c] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:58:32.517Z [jfrou] [ERROR] [7bca83c363bdd35d] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 49 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:32.627Z [jfrou] [WARN ] [5988e17f56f31462] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:32.628Z [jfrou] [WARN ] [5988e17f56f31462] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:33.505Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 160 Elapsed 4.33 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:58:33 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:58:33.713Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T14:58:33.772Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T14:58:33.773Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 3m12s) [usage_service] 2022-06-10T14:58:34.538Z [jfac ] [WARN ] [75796cc5059db9ec] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-10] - 5 records lost 2022-06-10T14:58:34.653Z [jfac ] [WARN ] [4a81e8cc2a3dca63] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-5] - 5 records lost 2022-06-10T14:58:37.630Z [jfrou] [ERROR] [5988e17f56f31462] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 50 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:37.742Z [jfrou] [WARN ] [48223cd03a43b805] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:37.742Z [jfrou] [WARN ] [48223cd03a43b805] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:37.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:58:37.948Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:58:38.161Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:42.747Z [jfrou] [ERROR] [48223cd03a43b805] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 51 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:42.862Z [jfrou] [WARN ] [149728bf69d8bcd2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:42.863Z [jfrou] [WARN ] [149728bf69d8bcd2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:44.681Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 170 Elapsed 4.52 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:58:44 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:58:44.733Z [jfrou] [ERROR] [2312fa371b492786] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:58:47.856Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:58:47.869Z [jfrou] [ERROR] [149728bf69d8bcd2] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 52 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:47.929Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:58:48.002Z [jfrou] [WARN ] [127620cc869a40ad] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:48.002Z [jfrou] [WARN ] [127620cc869a40ad] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:48.179Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:53.005Z [jfrou] [ERROR] [127620cc869a40ad] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 53 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:53.113Z [jfrou] [WARN ] [5c0b7f2dead85565] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:53.113Z [jfrou] [WARN ] [5c0b7f2dead85565] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:58:54.722Z [jfac ] [WARN ] [5988e17f56f31462] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-9] - 5 records lost 2022-06-10T14:58:54.725Z [jfac ] [ERROR] [2312fa371b492786] [.e.m.DefaultExceptionMapper:30] [27.0.0.1-8040-exec-6] - Request GET /access/v1/topology failed: null java.lang.reflect.UndeclaredThrowableException: null at com.sun.proxy.$Proxy120.next(Unknown Source) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:355) at org.jfrog.storage.SqlDaoHelper.findAllByQuery(SqlDaoHelper.java:340) at org.jfrog.storage.SqlDaoHelper.getAll(SqlDaoHelper.java:60) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl.getTopology(AccessTopologyDaoImpl.java:121) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$FastClassBySpringCGLIB$$5dcdde5e.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698) at org.jfrog.access.server.db.dao.AccessTopologyDaoImpl$$EnhancerBySpringCGLIB$$643f6ddf.getTopology() at org.jfrog.access.server.service.db.TopologyStorageServiceImpl.getTopology(TopologyStorageServiceImpl.java:217) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$FastClassBySpringCGLIB$$f6f98c55.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) at org.jfrog.access.server.service.db.TopologyStorageServiceImpl$$EnhancerBySpringCGLIB$$f58cb2bd.getTopology() at org.jfrog.access.server.service.endpoint.TopologyServiceImpl.getTopology(TopologyServiceImpl.java:107) at org.jfrog.access.server.rest.resource.topology.TopologyResource.buildTopologyResponse(TopologyResource.java:65) at org.jfrog.access.server.rest.resource.topology.TopologyResource.getTopology(TopologyResource.java:51) at jdk.internal.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:111) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.UsernameAttributeFilter.doFilterInternal(UsernameAttributeFilter.java:35) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:149) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:103) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:89) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:110) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.RequestLoggingFilter.doFilterInternal(RequestLoggingFilter.java:52) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:55) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.jfrog.access.filter.TracingFilter.doFilterInternal(TracingFilter.java:47) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:211) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:183) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:354) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:267) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:126) at org.springframework.boot.web.servlet.support.ErrorPageFilter.access$000(ErrorPageFilter.java:64) at org.springframework.boot.web.servlet.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:101) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.springframework.boot.web.servlet.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:119) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.jfrog.access.filter.LoggerConfigAssignerFilter.doFilterInternal(LoggerConfigAssignerFilter.java:43) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:540) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:289) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:359) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1735) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.lang.reflect.InvocationTargetException: null at jdk.internal.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jfrog.storage.wrapper.ResultSetWrapper.invoke(ResultSetWrapper.java:85) ... 146 common frames omitted Caused by: java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1506, S} , APP, SELECT * FROM access_topology Granted XID : {1481, X} Lock : ROW, ACCESS_TOPOLOGY, (1,12) Waiting XID : {1481, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Granted XID : {1451, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1451, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 1506. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source) at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source) at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java) ... 150 common frames omitted Caused by: org.apache.derby.iapi.error.StandardException: A lock could not be obtained due to a deadlock, cycle of locks and waiters is: Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1506, S} , APP, SELECT * FROM access_topology Granted XID : {1481, X} Lock : ROW, ACCESS_TOPOLOGY, (1,12) Waiting XID : {1481, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? Granted XID : {1451, X} Lock : ROW, ACCESS_TOPOLOGY, (1,11) Waiting XID : {1451, X} , APP, UPDATE access_topology SET state = ?, last_updated = ? WHERE endpoint_id = ? . The selected victim is XID : 1506. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.services.locks.Deadlock.buildException(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.lockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentLockSet.zeroDurationLockObject(Unknown Source) at org.apache.derby.impl.services.locks.AbstractPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.services.locks.ConcurrentPool.zeroDurationlockObject(Unknown Source) at org.apache.derby.impl.store.raw.xact.RowLocking2nohold.lockRecordForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.lockPositionForRead(Unknown Source) at org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(Unknown Source) at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(Unknown Source) at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(Unknown Source) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(Unknown Source) ... 153 common frames omitted 2022-06-10T14:58:54.816Z [jfac ] [WARN ] [7bca83c363bdd35d] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-6] - 5 records lost 2022-06-10T14:58:55.699Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 180 Elapsed 4.7 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:58:55 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:58:57.866Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:58:57.934Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:58:58.116Z [jfrou] [ERROR] [5c0b7f2dead85565] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 54 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:58:58.188Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:58:58.236Z [jfrou] [WARN ] [1d763129054966a1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:58:58.236Z [jfrou] [WARN ] [1d763129054966a1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:03.239Z [jfrou] [ERROR] [1d763129054966a1] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 55 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:03.349Z [jfrou] [WARN ] [740aaaf9118d1f78] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:03.350Z [jfrou] [WARN ] [740aaaf9118d1f78] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:06.996Z [jfac ] [WARN ] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:177 ] [jf-common-pool-3 ] - Retry 190 Elapsed 4.89 minutes failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:06 GMT,content-length=19) DATA----------------------------- Service Unavailable. Trying again 2022-06-10T14:59:07.858Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:59:07.933Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:59:08.217Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:59:08.359Z [jfrou] [ERROR] [740aaaf9118d1f78] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 56 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:08.477Z [jfrou] [WARN ] [086713956ad93936] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:08.477Z [jfrou] [WARN ] [086713956ad93936] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:10.071Z [jfrou] [ERROR] [62db097961380e18] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:59:13.482Z [jfrou] [ERROR] [086713956ad93936] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 57 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:13.604Z [jfrou] [WARN ] [0bcdd13a4505c6e5] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:13.604Z [jfrou] [WARN ] [0bcdd13a4505c6e5] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:15.042Z [jfac ] [WARN ] [48223cd03a43b805] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-8] - 5 records lost 2022-06-10T14:59:17.858Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:59:17.939Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:59:18.228Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:59:18.607Z [jfrou] [ERROR] [0bcdd13a4505c6e5] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 58 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:18.720Z [jfrou] [WARN ] [26ba6a3cb03a72ef] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:18.720Z [jfrou] [WARN ] [26ba6a3cb03a72ef] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:18.776Z [jfac ] [ERROR] [ec20a1b61cc7f41e] [o.j.c.ExecutionUtils:190 ] [jf-common-pool-3 ] - Last retry failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:18 GMT,content-length=19) DATA----------------------------- Service Unavailable. Not trying again (exceeded number of attempts (200)) 2022-06-10T14:59:18.781Z [jfac ] [ERROR] [ec20a1b61cc7f41e] [a.s.b.AccessServerRegistrar:79] [jf-common-pool-1 ] - Could not register access java.util.concurrent.CompletionException: java.util.concurrent.ExecutionException: org.jfrog.common.ExecutionFailed: Last retry failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:18 GMT,content-length=19) DATA----------------------------- Service Unavailable. Not trying again (exceeded number of attempts (200)) at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:787) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) at org.jfrog.common.ExecutionUtils.handleFunctionExecution(ExecutionUtils.java:135) at org.jfrog.common.ExecutionUtils.lambda$generateExecutionRunnable$1(ExecutionUtils.java:102) at org.jfrog.common.ExecutionUtils$MDCRunnableDecorator.run(ExecutionUtils.java:207) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.util.concurrent.ExecutionException: org.jfrog.common.ExecutionFailed: Last retry failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:18 GMT,content-length=19) DATA----------------------------- Service Unavailable. Not trying again (exceeded number of attempts (200)) at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999) at org.jfrog.access.server.service.communication.ServerCommunicationValidator.pingWithRetries(ServerCommunicationValidator.java:33) at org.jfrog.access.server.bootstrap.AccessServerRegistrar.pingSelf(AccessServerRegistrar.java:108) at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:783) ... 12 common frames omitted Caused by: org.jfrog.common.ExecutionFailed: Last retry failed: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:18 GMT,content-length=19) DATA----------------------------- Service Unavailable. Not trying again (exceeded number of attempts (200)) at org.jfrog.common.ExecutionUtils.handleStopError(ExecutionUtils.java:191) at org.jfrog.common.ExecutionUtils.handleFunctionExecution(ExecutionUtils.java:138) ... 9 common frames omitted Caused by: org.jfrog.common.RetryException: Couldn't access another access peer. [localhost:8046]. Status code: UNAVAILABLE. HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:18 GMT,content-length=19) DATA----------------------------- Service Unavailable at org.jfrog.access.server.service.communication.grpc.GrpcRetryableCallSupplier.handleStatusRuntimeException(GrpcRetryableCallSupplier.java:44) at org.jfrog.access.server.service.communication.grpc.GrpcRetryableCallSupplier.handleGrpcInvocationException(GrpcRetryableCallSupplier.java:25) at org.jfrog.access.server.service.communication.grpc.GrpcRetryableCallSupplier.lambda$createRetryableCall$0(GrpcRetryableCallSupplier.java:16) at org.jfrog.common.ExecutionUtils.handleFunctionExecution(ExecutionUtils.java:135) ... 9 common frames omitted Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: HTTP status code 503 invalid content-type: text/plain; charset=utf-8 headers: Metadata(:status=503,content-type=text/plain; charset=utf-8,date=Fri, 10 Jun 2022 14:59:18 GMT,content-length=19) DATA----------------------------- Service Unavailable at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262) at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243) at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156) at org.jfrog.access.proto.generated.SystemResourceGrpc$SystemResourceBlockingStub.ping(SystemResourceGrpc.java:178) at org.jfrog.access.server.service.communication.ServerCommunicationValidator.lambda$pingWithRetries$0(ServerCommunicationValidator.java:31) at org.jfrog.access.server.service.communication.grpc.GrpcRetryableCallSupplier.handleGrpcInvocationException(GrpcRetryableCallSupplier.java:23) ... 11 common frames omitted 2022-06-10T14:59:23.744Z [jfrou] [ERROR] [26ba6a3cb03a72ef] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 59 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:23.860Z [jfrou] [WARN ] [1a83c0f528a00dd3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:23.860Z [jfrou] [WARN ] [1a83c0f528a00dd3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:25.174Z [jfrou] [ERROR] [7f1246b255ab22ab] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:59:27.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:59:27.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:59:28.169Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:59:28.866Z [jfrou] [ERROR] [1a83c0f528a00dd3] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 60 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:28.975Z [jfrou] [WARN ] [3bc84939409e192e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:28.976Z [jfrou] [WARN ] [3bc84939409e192e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:33.995Z [jfrou] [ERROR] [3bc84939409e192e] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 61 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:34.110Z [jfrou] [WARN ] [69bb1c2d54cb87d6] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:34.110Z [jfrou] [WARN ] [69bb1c2d54cb87d6] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:35.219Z [jfac ] [WARN ] [5c0b7f2dead85565] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-2] - 5 records lost 2022-06-10T14:59:35.654Z [jfac ] [WARN ] [1d763129054966a1] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-4] - 5 records lost 2022-06-10T14:59:37.859Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:59:37.945Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:59:38.176Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:59:39.113Z [jfrou] [ERROR] [69bb1c2d54cb87d6] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 62 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:39.221Z [jfrou] [WARN ] [6772314cc733624b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:39.222Z [jfrou] [WARN ] [6772314cc733624b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:44.224Z [jfrou] [ERROR] [6772314cc733624b] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 63 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:44.332Z [jfrou] [WARN ] [1f95d325bc56dc24] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:44.332Z [jfrou] [WARN ] [1f95d325bc56dc24] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:47.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:59:47.937Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:59:48.191Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:59:49.335Z [jfrou] [ERROR] [1f95d325bc56dc24] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 64 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:49.444Z [jfrou] [WARN ] [0d2a582143b30244] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:49.444Z [jfrou] [WARN ] [0d2a582143b30244] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:50.988Z [jfrou] [ERROR] [4434cb66fc0f5179] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T14:59:54.447Z [jfrou] [ERROR] [0d2a582143b30244] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 65 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:54.555Z [jfrou] [WARN ] [0b20d0189e4fd6ae] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:54.556Z [jfrou] [WARN ] [0b20d0189e4fd6ae] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T14:59:55.889Z [jfac ] [WARN ] [0bcdd13a4505c6e5] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-6] - 5 records lost 2022-06-10T14:59:55.893Z [jfac ] [WARN ] [69bb1c2d54cb87d6] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-15] - 5 records lost 2022-06-10T14:59:56.359Z [jfac ] [WARN ] [26ba6a3cb03a72ef] [a.s.s.e.TopologyServiceImpl:95] [-default-executor-10] - 5 records lost 2022-06-10T14:59:57.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T14:59:57.932Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T14:59:58.172Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T14:59:59.351Z [jfac ] [WARN ] [1a83c0f528a00dd3] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-5] - 5 records lost 2022-06-10T14:59:59.559Z [jfrou] [ERROR] [0b20d0189e4fd6ae] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 66 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T14:59:59.671Z [jfrou] [WARN ] [0383ee1e3db9d867] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T14:59:59.673Z [jfrou] [WARN ] [0383ee1e3db9d867] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:04.676Z [jfrou] [ERROR] [0383ee1e3db9d867] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 67 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T15:00:04.787Z [jfrou] [WARN ] [15563e66274c8906] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:04.792Z [jfrou] [WARN ] [15563e66274c8906] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:06.093Z [jfrou] [ERROR] [745bff362133e10f] [external_topology.go:78 ] [main ] - Failed fetching external topology from Access: Get "http://localhost:8040/access/api/v1/topology": context deadline exceeded 2022-06-10T15:00:07.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:00:07.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:00:08.169Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:00:09.796Z [jfrou] [ERROR] [15563e66274c8906] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 68 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T15:00:09.911Z [jfrou] [WARN ] [72adce30ac7d52d3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:09.912Z [jfrou] [WARN ] [72adce30ac7d52d3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:14.918Z [jfrou] [ERROR] [72adce30ac7d52d3] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 69 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T15:00:15.031Z [jfrou] [WARN ] [4e1e72c1bf23619b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:15.032Z [jfrou] [WARN ] [4e1e72c1bf23619b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:17.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:00:17.939Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:00:18.190Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:00:19.519Z [jfac ] [WARN ] [6772314cc733624b] [a.s.s.e.TopologyServiceImpl:95] [c-default-executor-9] - 5 records lost 2022-06-10T15:00:20.037Z [jfrou] [ERROR] [4e1e72c1bf23619b] [local_topology.go:134 ] [main ] - periodic send heartbeat failed for 70 consecutive times. Last error: failed sending heartbeat information to Access: failed closing Access grpc client: closing heartbeat client and waiting for response timed-out 2022-06-10T15:00:20.152Z [jfrou] [WARN ] [35291d5e64d839d1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:20.152Z [jfrou] [WARN ] [35291d5e64d839d1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:20.202Z [jfrou] [INFO ] [35291d5e64d839d1] [local_topology.go:147 ] [main ] - Periodic send heartbeat succeeded after 70 consecutive failures. 2022-06-10T15:00:25.156Z [jfrou] [WARN ] [484a14be559787a2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:25.156Z [jfrou] [WARN ] [484a14be559787a2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:27.876Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:00:27.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:00:28.176Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:00:30.144Z [jfrou] [WARN ] [2f9179fe4bbaa958] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:30.144Z [jfrou] [WARN ] [2f9179fe4bbaa958] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:35.145Z [jfrou] [WARN ] [2d3492060263071c] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:35.145Z [jfrou] [WARN ] [2d3492060263071c] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:37.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:00:37.928Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:00:38.156Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:00:40.159Z [jfrou] [WARN ] [5df596757278b428] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:40.160Z [jfrou] [WARN ] [5df596757278b428] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:45.149Z [jfrou] [WARN ] [16d6d7d645d0b9f2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:45.149Z [jfrou] [WARN ] [16d6d7d645d0b9f2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:47.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:00:47.921Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:00:48.181Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:00:50.150Z [jfrou] [WARN ] [30f51c10af7e9d2f] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:50.150Z [jfrou] [WARN ] [30f51c10af7e9d2f] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:55.148Z [jfrou] [WARN ] [17b41a4fbff0ff9e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:00:55.149Z [jfrou] [WARN ] [17b41a4fbff0ff9e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:00:57.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:00:57.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:00:58.174Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:01:00.151Z [jfrou] [WARN ] [4542e83eeaaa0a05] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:00.151Z [jfrou] [WARN ] [4542e83eeaaa0a05] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:05.161Z [jfrou] [WARN ] [02501e610f8d2cc3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:05.161Z [jfrou] [WARN ] [02501e610f8d2cc3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:07.875Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:01:07.930Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:01:08.188Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:01:10.160Z [jfrou] [WARN ] [73ed50e4c7dc22be] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:10.160Z [jfrou] [WARN ] [73ed50e4c7dc22be] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:15.167Z [jfrou] [WARN ] [0acc8f3fa6d70fb3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:15.167Z [jfrou] [WARN ] [0acc8f3fa6d70fb3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:17.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:01:17.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:01:18.169Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:01:20.156Z [jfrou] [WARN ] [3cc68f0e5e5fe45a] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:20.156Z [jfrou] [WARN ] [3cc68f0e5e5fe45a] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:25.160Z [jfrou] [WARN ] [609404a067a25510] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:25.160Z [jfrou] [WARN ] [609404a067a25510] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:27.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:01:27.919Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:01:28.151Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:01:30.159Z [jfrou] [WARN ] [407c0cbb42cfaf0f] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:30.159Z [jfrou] [WARN ] [407c0cbb42cfaf0f] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:35.160Z [jfrou] [WARN ] [0016b1992c04e788] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:35.160Z [jfrou] [WARN ] [0016b1992c04e788] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:37.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:01:37.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:01:38.147Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:01:40.159Z [jfrou] [WARN ] [5a642a038ec2bc69] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:40.159Z [jfrou] [WARN ] [5a642a038ec2bc69] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:45.169Z [jfrou] [WARN ] [19bd7e5623c049b2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:45.169Z [jfrou] [WARN ] [19bd7e5623c049b2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:45.948Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T15:01:46.059Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T15:01:46.059Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 6m24s) [usage_service] 2022-06-10T15:01:47.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:01:47.928Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:01:48.146Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:01:50.159Z [jfrou] [WARN ] [59336aafe80e9451] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:50.159Z [jfrou] [WARN ] [59336aafe80e9451] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:55.166Z [jfrou] [WARN ] [0ef7543ff0a66c52] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:01:55.166Z [jfrou] [WARN ] [0ef7543ff0a66c52] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:01:57.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:01:57.930Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:01:58.178Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:02:00.160Z [jfrou] [WARN ] [5f00cc811ba47918] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:00.161Z [jfrou] [WARN ] [5f00cc811ba47918] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:05.161Z [jfrou] [WARN ] [540b273789ef29db] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:05.161Z [jfrou] [WARN ] [540b273789ef29db] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:07.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:02:07.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:02:08.158Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:02:10.170Z [jfrou] [WARN ] [2697e251a12da194] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:10.170Z [jfrou] [WARN ] [2697e251a12da194] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:15.162Z [jfrou] [WARN ] [2a5721d0e787a811] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:15.162Z [jfrou] [WARN ] [2a5721d0e787a811] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:17.862Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:02:17.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:02:18.196Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:02:20.167Z [jfrou] [WARN ] [27bb1b33f881e7b6] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:20.167Z [jfrou] [WARN ] [27bb1b33f881e7b6] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:25.177Z [jfrou] [WARN ] [1ec365431d1c4f09] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:25.178Z [jfrou] [WARN ] [1ec365431d1c4f09] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:27.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:02:27.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:02:28.156Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:02:30.169Z [jfrou] [WARN ] [5d89489212ba5ae5] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:30.170Z [jfrou] [WARN ] [5d89489212ba5ae5] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:35.171Z [jfrou] [WARN ] [0181f48524140777] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:35.171Z [jfrou] [WARN ] [0181f48524140777] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:37.865Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:02:37.928Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:02:38.161Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:02:40.168Z [jfrou] [WARN ] [730100257eab8a94] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:40.168Z [jfrou] [WARN ] [730100257eab8a94] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:45.167Z [jfrou] [WARN ] [39f02365e44fdf23] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:45.167Z [jfrou] [WARN ] [39f02365e44fdf23] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:47.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:02:47.925Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:02:48.161Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:02:50.187Z [jfrou] [WARN ] [09c7d64e38b8a3ea] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:50.188Z [jfrou] [WARN ] [09c7d64e38b8a3ea] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:55.179Z [jfrou] [WARN ] [090b2626b7f88798] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:02:55.180Z [jfrou] [WARN ] [090b2626b7f88798] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:02:57.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:02:57.921Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:02:58.178Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:03:00.172Z [jfrou] [WARN ] [6f5634e6d3b3bbbc] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:00.173Z [jfrou] [WARN ] [6f5634e6d3b3bbbc] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:05.180Z [jfrou] [WARN ] [0aed3d951d4ffe08] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:05.181Z [jfrou] [WARN ] [0aed3d951d4ffe08] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:07.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:03:07.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:03:08.164Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:03:10.175Z [jfrou] [WARN ] [678d60a646ccf66f] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:10.175Z [jfrou] [WARN ] [678d60a646ccf66f] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:15.176Z [jfrou] [WARN ] [73b4ec929d3773db] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:15.176Z [jfrou] [WARN ] [73b4ec929d3773db] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:17.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:03:17.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:03:18.162Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:03:20.179Z [jfrou] [WARN ] [534f14aea4ee3b9b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:20.179Z [jfrou] [WARN ] [534f14aea4ee3b9b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:25.178Z [jfrou] [WARN ] [48b24038a0f4c44a] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:25.178Z [jfrou] [WARN ] [48b24038a0f4c44a] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:27.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:03:27.931Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:03:28.187Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:03:30.182Z [jfrou] [WARN ] [728f3d867a9030d0] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:30.183Z [jfrou] [WARN ] [728f3d867a9030d0] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:35.179Z [jfrou] [WARN ] [19496363e3036329] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:35.179Z [jfrou] [WARN ] [19496363e3036329] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:37.860Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:03:37.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:03:38.170Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:03:40.182Z [jfrou] [WARN ] [643b8527f15719f8] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:40.182Z [jfrou] [WARN ] [643b8527f15719f8] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:45.180Z [jfrou] [WARN ] [218a16b1336bdb18] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:45.180Z [jfrou] [WARN ] [218a16b1336bdb18] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:47.868Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:03:47.927Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:03:48.150Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:03:50.191Z [jfrou] [WARN ] [79a4ba6383fb9935] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:50.191Z [jfrou] [WARN ] [79a4ba6383fb9935] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:55.205Z [jfrou] [WARN ] [40711d32479e62ee] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:03:55.207Z [jfrou] [WARN ] [40711d32479e62ee] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:03:57.856Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:03:57.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:03:58.196Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:04:00.182Z [jfrou] [WARN ] [59c548409ac7e802] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:00.183Z [jfrou] [WARN ] [59c548409ac7e802] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:05.193Z [jfrou] [WARN ] [77fffcae7173e83a] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:05.193Z [jfrou] [WARN ] [77fffcae7173e83a] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:07.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:04:07.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:04:08.159Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:04:10.185Z [jfrou] [WARN ] [4761c4bdc2fadf29] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:10.186Z [jfrou] [WARN ] [4761c4bdc2fadf29] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:15.184Z [jfrou] [WARN ] [23cd4fbc4cc2f5b5] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:15.184Z [jfrou] [WARN ] [23cd4fbc4cc2f5b5] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:17.863Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:04:17.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:04:18.176Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:04:20.185Z [jfrou] [WARN ] [0fa4ec5516bfb129] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:20.185Z [jfrou] [WARN ] [0fa4ec5516bfb129] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:25.187Z [jfrou] [WARN ] [184a1140f10be15c] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:25.187Z [jfrou] [WARN ] [184a1140f10be15c] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:27.853Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:04:27.931Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:04:28.160Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:04:30.185Z [jfrou] [WARN ] [5e7e422f0ecccebe] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:30.185Z [jfrou] [WARN ] [5e7e422f0ecccebe] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:35.196Z [jfrou] [WARN ] [1786f34224d714aa] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:35.196Z [jfrou] [WARN ] [1786f34224d714aa] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:37.852Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:04:37.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:04:38.175Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:04:40.188Z [jfrou] [WARN ] [32fd2d3aad8dfbe3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:40.189Z [jfrou] [WARN ] [32fd2d3aad8dfbe3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:45.206Z [jfrou] [WARN ] [0d254d6f4593bb41] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:45.206Z [jfrou] [WARN ] [0d254d6f4593bb41] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:47.853Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:04:47.938Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:04:48.182Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:04:50.201Z [jfrou] [WARN ] [209d3906723cb778] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:50.201Z [jfrou] [WARN ] [209d3906723cb778] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:55.197Z [jfrou] [WARN ] [6dc9145133d8e6f2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:04:55.198Z [jfrou] [WARN ] [6dc9145133d8e6f2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:04:57.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:04:57.949Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:04:58.217Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:05:00.194Z [jfrou] [WARN ] [0e36c532512c4bb2] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:00.194Z [jfrou] [WARN ] [0e36c532512c4bb2] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:05.201Z [jfrou] [WARN ] [0dc1e327fbae91aa] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:05.201Z [jfrou] [WARN ] [0dc1e327fbae91aa] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:07.861Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:05:07.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:05:08.169Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:05:10.197Z [jfrou] [WARN ] [18caf0608a4e8135] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:10.198Z [jfrou] [WARN ] [18caf0608a4e8135] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:15.201Z [jfrou] [WARN ] [1270ba9b5ffe93ab] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:15.201Z [jfrou] [WARN ] [1270ba9b5ffe93ab] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:17.866Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:05:17.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:05:18.170Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:05:20.198Z [jfrou] [WARN ] [1fd2274ce0a55517] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:20.199Z [jfrou] [WARN ] [1fd2274ce0a55517] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:25.201Z [jfrou] [WARN ] [58a995d25e6ea583] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:25.201Z [jfrou] [WARN ] [58a995d25e6ea583] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:27.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:05:27.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:05:28.181Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:05:30.199Z [jfrou] [WARN ] [69bbf4d588910acb] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:30.200Z [jfrou] [WARN ] [69bbf4d588910acb] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:35.210Z [jfrou] [WARN ] [26426e378120a7fc] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:35.211Z [jfrou] [WARN ] [26426e378120a7fc] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:37.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:05:37.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:05:38.179Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:05:40.200Z [jfrou] [WARN ] [49552cf3479ec1c0] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:40.200Z [jfrou] [WARN ] [49552cf3479ec1c0] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:45.204Z [jfrou] [WARN ] [1822c9ef82733978] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:45.204Z [jfrou] [WARN ] [1822c9ef82733978] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:47.853Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:05:47.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:05:48.159Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:05:50.202Z [jfrou] [WARN ] [12cf7d86b4e0bbd8] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:50.202Z [jfrou] [WARN ] [12cf7d86b4e0bbd8] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:55.206Z [jfrou] [WARN ] [62de00645a9d208b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:05:55.206Z [jfrou] [WARN ] [62de00645a9d208b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:05:57.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:05:57.933Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:05:58.173Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:06:00.205Z [jfrou] [WARN ] [7e335d1f38b2ba52] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:00.205Z [jfrou] [WARN ] [7e335d1f38b2ba52] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:05.214Z [jfrou] [WARN ] [63762bac04e6ea02] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:05.214Z [jfrou] [WARN ] [63762bac04e6ea02] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:07.856Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:06:07.945Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:06:08.163Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:06:10.208Z [jfrou] [WARN ] [5b588b300fe62c71] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:10.209Z [jfrou] [WARN ] [5b588b300fe62c71] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:15.207Z [jfrou] [WARN ] [6057ae0148524e4d] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:15.207Z [jfrou] [WARN ] [6057ae0148524e4d] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:17.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:06:17.919Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:06:18.149Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:06:20.224Z [jfrou] [WARN ] [4ff25eae27017f16] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:20.225Z [jfrou] [WARN ] [4ff25eae27017f16] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:25.209Z [jfrou] [WARN ] [330b3e81325a04ac] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:25.209Z [jfrou] [WARN ] [330b3e81325a04ac] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:27.856Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:06:27.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:06:28.150Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:06:30.206Z [jfrou] [WARN ] [06b5ce2ceb2d2d13] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:30.206Z [jfrou] [WARN ] [06b5ce2ceb2d2d13] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:35.208Z [jfrou] [WARN ] [6ba75118212c4ec3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:35.208Z [jfrou] [WARN ] [6ba75118212c4ec3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:37.857Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:06:37.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:06:38.160Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:06:40.211Z [jfrou] [WARN ] [4ec5763b0607387e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:40.211Z [jfrou] [WARN ] [4ec5763b0607387e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:45.211Z [jfrou] [WARN ] [23d16cdba6cef00f] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:45.211Z [jfrou] [WARN ] [23d16cdba6cef00f] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:47.856Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:06:47.933Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:06:48.148Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:06:50.211Z [jfrou] [WARN ] [2e8483dd9a5d5c14] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:50.211Z [jfrou] [WARN ] [2e8483dd9a5d5c14] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:55.243Z [jfrou] [WARN ] [4f0bb37c9ce40ce3] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:06:55.247Z [jfrou] [WARN ] [4f0bb37c9ce40ce3] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:06:57.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:06:57.925Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:06:58.141Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:07:00.215Z [jfrou] [WARN ] [7c2df9b688d3beb1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:00.215Z [jfrou] [WARN ] [7c2df9b688d3beb1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:05.228Z [jfrou] [WARN ] [77a333f8c1f32a25] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:05.229Z [jfrou] [WARN ] [77a333f8c1f32a25] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:07.852Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:07:07.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:07:08.170Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:07:10.224Z [jfrou] [WARN ] [434d8fde5695cf0a] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:10.224Z [jfrou] [WARN ] [434d8fde5695cf0a] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:15.230Z [jfrou] [WARN ] [29ae139358e40c1e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:15.230Z [jfrou] [WARN ] [29ae139358e40c1e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:17.859Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:07:17.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:07:18.176Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:07:20.214Z [jfrou] [WARN ] [62dcb6a8e1daaf4f] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:20.215Z [jfrou] [WARN ] [62dcb6a8e1daaf4f] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:25.216Z [jfrou] [WARN ] [09ac683e17ca5392] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:25.216Z [jfrou] [WARN ] [09ac683e17ca5392] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:27.852Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:07:27.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:07:28.185Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:07:30.220Z [jfrou] [WARN ] [2fcf79f06761952b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:30.220Z [jfrou] [WARN ] [2fcf79f06761952b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:35.217Z [jfrou] [WARN ] [5142307ad25f24f0] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:35.217Z [jfrou] [WARN ] [5142307ad25f24f0] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:37.853Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:07:37.925Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:07:38.148Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:07:40.219Z [jfrou] [WARN ] [3d00f3a3737ae278] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:40.219Z [jfrou] [WARN ] [3d00f3a3737ae278] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:45.219Z [jfrou] [WARN ] [27f016015131659f] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:45.219Z [jfrou] [WARN ] [27f016015131659f] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:47.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:07:47.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:07:48.154Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:07:50.224Z [jfrou] [WARN ] [193c7d71f94edc32] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:50.225Z [jfrou] [WARN ] [193c7d71f94edc32] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:55.225Z [jfrou] [WARN ] [0505add4d33752e0] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:07:55.226Z [jfrou] [WARN ] [0505add4d33752e0] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:07:57.869Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:07:57.931Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:07:58.169Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:08:00.238Z [jfrou] [WARN ] [1fd89f2b8220de2a] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:00.238Z [jfrou] [WARN ] [1fd89f2b8220de2a] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:05.226Z [jfrou] [WARN ] [640494615a2160e8] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:05.226Z [jfrou] [WARN ] [640494615a2160e8] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:07.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:08:07.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:08:08.163Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:08:10.186Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:139 ] [main ] - Starting usage collection [usage_service] 2022-06-10T15:08:10.224Z [jfrou] [WARN ] [3254994f2a2092a8] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:10.224Z [jfrou] [WARN ] [3254994f2a2092a8] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:10.291Z [jfcon] [INFO ] [40e550fc80ab73a0] [usage_service.go:153 ] [main ] - Finished usage collection [usage_service] 2022-06-10T15:08:10.291Z [jfcon] [ERROR] [40e550fc80ab73a0] [usage_service.go:130 ] [main ] - Usage flow failed; failed to fetch cunsumption files, response status: 404 Not Found (returned 404) (Next try in 12m48s) [usage_service] 2022-06-10T15:08:15.236Z [jfrou] [WARN ] [591cab41d91df798] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:15.237Z [jfrou] [WARN ] [591cab41d91df798] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:17.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:08:17.921Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:08:18.140Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:08:20.229Z [jfrou] [WARN ] [2a3d6f944492b8c1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:20.229Z [jfrou] [WARN ] [2a3d6f944492b8c1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:25.228Z [jfrou] [WARN ] [777c135fa5c0b546] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:25.228Z [jfrou] [WARN ] [777c135fa5c0b546] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:27.852Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:08:27.929Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:08:28.145Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:08:30.227Z [jfrou] [WARN ] [038db89f0a9c8388] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:30.228Z [jfrou] [WARN ] [038db89f0a9c8388] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:35.231Z [jfrou] [WARN ] [51dfcc32fcc7d913] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:35.231Z [jfrou] [WARN ] [51dfcc32fcc7d913] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:37.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:08:37.923Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:08:38.160Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:08:40.231Z [jfrou] [WARN ] [44b528a62d50b94c] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:40.232Z [jfrou] [WARN ] [44b528a62d50b94c] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:45.230Z [jfrou] [WARN ] [420e309bf64f95a4] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:45.231Z [jfrou] [WARN ] [420e309bf64f95a4] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:47.859Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:08:47.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:08:48.145Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:08:50.238Z [jfrou] [WARN ] [6b3330428b676b3b] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:50.239Z [jfrou] [WARN ] [6b3330428b676b3b] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:55.230Z [jfrou] [WARN ] [75246d8c154d0f60] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:08:55.231Z [jfrou] [WARN ] [75246d8c154d0f60] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:08:57.871Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:08:57.924Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:08:58.146Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:09:00.230Z [jfrou] [WARN ] [54f9dd600c6a972e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:00.231Z [jfrou] [WARN ] [54f9dd600c6a972e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:05.237Z [jfrou] [WARN ] [4a09137da81fd092] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:05.237Z [jfrou] [WARN ] [4a09137da81fd092] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:07.855Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:09:07.921Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:09:08.144Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:09:10.244Z [jfrou] [WARN ] [3b0d6e5626b548cb] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:10.244Z [jfrou] [WARN ] [3b0d6e5626b548cb] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:15.232Z [jfrou] [WARN ] [57c7ed6b813fdfb1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:15.232Z [jfrou] [WARN ] [57c7ed6b813fdfb1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:17.856Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:09:17.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:09:18.164Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:09:20.238Z [jfrou] [WARN ] [1fdc1bc0c90cb53e] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:20.238Z [jfrou] [WARN ] [1fdc1bc0c90cb53e] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:25.235Z [jfrou] [WARN ] [2e9f5cc396fdef06] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:25.235Z [jfrou] [WARN ] [2e9f5cc396fdef06] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:27.862Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:09:27.922Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:09:28.189Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:09:30.244Z [jfrou] [WARN ] [18a56a930de79cf7] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:30.245Z [jfrou] [WARN ] [18a56a930de79cf7] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:35.236Z [jfrou] [WARN ] [28be34b569fc13db] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:35.236Z [jfrou] [WARN ] [28be34b569fc13db] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:37.854Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:09:37.926Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:09:38.176Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:09:40.237Z [jfrou] [WARN ] [1084ff6bf9782fae] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:40.237Z [jfrou] [WARN ] [1084ff6bf9782fae] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:45.257Z [jfrou] [WARN ] [2d051494502f2cf1] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:45.258Z [jfrou] [WARN ] [2d051494502f2cf1] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy" 2022-06-10T15:09:47.863Z [jfevt] [WARN ] [5461784fd2183141] [proxy_synchronizer.go:22 ] [main ] - Proxy cache is not updated because of error: failure to get the list of proxies from Artifactory (returned 404) [proxy_cache] 2022-06-10T15:09:47.920Z [jfevt] [WARN ] [5461784fd2183141] [base_url_cache.go:70 ] [main ] - Base URL cache is not updated because of error: failed to fetch base URL from Artifactory (returned 404) [baseurl_cache] 2022-06-10T15:09:48.162Z [jfevt] [WARN ] [5461784fd2183141] [domain_client.go:104 ] [main ] - Error occurred when refreshing domain cache mandatory domains are missing: could not get domains from Artifactory: Fetch domains from http://localhost:8046/artifactory/api/events/domains failed (returned 404) [domain_client] 2022-06-10T15:09:50.244Z [jfrou] [WARN ] [20168fd9cca8bc00] [local_topology_helper.go:65 ] [main ] - Missing required services: [jfrt jffe jfob] 2022-06-10T15:09:50.244Z [jfrou] [WARN ] [20168fd9cca8bc00] [local_topology.go:274 ] [main ] - Readiness test failed with the following error: "required node services are missing or unhealthy"