Nexus was not able to be reached and we are getting a 502 error. We had the same issue a couple of months ago, which we resolved by upgrading the version from 3.41.0-01 to 3.50.0-01. At that time, we determined that the issue was possibly caused by a corrupt file. However, it seems the issue has occurred again.
Looking at the logs, it shows that the system shut down and was restarted. During startup, Nexus failed to resolve the artifact com.sonatype.nexus.assemblies:nexus-flags-feature:xml:features:3.50.0-01 from the Maven Central repository. This issue, indicated by a java.io.IOException, may be due to the artifact being missing, a misconfigured repository, or network problems.
I’m not really familiar with this service and would like some advice what could be the issues.
I got the service back up by rebooting and restarting the service a couple of times, which still don’t understand the root cause of the issue. I verified the filesystem usage, which has a total size of about 20 GB, with around 11.7 GB used and 8.2 GB available, representing 59% of the space used. I’m thinking it might be due to the cache being full with old data, but I’m not sure.
df -m
Filesystem 1M-blocks Used Available Use% Mounted on
devtmpfs 3854 1 3854 1% /dev
tmpfs 3864 1 3864 1% /dev/shm
/dev/nvme0n1p1 20029 11749 8183 59% /
Error logs
2024-07-26 18:30:00,022+0000 INFO [quartz-17-thread-5] *SYSTEM org.sonatype.nexus.scheduling.TaskSchedulerImpl - Task 'System - Repository Health Check: docker-proxy' [healthcheck] scheduled: hourly
2024-07-26 18:30:00,038+0000 INFO [quartz-17-thread-5] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'System - Repository Health Check: docker-proxy' [healthcheck] state change RUNNING -> WAITING (OK)
2024-07-26 18:30:00,044+0000 INFO [quartz-17-thread-6] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'Storage facet cleanup' [repository.storage-facet-cleanup] state change WAITING -> RUNNING
2024-07-26 18:30:00,098+0000 INFO [quartz-17-thread-6] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'Storage facet cleanup' [repository.storage-facet-cleanup] state change RUNNING -> WAITING (OK)
2024-07-26 18:30:00,483+0000 INFO [quartz-17-thread-4] *SYSTEM org.sonatype.nexus.scheduling.TaskSchedulerImpl - Task 'System - Repository Health Check: pypi-proxy' [healthcheck] scheduled: hourly
2024-07-26 18:30:00,485+0000 INFO [quartz-17-thread-4] *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskInfo - Task 'System - Repository Health Check: pypi-proxy' [healthcheck] state change RUNNING -> WAITING (OK)
2024-07-26 18:30:01,381+0000 INFO [jetty-main-1] *SYSTEM org.sonatype.nexus.siesta.SiestaServlet - Initialized
2024-07-26 18:30:01,402+0000 INFO [jetty-main-1] *SYSTEM org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Initialized
2024-07-26 18:30:01,506+0000 INFO [jetty-main-1] *SYSTEM org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.w.WebAppContext@5bdce1b7{Sonatype Nexus,/,file:///opt/nexus/nexus-3.50.0-01/public/,AVAILABLE}
2024-07-26 18:30:01,648+0000 INFO [jetty-main-1] *SYSTEM org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@76d5ebe1{HTTP/1.1, (http/1.1)}{0.0.0.0:8081}
2024-07-26 18:30:01,649+0000 INFO [jetty-main-1] *SYSTEM org.eclipse.jetty.server.Server - Started @70710ms
2024-07-26 18:30:01,649+0000 INFO [jetty-main-1] *SYSTEM org.sonatype.nexus.bootstrap.jetty.JettyServer -
-------------------------------------------------
Started Sonatype Nexus OSS 3.50.0-01
-------------------------------------------------
2024-07-26 18:30:01,663+0000 INFO [jetty-main-1] *SYSTEM org.eclipse.jetty.server.AbstractConnector - Started ServerConnector@62441b53{HTTP/1.1, (http/1.1)}{0.0.0.0:8082}
2024-07-26 18:30:02,727+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:03,661+0000 INFO [qtp1544016644-82] *UNKNOWN org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Enabling session validation scheduler...
2024-07-26 18:30:03,709+0000 INFO [qtp1544016644-82] *UNKNOWN org.sonatype.nexus.internal.security.anonymous.AnonymousManagerImpl - Loaded configuration: OrientAnonymousConfiguration{enabled=true, userId='anonymous', realmName='NexusAuthorizingRealm'}
2024-07-26 18:30:05,537+0000 INFO [qtp1544016644-86] *UNKNOWN org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for npm-proxy changed from READY to AVAILABLE - reason n/a for n/a
2024-07-26 18:30:09,356+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:14,632+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:15,524+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:16,997+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:33,609+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:42,739+0000 INFO [elasticsearch[7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [7468D7B6-7AF3FB80-64589C43-2BC7A5DC-1AE83DC6] [5487f488ab41b476626080dfcae92a9efd87d9d2] update_mapping [component]
2024-07-26 18:30:50,283+0000 WARN [event-20-thread-6] anonymous org.sonatype.nexus.transaction.RetryController - Exceeded retry limit: 8/8 (org.sonatype.nexus.repository.browse.internal.orient.BrowseNodeCollisionException: Node already has an asset^M
DB name="component")
2024-07-26 18:30:50,295+0000 WARN [event-20-thread-6] anonymous org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeManager - Problem generating browse nodes for Asset{metadata=AttachedEntityMetadata{schema=asset, document=#20:103777{bucket:#16:8,format:npm,last_updated:Fri Jul 26 18:30:48 UTC 2024,attributes:[5],component:null,name:check-error,size:26289,content_type:application/json,created_by:anonymous,created_by_ip:10.32.2.73,blob_ref:npm@9e6c5919-4343-42c8-b1a5-ce7e6724f232,last_downloaded:Fri Jul 26 18:30:48 UTC 2024,blob_created:Thu Mar 17 21:26:08 UTC 2022,blob_updated:Thu May 09 17:05:14 UTC 2024} v723}, name=check-error}
org.sonatype.nexus.repository.browse.internal.orient.BrowseNodeCollisionException: Node already has an asset^M
DB name="component"
at org.sonatype.nexus.repository.browse.internal.orient.BrowseNodeEntityAdapter.createAssetNode(BrowseNodeEntityAdapter.java:308)
at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeStore.lambda$1(OrientBrowseNodeStore.java:168)
at org.sonatype.nexus.orient.transaction.OrientOperations.lambda$2(OrientOperations.java:62)
at org.sonatype.nexus.transaction.OperationPoint.lambda$0(OperationPoint.java:53)
at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64)
at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:58)
at org.sonatype.nexus.transaction.Operations.proceedWithTransaction(Operations.java:237)
at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:228)
at org.sonatype.nexus.transaction.Operations.run(Operations.java:180)
at org.sonatype.nexus.orient.transaction.OrientOperations.run(OrientOperations.java:62)
at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeStore.createAssetNode(OrientBrowseNodeStore.java:168)
at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:54)
at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeManager.createBrowseNodes(OrientBrowseNodeManager.java:137)
at org.sonatype.nexus.repository.browse.internal.orient.OrientBrowseNodeManager.createFromAsset(OrientBrowseNodeManager.java:80)