Nexus service stopped running and I cant start it

deirdrerodgers · February 27, 2023, 5:11pm

I’ve inherited a nexus server from another team that have left the company I work for. I never worked with nexus before so I’m a bit in the dark. We are hosting the server on Ubuntu server in Azure and yesterday it stopped responding out of nowhere. The server hadnt been touched or updated or anything.

I connected to it and saw that the nexus.service was not running so I tried restarting (no error output) and I get the below when I check the status.

[ansible@nexus01 ~]$ sudo systemctl status nexus
● nexus.service - nexus service
Loaded: loaded (/etc/systemd/system/nexus.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2023-02-27 16:58:16 UTC; 3min 4s ago
Process: 16113 ExecStop=/opt/nexus/bin/nexus stop (code=exited, status=0/SUCCESS)
Process: 15961 ExecStart=/opt/nexus/bin/nexus start (code=exited, status=0/SUCCESS)
Main PID: 16091 (code=exited, status=255)

Feb 27 16:58:16 nexus01.du.ccs.ai systemd[16113]: Executing: /opt/nexus/bin/nexus stop
Feb 27 16:58:16 nexus01.du.ccs.ai nexus[16113]: Shutting down nexus
Feb 27 16:58:16 nexus01.du.ccs.ai nexus[16113]: nexus is not running.
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: Child 16113 belongs to nexus.service
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: nexus.service: control process exited, code=exited status=0
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: nexus.service got final SIGCHLD for state stop
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: nexus.service changed stop → failed
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: Unit nexus.service entered failed state.
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: nexus.service failed.
Feb 27 16:58:16 nexus01.du.ccs.ai systemd[1]: nexus.service: cgroup is empty

I tried rebooting the server but no luck. The nexuslog didnt show anything interesting or alarming but then again I am not familiar with the logs. it was mostly INFO level with some WARN level messages

mpiggott · February 27, 2023, 8:02pm

Your nexus application should have a log which you should look at to see if there are errors occurring during startup.

deirdrerodgers · February 28, 2023, 9:59am

I checked all logs and cant see anything corresponding to the times when I try and start the service.

The last thing I see in the nexus logs are for yesterday

rom READY to AVAILABLE - reason n/a for n/a
2023-02-27 13:59:27,611+0000 INFO  [qtp1569194830-459]  *UNKNOWN org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for fn-release-repo changed from READY to AVAILABLE - reason n/a for n/a
2023-02-27 13:59:27,743+0000 INFO  [qtp1569194830-459]  *UNKNOWN org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for everit-org changed from READY to AVAILABLE - reason n/a for n/a
2023-02-27 13:59:27,793+0000 INFO  [qtp1569194830-459]  *UNKNOWN org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for org-owasp changed from READY to AVAILABLE - reason n/a for n/a
2023-02-27 13:59:28,242+0000 INFO  [qtp1569194830-459]  *UNKNOWN org.sonatype.nexus.repository.httpclient.internal.HttpClientFacetImpl - Repository status for redhat-ga changed from READY to AVAILABLE - reason n/a for n/a
2023-02-27 13:59:54,607+0000 WARN  [SIGTERM handler]  *SYSTEM com.orientechnologies.orient.core.OSignalHandler - Received signal: SIGTERM
2023-02-27 13:59:54,616+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Destroyed
2023-02-27 13:59:54,617+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.siesta.SiestaServlet - Destroyed
2023-02-27 13:59:54,632+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusContextListener - Uptime: 2 hours, 24 minutes, 15 seconds and 567 milliseconds (nexus-oss-edition/3.18.1.01)
2023-02-27 13:59:54,632+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Shutting down
2023-02-27 13:59:54,633+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop TASKS
2023-02-27 13:59:54,634+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.quartz.internal.QuartzSchedulerSPI - Scheduler put into stand-by mode
2023-02-27 13:59:54,638+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop CAPABILITIES
2023-02-27 13:59:54,641+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop SERVICES
2023-02-27 13:59:54,964+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop SECURITY
2023-02-27 13:59:54,965+0000 INFO  [FelixStartLevel]  *SYSTEM org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Disabled session validation scheduler.
2023-02-27 13:59:54,965+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop EVENTS
2023-02-27 13:59:54,965+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop SCHEMAS
2023-02-27 13:59:54,967+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop UPGRADE
2023-02-27 13:59:54,967+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop RESTORE
2023-02-27 13:59:54,967+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop STORAGE
2023-02-27 13:59:54,968+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'vaadin#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,968+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'nuget-org-proxy#odata-query-cache' removed from EhcacheManager.
2023-02-27 13:59:54,968+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'archiva-nexus-snapshot#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,969+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'hortonworks#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,969+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'npm-private-ue-mgmt#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,969+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'spring-snapshots#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,969+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'NexusAuthorizingRealm.authorizationCache' removed from EhcacheManager.
2023-02-27 13:59:54,970+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'spring-milestone-local#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,970+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'maven-snapshots-ue-mgmt#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,970+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'archiva-nexus-release#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,970+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'npm-registry#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,970+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'spring-snapshot-local#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,970+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'fn-release-repo#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,971+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'confluent-io#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,971+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'spring-releases#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,971+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'apache-snapshots#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,971+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'artifact-signatures-ue-mgmt#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,972+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'NexusAuthenticatingRealm.authenticationCache' removed from EhcacheManager.
2023-02-27 13:59:54,972+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'springsource#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,972+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'conjars-org#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,972+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'dl-bintray#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,972+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'boundless#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,972+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'graphql-java#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,973+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'maven-releases-ue-mgmt#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,973+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'osgeo#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,973+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'everit-org#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,973+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'jitpack#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,973+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'nuget-group#odata-query-cache' removed from EhcacheManager.
2023-02-27 13:59:54,974+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'shiro-activeSessionCache' removed from EhcacheManager.
2023-02-27 13:59:54,974+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'spring-milestones#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,975+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'LdapRealm.authorizationCache' removed from EhcacheManager.
2023-02-27 13:59:54,975+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'org-owasp#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,975+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'digitalvault-release-ue-mgmt#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,975+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'redhat-ga#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,975+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'docker-hub#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,975+0000 INFO  [FelixStartLevel]  *SYSTEM org.ehcache.core.EhcacheManager - Cache 'nuget-org-proxy#negative-cache' removed from EhcacheManager.
2023-02-27 13:59:54,979+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.cache.internal.ehcache.EhCacheManagerProvider - Cache-manager closed
2023-02-27 13:59:54,979+0000 INFO  [FelixStartLevel]  *SYSTEM org.elasticsearch.node - [EE1BEE78-37B21A24-73F743EB-E81B417F-1947272C] stopping ...
2023-02-27 13:59:55,021+0000 INFO  [FelixStartLevel]  *SYSTEM org.elasticsearch.node - [EE1BEE78-37B21A24-73F743EB-E81B417F-1947272C] stopped
2023-02-27 13:59:55,021+0000 INFO  [FelixStartLevel]  *SYSTEM org.elasticsearch.node - [EE1BEE78-37B21A24-73F743EB-E81B417F-1947272C] closing ...
2023-02-27 13:59:55,028+0000 INFO  [FelixStartLevel]  *SYSTEM org.elasticsearch.node - [EE1BEE78-37B21A24-73F743EB-E81B417F-1947272C] closed
2023-02-27 13:59:55,030+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseManagerImpl - Stopping 3 pools
2023-02-27 13:59:55,030+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseManagerImpl - Stopping pool: security
2023-02-27 13:59:55,030+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseManagerImpl - Stopping pool: component
2023-02-27 13:59:55,030+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseManagerImpl - Stopping pool: config
2023-02-27 13:59:55,030+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl$1 - OrientDB Server is shutting down...
2023-02-27 13:59:55,030+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl$1 - Shutting down protocols
2023-02-27 13:59:55,031+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.server.plugin.OServerPluginManager - Shutting down plugins:
2023-02-27 13:59:55,031+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.server.plugin.OServerPluginManager - - jmx
2023-02-27 13:59:55,031+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl$1 - OrientDB Server shutdown complete
2023-02-27 13:59:55,032+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.core.Orient - Orient Engine is shutting down...
2023-02-27 13:59:55,033+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.core.Orient - - shutdown storage: component...
2023-02-27 13:59:55,193+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.core.Orient - - shutdown storage: security...
2023-02-27 13:59:55,233+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.core.Orient - - shutdown storage: config...
2023-02-27 13:59:55,281+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.core.Orient - - shutdown storage: OSystem...
2023-02-27 13:59:55,837+0000 INFO  [FelixStartLevel]  *SYSTEM com.orientechnologies.orient.core.Orient - OrientDB Engine shutdown complete
2023-02-27 13:59:55,837+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl - Shutdown
2023-02-27 13:59:55,837+0000 INFO  [FelixStartLevel]  *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Stop KERNEL

deirdrerodgers · February 28, 2023, 10:36am

journalctl logs show below when I do a systemctl start nexus.service

Feb 28 10:25:30 nexus01. systemd[1]: Received SIGCHLD from PID 12697 (nexus).
Feb 28 10:25:30 nexus01. systemd[1]: Child 12697 (nexus) died (code=exited, status=0/SUCCESS)
Feb 28 10:25:30 nexus01. systemd[1]: Child 12697 belongs to nexus.service
Feb 28 10:25:30 nexus01. systemd[1]: nexus.service: control process exited, code=exited status=0
Feb 28 10:25:30 nexus01. systemd[1]: nexus.service got final SIGCHLD for state start
Feb 28 10:25:30 nexus01. systemd[1]: Main PID guessed: 12827
Feb 28 10:25:30 nexus01. systemd[1]: nexus.service changed start -> running
Feb 28 10:25:30 nexus01. systemd[1]: Job nexus.service/start finished, result=done
Feb 28 10:25:30 nexus01. systemd[1]: Started nexus service.
-- Subject: Unit nexus.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit nexus.service has finished starting up.
-- 
-- The start-up result is done.
Feb 28 10:25:30 nexus01. systemd[1]: Sent message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=JobRemoved cookie=8 reply_cookie=0 error=n/a
Feb 28 10:25:30 nexus01. systemd[1]: Sent message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=JobRemoved cookie=1803 reply_cookie=0 error=n/a
Feb 28 10:25:30 nexus01. systemd[1]: Sent message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/unit/nexus_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=9 reply_cookie=0 error=n/
Feb 28 10:25:30 nexus01. polkitd[1098]: Unregistered Authentication Agent for unix-process:12691:170387 (system bus name :1.91, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Feb 28 10:25:30 nexus01. systemd[1]: Sent message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/unit/nexus_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=10 reply_cookie=0 error=n
Feb 28 10:25:30 nexus01. systemd[1]: Sent message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/unit/nexus_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=1804 reply_cookie=0 error
Feb 28 10:25:30 nexus01. systemd[1]: Sent message type=signal sender=n/a destination=n/a object=/org/freedesktop/systemd1/unit/nexus_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=1805 reply_cookie=0 error
Feb 28 10:25:30 nexus01. systemd[1]: Got message type=signal sender=org.freedesktop.DBus destination=n/a object=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameOwnerChanged cookie=179 reply_cookie=0 error=n/a
Feb 28 10:25:30 nexus01. systemd[1]: Got disconnect on private connection.
Feb 28 10:25:30 nexus01. systemd[1]: Received SIGCHLD from PID 12827 (java).
Feb 28 10:25:30 nexus01. systemd[1]: Child 12827 (java) died (code=exited, status=255/n/a)
Feb 28 10:25:30 nexus01. systemd[1]: Child 12827 belongs to nexus.service
Feb 28 10:25:30 nexus01. systemd[1]: nexus.service: main process exited, code=exited, status=255/n/a
Feb 28 10:25:30 nexus01. systemd[1]: About to execute: /opt/nexus/bin/nexus stop
Feb 28 10:25:30 nexus01. systemd[1]: Forked /opt/nexus/bin/nexus as 12848
Feb 28 10:25:30 nexus01. systemd[1]: nexus.service changed running -> stop