We are using Gradle running in Bitbucket Cloud to publish built artifacts to OSS Nexus hosted in our own colo. We frequently see the following error during the gradle publish task:
Task :publishMavenPublicationToMavenRepository
Task :publishMavenPublicationToMavenRepository FAILED
FAILURE: Build failed with an exception.
- What went wrong:
Execution failed for task ‘:publishMavenPublicationToMavenRepository’.
Failed to publish publication ‘maven’ to repository ‘maven’
Could not GET ‘https://shrek.raveu.com:48082/nexus/content/repositories/releases//maven-metadata.xml’.
Read timed out
Nexus OSS v3.25.0
Running behind Apache reverse proxy
In the Nexus log at the same time:
2020-12-12 23:31:00,442-0500 WARN [qtp1348605207-82504] user org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker - Error processing metadata for path: /maven-metadata.xml
org.sonatype.nexus.repository.maven.internal.hosted.metadata.MetadataException: Error processing metadata for path: /maven-metadata.xml
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker.processMetadata(OrientMetadataRebuilder.java:499)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker.rebuildMetadataInner(OrientMetadataRebuilder.java:572)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker.rebuildMetadata(OrientMetadataRebuilder.java:452)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder.rebuildInTransaction(OrientMetadataRebuilder.java:155)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.rebuildMetadata(MavenFacetImpl.java:257)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.doGet(MavenFacetImpl.java:232)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl$$EnhancerByGuice$$716c86b9.CGLIB$doGet$9()
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl$$EnhancerByGuice$$716c86b9$$FastClassByGuice$$d6ab573d.invoke()
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:76)
at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:57)
at org.sonatype.nexus.transaction.TransactionInterceptor.proceedWithTransaction(TransactionInterceptor.java:66)
at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:55)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:78)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:54)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl$$EnhancerByGuice$$716c86b9.doGet()
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.get(MavenFacetImpl.java:212)
at org.sonatype.nexus.repository.maven.internal.hosted.HostedHandler.doGet(HostedHandler.java:75)
at org.sonatype.nexus.repository.maven.internal.hosted.HostedHandler.handle(HostedHandler.java:61)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.storage.LastDownloadedHandler.handle(LastDownloadedHandler.java:59)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:46)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.maven.internal.orient.OrientVersionPolicyHandler.handle(OrientVersionPolicyHandler.java:56)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:42)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.routing.internal.RoutingRuleHandler.handle(RoutingRuleHandler.java:52)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:51)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:58)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:65)
at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
at org.sonatype.nexus.repository.group.GroupHandler.getAll(GroupHandler.java:183)
at org.sonatype.nexus.repository.group.GroupHandler.getAll(GroupHandler.java:151)
at org.sonatype.nexus.repository.maven.internal.group.MergingGroupHandler.doGet(MergingGroupHandler.java:91)
at org.sonatype.nexus.repository.group.GroupHandler.handle(GroupHandler.java:95)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:46)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:42)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.routing.internal.RoutingRuleHandler.handle(RoutingRuleHandler.java:52)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:51)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:58)
at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:65)
at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:212)
at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:174)
at org.sonatype.nexus.repository.httpbridge.internal.LegacyViewServlet.doService(LegacyViewServlet.java:73)
at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184)
at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:387)
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:80)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:116)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:112)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:79)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:101)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133)
at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:239)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:505)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.sonatype.nexus.repository.storage.MissingBlobException: Blob default@28160897-776AD6DD-38FDF2D0-EF608E20-D1988242:8655c5bd-61e2-4fa1-9e96-5449e180d269 exists in metadata, but is missing from the blobstore
at org.sonatype.nexus.repository.storage.StorageTxImpl.requireBlob(StorageTxImpl.java:965)
at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.sonatype.nexus.common.stateguard.SimpleMethodInvocation.proceed(SimpleMethodInvocation.java:53)
at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
at org.sonatype.nexus.common.stateguard.StateGuardAspect$1.invoke(StateGuardAspect.java:63)
at com.sun.proxy.$Proxy213.requireBlob(Unknown Source)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.doGet(MavenFacetImpl.java:226)
at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:49)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.get(MavenFacetImpl.java:212)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataUtils.read(OrientMetadataUtils.java:72)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataUpdater.lambda$1(OrientMetadataUpdater.java:127)
at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64)
at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:217)
at org.sonatype.nexus.transaction.Operations.call(Operations.java:166)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataUpdater.replace(OrientMetadataUpdater.java:123)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataUpdater.processMetadata(OrientMetadataUpdater.java:77)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker.processMetadata(OrientMetadataRebuilder.java:496)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker.rebuildMetadataInner(OrientMetadataRebuilder.java:572)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder$Worker.rebuildMetadata(OrientMetadataRebuilder.java:452)
at org.sonatype.nexus.repository.maven.internal.orient.OrientMetadataRebuilder.rebuildInTransaction(OrientMetadataRebuilder.java:155)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.rebuildMetadata(MavenFacetImpl.java:257)
at org.sonatype.nexus.repository.maven.internal.orient.MavenFacetImpl.doGet(MavenFacetImpl.java:232)
at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:57)
at org.sonatype.nexus.transaction.TransactionInterceptor.proceedWithTransaction(TransactionInterceptor.java:66)
at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:55)
… 135 common frames omitted
2020-12-12 23:31:00,443-0500 WARN [qtp1348605207-82504] user org.sonatype.nexus.blobstore.file.FileBlobStore - Attempt to access soft-deleted blob 8655c5bd-61e2-4fa1-9e96-5449e180d269 attributes: /u01/opt/CI/sonatype-work/nexus3/blobs/default/content/vol-03/chap-24/8655c5bd-61e2-4fa1-9e96-5449e180d269.properties {deletedDateTime=1607833840795, deleted=true, @BlobStore.created-by=raveBuild, creationTime=1607747419534, @BlobStore.created-by-ip=34.232.119.183, @BlobStore.content-type=application/xml, sha1=80d343d526cfa658b2daecc376efe1f49b982028, @BlobStore.blob-name=/maven-metadata.xml, deletedReason=Updating asset AttachedEntityId{asset->#155:48872}, @Bucket.repo-name=releases, size=58665}
Any advice on how to resolve would be greatly appreciated.