Nexus3 Delete repository later nexus.log

Hi,
my blobs very big, almost 10T,so i Delete proxy repository , but nexus.log has a lot of mist
taskes,

  1. JVM error ‘OutOfMemoryError : Direct buffer memory’,so i restart nexus
    2.later Error deleting blob
    In the log the below entry appears when the tasks exits.
    Nexus OSS 3.21.1-01
    Any idea how to restore this?
    Thanks in advance!
    Balázs
2021-08-19 22:03:20,457+0800 WARN  [quartz-3-thread-13]  *SYSTEM org.sonatype.nexus.repository.storage.BucketDeleter - Error deleting blob default@B00A9B0F-6634DEEE-834C6E45-9A028D80-5E2DEDA0:b63dccc0-a739-497c-9d3f-e77d601ed2bc, skipping
org.sonatype.nexus.blobstore.api.BlobStoreException: BlobId: b63dccc0-a739-497c-9d3f-e77d601ed2bc, java.io.IOException: Invalid argument, Cause: Invalid argument
        at org.sonatype.nexus.blobstore.file.FileBlobStore.doDelete(FileBlobStore.java:483)
        at org.sonatype.nexus.blobstore.BlobStoreSupport.delete(BlobStoreSupport.java:146)
        at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
        at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
        at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
        at org.sonatype.nexus.repository.storage.BucketDeleter.deleteBlob(BucketDeleter.java:180)
        at org.sonatype.nexus.repository.storage.BucketDeleter.commitBatch(BucketDeleter.java:159)
        at org.sonatype.nexus.repository.storage.BucketDeleter.lambda$0(BucketDeleter.java:93)
        at org.sonatype.nexus.orient.transaction.OrientOperations.lambda$2(OrientOperations.java:62)
        at org.sonatype.nexus.transaction.OperationPoint.lambda$0(OperationPoint.java:53)
        at org.sonatype.nexus.transaction.OperationPoint.proceed(OperationPoint.java:64)
        at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:57)
        at org.sonatype.nexus.transaction.Operations.proceedWithTransaction(Operations.java:232)
        at org.sonatype.nexus.transaction.Operations.transactional(Operations.java:223)
        at org.sonatype.nexus.transaction.Operations.run(Operations.java:175)
        at org.sonatype.nexus.orient.transaction.OrientOperations.run(OrientOperations.java:62)
        at org.sonatype.nexus.repository.storage.BucketDeleter.deleteBucket(BucketDeleter.java:81)
        at org.sonatype.nexus.repository.storage.internal.StorageFacetManagerImpl.deleteBucket(StorageFacetManagerImpl.java:127)
        at org.sonatype.nexus.repository.storage.internal.StorageFacetManagerImpl.lambda$0(StorageFacetManagerImpl.java:100)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.LongPipeline.reduce(LongPipeline.java:438)
        at java.util.stream.LongPipeline.sum(LongPipeline.java:396)
        at java.util.stream.ReferencePipeline.count(ReferencePipeline.java:526)
        at org.sonatype.nexus.repository.storage.internal.StorageFacetManagerImpl.performDeletions(StorageFacetManagerImpl.java:108)
        at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
        at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
        at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:53)
        at org.sonatype.nexus.repository.storage.internal.StorageFacetCleanupTask.execute(StorageFacetCleanupTask.java:52)
        at org.sonatype.nexus.repository.storage.internal.StorageFacetCleanupTask.execute(StorageFacetCleanupTask.java:1)
        at org.sonatype.nexus.scheduling.TaskSupport.call(TaskSupport.java:100)
        at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.doExecute(QuartzTaskJob.java:143)
        at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.execute(QuartzTaskJob.java:106)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
        at org.sonatype.nexus.quartz.internal.QuartzThreadPool.lambda$0(QuartzThreadPool.java:143)
        at org.sonatype.nexus.thread.internal.MDCAwareRunnable.run(MDCAwareRunnable.java:40)
        at org.apache.shiro.subject.support.SubjectRunnable.doRun(SubjectRunnable.java:120)
        at org.apache.shiro.subject.support.SubjectRunnable.run(SubjectRunnable.java:108)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Invalid argument
        at java.io.RandomAccessFile.setLength(Native Method)
        at com.squareup.tape.QueueFile.setLength(QueueFile.java:406)
        at com.squareup.tape.QueueFile.expandIfNecessary(QueueFile.java:375)
        at com.squareup.tape.QueueFile.add(QueueFile.java:305)
        at com.squareup.tape.QueueFile.add(QueueFile.java:287)
        at org.sonatype.nexus.blobstore.file.FileBlobStore.doDelete(FileBlobStore.java:477)
        ... 46 common frames omitted

before restart nexus,i modify nexus.vmoptions MaxDirectMemorySize

My guess is something got badly corrupted due to the jvm running out of memory. If you have a support contact you should definitely reach out to them for help.

You could try running the blobstore repair tasks, but if it’s still trying to delete things I think you just need to wait for the task to finish. That error indicates that it’s just skipping things that are incorrect/corrupted so I’d expect it to keep going until it’s deleted everything that isn’t corrupted.

1 Like