Blobstore tmp file cleanup

Sonatype Nexus Repository
1

I’m investigating high disk utilization for our Nexus Repository Manager (3.28.1) implementation (mostly nuget/helm/docker repos). Querying orient for repository sizes shows a massive discrepancy between reported total repo size and disk utilization.

While trying to figure out what’s using the disk, I came across the blobs/blobstore/content/tmp directory. In that dir, I found tons of tmp files (hundreds of GB) going back a long time.

I’m having a hard time finding documentation on what’s stored in that content/tmp directory and how to properly clean it up. I have scheduled cleanup/compaction tasks that seem to be working properly, but aren’t touching this area.

Can anyone provide insight regarding what’s stored in this tmp area and how it should properly be cleaned? Or maybe what cleaning task isn’t working as intended? Is it safe to simply delete older files in that dir?

Thanks!

2

Update: I removed this files from a Nexus test environment and our blobstore appears unaffected. However, the “total size” of our blobstore, as reported by Nexus, reflects the size before removing the tmp files. Restarting/running cleanup tasks/etc doesn’t seem to bring that back to reality. This makes me think that I probably shouldn’t have simply removed those files.

If anyone has a suggestion of a better way to clean up tmp files, I’m all ears. I’d be happy to restore my test environment and experiment.

3

If you haven’t, configure the Compact Blobstore task.

4

I have configured that task. It doesn’t clean these files or reconcile the blobstore size.

5

Anyone?

Is there another area I should post this where it might get more traction? Someone must know what’s stored in the content/tmp directory.

6

I have this same issue. The “tmp” directory eats more space than all the rest of the system. Is there a safe way to clean it up and keep Nexus 3 healthy and working? I tried restarting it, but without success, so it looks like more “permanent” than really “tmp”. I don’t want to manually remove files there as it might affect the system somehow.

7

Nexus Repo uses a quasi-transactional implementation for file blob stores to ensure partial content cannot ever enter blob storage. Files uploaded are first stored to the blob store’s tmp directory, then after they have been full y uploaded and validated they are renamed into their final location in the file blob store.

Here’s some further clarification:

  1. If Nexus Repo is not running, you can safely remove any files in the tmp directory
  2. Files in tmp should not stay around a long time
  3. There is an exception to point 2, due to the way docker works you periodically need to run a “docker - remove incomplete uploads” task**

Rich

** https://help.sonatype.com/display/NXRM3/Tasks#Tasks-Docker-Deleteincompleteuploads

8

Thanks Rich! Implementing this cleanup task now. I’ll update this thread with the results.

9

Ran the “Docker - Delete incomplete uploads” task and still have tons of old files in content/tmp. This is not the solution to my particular issue.

10

Hi,
has there been any solution to this problem?
I am facing the same issue using Nexus 3.32.1 - “tmp” grows and grows , so that there is no other chance than to delete stuff in there manually.
I actually have to do this nearly every day - …at runtime, cause I cannot shutdown Nexus every day ( or night ).
As far as I can say - this is not only valid for “Docker” but also when using “APT” repositories. No matter how often cleanup tasks are running - the “tmp” dir still increases.
Also tested a fresh setup with Nexus 3.38.0 ( running in docker ) and only using “APT” repos - still the same. 500 MB debian packages uploaded , produces 1.7 GB data in “tmp”.
Any help on that?

11

It may be worth opening a bug on our JIRA instance in the hopes that we may prioritize the issue. Please describe the exact steps you followed and as much detail about the environment including the filesystem used so that we can easily duplicate the problem. issues.sonatype.org - please use the “Dev - Nexus Repo (NEXUS)” project.