Previously our nexus repo bucket was stable at 2TB, early July we upgraded the version of Nexus to 3.32.0-03 from 3.17.0-01. The date of this coincides with the increase in storage. [image] I’ve used the S3 Storagelens to get a breakdown of the objects, we thought it might be failed multipart upl…

The main suggestions I would have: Ensure you have not specified -1 in your blob store configuration for Expiration Days - Configuring Blob Stores Ensure you have configured the tasks for Docker Delete Unused, and Docker Delete incomplete upload Generally you should try to move to a significantly…

[image] Matthew Piggott: Docker Delete incomplete upload Thanks so much for responding Matthew, I’ve been looking into the deletion of unused docker images, there’s no cleanup policy but there is a cleanup task which executes daily to Cleanup repositories using their associated policies, afte…

Docker Delete incomplete upload This job is responsible for removing partial uploads (the Docker client uploads layers in chunks over multiple requests) when uploads are not completed then you may have partial uploads left behind.

https://help.sonatype.com/en/cleanup-policies.html#docker-cleanup-strategies The other task referenced Docker Delete Unused identifies layers that aren’t referenced by tagged manifests and removes them.

[image] Matthew Piggott: Docker Delete incomplete upload I’m not sure if this shows up on AWS but there is a filter for incomplete multipart uploads and this only represented about 20 GB, so not that significant in terms of the 28 TB we’re looking at.

I haven’t been given the green light to delete unused docker images yet. I was trying to look into how frequently Nexus communicates with AWS, perhaps limiting that. I’ve noticed that in the AWS bucket when we show the versions, there are thousands of versions for the same object, some being upload…

[image] Nir Frankel: I’m not sure if this shows up on AWS but there is a filter for incomplete multipart uploads and this only represented about 20 GB, so not that significant in terms of the 28 TB we’re looking at. This is something internal to Nexus.

[image] Nir Frankel: I was trying to look into how frequently Nexus communicates with AWS, perhaps limiting that. I’ve noticed that in the AWS bucket when we show the versions, there are thousands of versions for the same object, some being uploaded a few seconds a part. Versioning has always be…

I just used the orientDB console and calculated the individual sizes of each repo. Is it normal to see discrepancies between the total size of the blob store and the sum of the repos calculated using the orientDB console? Or are the two not related? For example we have a Docker repo: 109376846122…

S3 Storage increased 10x over 8 months

Sonatype Nexus Repository

mpiggott (Matthew Piggott) March 12, 2024, 4:58pm 11

In some circumstances the blob store size can get out of sync.

Topic		Replies	Views
Slow S3 blobstore docker image ingest rate Sonatype Nexus Repository help	1	1887	June 29, 2018
Kubernetes Nexus from the persistence volume to S3 Sonatype Nexus Repository	0	164	August 14, 2024
Blob storage and disk usage don't match Sonatype Nexus Repository	2	1756	August 31, 2022
Nexus S3 blob store doesn't work Sonatype Nexus Repository	2	6151	February 19, 2024
Cannot upload to s3 blobstore Sonatype Nexus Repository	2	911	July 31, 2023

S3 Storage increased 10x over 8 months

Related topics