How to undo Asset Soft-Delete in NXRM3?

help

#1

As part of a test with NEXUS3 as nuget package repo, we ran into the problem of vanishing assets. For our users it is unclear if this is a system or a user error, and even though there’s a gut feeling, there is no proof that it wasn’t the system.

Apparently, restoring a soft-deleted asset in NEXUS 2 was as simple as moving it out of the .trash folder and back into the repo tree. What is the equivalent of this in NEXUS 3?


#2

Hi Edwin,

Sorry to hear you’re having some unexplained problems. It’d be helpful for me to understand exactly what you mean by “vanishing assets” - Does this mean that you can no longer see the asset in the browse UI / search screens or is it that the NuGet client cannot see it (i.e. it is missing from the NuGet feed for some reason)?

What version of NXRM3 are you running?

In NXRM3 the data is stored in two places; the binaries are stored on disk in the blob-store and then a reference to those binaries is stored in our internal database in the asset table. So there are some possibilities here:

  1. The blob has been marked as soft-deleted but the asset record is still available in the database - In this case you should still be able to see the asset in the browse UI but you will get a warning in the nexus logs when you try to fetch the binary. Each blob on disk has a corresponding .properties file which can contain a deleted=true flag. Removing this flag in these circumstances would restore the asset to you.
  2. The blob has been marked as soft-deleted and the asset record has also been deleted - This is more of a problem because at the moment you would have to remove the deleted flag (as above) but also manually rebuild the asset record. We have added a task to do this automatically for Npm, Yum and Maven but haven’t yet implemented it for NuGet.
  3. The blob is not soft-deleted and the asset record hasn’t been removed but the nupkg is not showing up in the NuGet feed - In this case I would file a bug ticket on our JIRA https://issues.sonatype.org with which packages have a problem and include a support zip.

In the first two possibilities you can get extra information by finding the blob .properties file for a removed asset and find the property “deletedReason”.

Joe


#3

Hi Joe,

I’m running NEXUS 3:
|version|3.10.0-04|
|edition|OSS|
|buildRevision|f7b56bf088e8e313a210acbccc8b0739f446b962|
|buildTimestamp|2018-04-03-1355-42621|

Multiple packages were uploaded to our nexus hosted Nuget repo. These packages were older versions of packages already in the repo. A couple of days later many of these packages were gone from the UI. I did find these packages in the filesystem with their properties file stating deleted=true and deletedReason=Deleting asset AttachedEntityId{asset->#19:26}

I can not find a log entry telling me who (or what) deleted these packages.
I am not able to restore these packages, aside from having them completely reuploaded.

I did try removing the deletion entries from the properties files and a reindex, but as you said this was futile. How would I rebuild the asset record?

Kind regards,
Edwin


#4

Edwin, when you say, “older versions of packages already in the repo,” are you saying that you republished older versions of components that had the exact same coordinates? Name, namespace and version?

If so, only one of those can exist at any one time in a given npm [edit: I mean NuGet, though npm has the same restriction] repository. It’s like overwriting a new file with an older copy; the previous (newer) version is deleted, because they share the same spot.

Is this what’s happening?


#5

What is happening is: 2. The blob has been marked as soft-deleted and the asset record has also been deleted

By older version, I mean for example:

  1. Foo-4.8.1.nupkg uploaded July 11
  2. Foo-3.5.2.nupkg uploaded July 13 (this one vanished)
  3. Foo-3.5.2.nupkg reuploaded July 24

Asset 1 is still there.
Asset 2 vanished form the UI and now has “deleted=true” in its .properties file.
Asset 3 is the package that was reuploaded after a new hire couldn’t get Foo-3.5.2.nupkg from the repository.

All systems keep the packages in their cache for efficiency, which is why the missing packages weren’t discovered sooner.
Another user might have deleted these “older version” assets unknowingly, but I can’t find this in the log, and I am unable to undo such deletion.
A limited amount of users had deletion rights, until we find a root cause this has been revoked.


#6

That’s clear, thanks—and it’s what I was concerned about.

There can be only one NuGet package called ‘Foo’ of version 3.5.2 in a single repository at any given time. Every time you publish something new to (“Foo”, “3.5.2”), you’re overwriting whatever’s alraedy there, just as if you were overwriting a file. You can’t have two files with the same path and file name on the same file system; you can’t have two packages with the same coordinate in the same repository. For NuGet, the “coordinate” is packageId and version.

Although many ecosystems allow it, we highly recommend against republishing binaries with reused coordinates. If at all possible, adjust your development practices to prevent this from happening.

When you visualize the whole pipeline from development, publication, to downstream caching (possibly in multiple business units, or in a tier of load-balanced read proxies), it can be impossible to know if consumers are referencing the “right one,” or (as happened to you) to tell if the wrong one is still lurking around.

Also, it means your historical record is lossy, since you’re not keeping all of the versions you may care about (for audit reasons, etc.)

In this specific case, you might want to hand-edit the July 13 package to have a unique version number. NuGet allows pre-release qualifiers, so you could have these two packages:

Foo-3.5.2.nupkg
Foo-3.5.2-weForgotToSignItOrWhatever.nupkg

Then you can use the UI Upload feature to upload the ‘old’ version to the repository, where it can live alongside the other one because it has a different coordinate.


#7

Actually, this is the desired situation:
image

We only reuploaded the package after it was already gone from the web UI and “deleted=true” (soft-deleted).


#8

So the question remains:

  1. How to find out how (read: by whom) the assets got deleted?
  2. How to restore the soft-deleted NuGet assets in NXRM3?