Restoring Component db lost all indexes, Nexus oss 3.22.0

Sonatype Nexus Repository
1

Restoring component db cause the lost of all indexes,

bash-4.4$ java -jar /opt/sonatype/nexus/lib/support/nexus-orient-console.jar

OrientDB console v.2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x) https://www.orientdb.com
Type ‘help’ to display all the supported commands.
orientdb> sh: tput: command not found
orientdb> create database plocal:/nexus-data/db/component

Creating database [plocal:/nexus-data/db/component] using the storage type [plocal]…
Database created successfully.

Current database is: plocal:/nexus-data/db/component
orientdb {db=component}> import database component-export.json.gz

Importing database database component-export.json.gz…
Started import of database ‘plocal:/nexus-data/db/component’ from component-export.json.gz…
Non merge mode (-merge=false): removing all default non security classes

  • Class E was removed.
  • Class V was removed.
  • Class ORestricted was removed.
  • Class OTriggered was removed.
  • Class OSchedule was removed.
  • Class OSequence was removed.
  • Class OFunction was removed.
    Removed 7 classes.
    Importing database info…OK
    Importing clusters…
  • Creating cluster ‘internal’…OK, assigned id=0
  • Creating cluster ‘default’…OK, assigned id=3
  • Creating cluster ‘orole’…OK, assigned id=4
  • Creating cluster ‘ouser’…OK, assigned id=5
  • Creating cluster ‘ofunction’…OK, assigned id=6
  • Creating cluster ‘osequence’…OK, assigned id=7
  • Creating cluster ‘oschedule’…OK, assigned id=8
  • Creating cluster ‘v’…OK, assigned id=9
  • Creating cluster ‘v_1’…OK, assigned id=10
  • Creating cluster ‘e’…OK, assigned id=11
  • Creating cluster ‘e_1’…OK, assigned id=12
  • Creating cluster ‘statushealthcheck’…OK, assigned id=13
  • Creating cluster ‘statushealthcheck_1’…OK, assigned id=14
  • Creating cluster ‘bucket’…OK, assigned id=15
  • Creating cluster ‘bucket_1’…OK, assigned id=16
  • Creating cluster ‘component’…OK, assigned id=17
  • Creating cluster ‘component_1’…OK, assigned id=18
  • Creating cluster ‘asset’…OK, assigned id=19
  • Creating cluster ‘asset_1’…OK, assigned id=20
  • Creating cluster ‘statushealthcheck_2’…OK, assigned id=21
  • Creating cluster ‘statushealthcheck_3’…OK, assigned id=22
  • Creating cluster ‘docker_foreign_layers’…OK, assigned id=23
  • Creating cluster ‘docker_foreign_layers_1’…OK, assigned id=24
  • Creating cluster ‘docker_foreign_layers_2’…OK, assigned id=25
  • Creating cluster ‘docker_foreign_layers_3’…OK, assigned id=26
  • Creating cluster ‘browse_node’…OK, assigned id=27
  • Creating cluster ‘browse_node_1’…OK, assigned id=28
  • Creating cluster ‘browse_node_2’…OK, assigned id=29
  • Creating cluster ‘browse_node_3’…OK, assigned id=30
    Rebuilding indexes of truncated clusters …
  • Cluster content was updated: rebuilding index ‘ORole.name’… Index ORole.name was successfully rebuilt.
  • Cluster content was updated: rebuilding index ‘OUser.name’… Index OUser.name was successfully rebuilt.
    Done 2 indexes were rebuilt.
    Done. Imported 29 clusters
    Importing database schema…OK (16 classes)

Importing records…

  • Imported 31,900 records into clusters: [bucket, bucket_1, component, component_1, internal, orole, ouser]. Total records imported so far: 31,900 (6,380.00/sec)
  • Imported 26,903 records into clusters: [asset, asset_1, component, component_1]. Total records imported so far: 58,803 (5,380.60/sec)
  • Imported 26,669 records into clusters: [asset, asset_1]. Total records imported so far: 85,472 (5,333.80/sec)
  • Imported 26,567 records into clusters: [asset, asset_1]. Total records imported so far: 112,039 (5,313.40/sec)
  • Imported 26,233 records into clusters: [asset, asset_1]. Total records imported so far: 138,272 (5,246.60/sec)
  • Imported 26,265 records into clusters: [asset, asset_1]. Total records imported so far: 164,537 (5,253.00/sec)
  • Imported 24,113 records into clusters: [asset, asset_1]. Total records imported so far: 188,650 (4,822.60/sec)
  • Imported 25,163 records into clusters: [asset, asset_1]. Total records imported so far: 213,813 (5,032.60/sec)
  • Imported 26,377 records into clusters: [asset, asset_1]. Total records imported so far: 240,190 (5,275.40/sec)
  • Imported 26,115 records into clusters: [asset, asset_1]. Total records imported so far: 266,305 (5,223.00/sec)
  • Imported 43,153 records into clusters: [asset, asset_1, browse_node, browse_node_1, browse_node_2, browse_node_3]. Total records imported so far: 309,458 (8,630.60/sec)
  • Imported 49,068 records into clusters: [browse_node, browse_node_1, browse_node_2, browse_node_3]. Total records imported so far: 358,526 (9,813.60/sec)
  • Imported 41,523 records into clusters: [browse_node, browse_node_1, browse_node_2, browse_node_3]. Total records imported so far: 400,049 (8,304.60/sec)
  • Imported 47,047 records into clusters: [browse_node, browse_node_1, browse_node_2, browse_node_3]. Total records imported so far: 447,096 (9,409.40/sec)
  • Imported 46,213 records into clusters: [browse_node, browse_node_1, browse_node_2, browse_node_3]. Total records imported so far: 493,309 (9,242.60/sec)Reading of set of RIDs of records which were detected as broken during database export
    1 were detected as broken during database export, links on those records will be removed from result database

Started migration of links (-migrateLinks=true). Links are going to be updated according to new RIDs:

  • Cluster statushealthcheck… Completed migration of 0 records in current cluster
  • Cluster default… Completed migration of 0 records in current cluster
  • Cluster oschedule… Completed migration of 0 records in current cluster
  • Cluster component_1… Completed migration of 16,907 records in current cluster
  • Cluster e_1… Completed migration of 0 records in current cluster
  • Cluster docker_foreign_layers_2… Completed migration of 0 records in current cluster
  • Cluster browse_node… Completed migration of 63,429 records in current cluster
  • Cluster docker_foreign_layers_1… Completed migration of 0 records in current cluster
  • Cluster statushealthcheck_3… Completed migration of 0 records in current cluster
  • Cluster browse_node_2…
    — Migrated 25,771 of 63,428 records (21,221.40/sec)
    — Completed migration of 63,428 records in current cluster
  • Cluster browse_node_3… Completed migration of 63,428 records in current cluster
  • Cluster statushealthcheck_1… Completed migration of 0 records in current cluster
  • Cluster asset_1…
    — Migrated 6,210 of 119,939 records (21,459.00/sec)
    — Migrated 61,606 of 119,939 records (11,079.20/sec)
    — Migrated 117,235 of 119,939 records (11,125.80/sec)
    — Completed migration of 119,939 records in current cluster
  • Cluster statushealthcheck_2… Completed migration of 0 records in current cluster
  • Cluster browse_node_1… Completed migration of 63,428 records in current cluster
  • Cluster e… Completed migration of 0 records in current cluster
  • Cluster docker_foreign_layers_3… Completed migration of 0 records in current cluster
  • Cluster ouser… Completed migration of 3 records in current cluster
  • Cluster v_1… Completed migration of 0 records in current cluster
  • Cluster docker_foreign_layers… Completed migration of 0 records in current cluster
  • Cluster osequence… Completed migration of 0 records in current cluster
  • Cluster bucket… Completed migration of 17 records in current cluster
  • Cluster component… Completed migration of 16,907 records in current cluster
  • Cluster bucket_1… Completed migration of 16 records in current cluster
  • Cluster orole… Completed migration of 3 records in current cluster
  • Cluster v… Completed migration of 0 records in current cluster
  • Cluster ofunction… Completed migration of 0 records in current cluster
  • Cluster asset…
    — Migrated 13,826 of 119,939 records (19,380.80/sec)
    — Migrated 66,565 of 119,939 records (10,547.80/sec)
    — Completed migration of 119,939 records in current cluster
    Total links updated: 527,444

Done. Imported 527,445 records in 113.79 secs

Importing indexes …

  • Index ‘OUser.name’…OK
  • Index ‘component_bucket_group_name_version_idx’…OK
  • Index ‘asset_bucket_component_name_idx’…$ANSI{green {db=component}} Exception during index ‘asset_bucket_component_name_idx’ creation
    com.orientechnologies.orient.core.storage.ORecordDuplicatedException: Cannot index record #19:43334: found duplicated key ‘OCompositeKey{keys=[#16:2, #17:11446, com/]}’ in index ‘asset_bucket_component_name_idx’ previously assigned to the record #19:36509
    DB name=“component” INDEX=asset_bucket_component_name_idx RID=#19:36509
    at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:47)
    at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:37)
    at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.put(OSBTree.java:855)
    at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.validatedPut(OSBTree.java:261)
    at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.validatedPut(OSBTreeIndexEngine.java:169)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doValidatedPutIndexValue(OAbstractPaginatedStorage.java:2519)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.validatedPutIndexValue(OAbstractPaginatedStorage.java:2498)
    at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:82)
    at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:35)
    at com.orientechnologies.orient.core.index.OIndexAbstract.populateIndex(OIndexAbstract.java:940)
    at com.orientechnologies.orient.core.index.OIndexAbstract.indexCluster(OIndexAbstract.java:1040)
    at com.orientechnologies.orient.core.index.OIndexAbstract.fillIndex(OIndexAbstract.java:539)
    at com.orientechnologies.orient.core.index.OIndexAbstract.create(OIndexAbstract.java:250)
    at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:129)
    at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:40)
    at com.orientechnologies.orient.core.index.OIndexManagerShared.createIndex(OIndexManagerShared.java:171)
    at com.orientechnologies.orient.core.index.OIndexManagerProxy.createIndex(OIndexManagerProxy.java:87)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importIndexes(ODatabaseImport.java:1307)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importDatabase(ODatabaseImport.java:186)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.importDatabase(OConsoleDatabaseApp.java:2214)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:63)
    Error on database import happened just before line 0, column 369228430
    com.orientechnologies.orient.core.storage.ORecordDuplicatedException: Cannot index record #19:43334: found duplicated key ‘OCompositeKey{keys=[#16:2, #17:11446, com]}’ in index ‘asset_bucket_component_name_idx’ previously assigned to the record #19:36509
    DB name=“component” INDEX=asset_bucket_component_name_idx RID=#19:36509
    at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:47)
    at com.orientechnologies.orient.core.index.OIndexUnique$1.validate(OIndexUnique.java:37)
    at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.put(OSBTree.java:855)
    at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.validatedPut(OSBTree.java:261)
    at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.validatedPut(OSBTreeIndexEngine.java:169)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doValidatedPutIndexValue(OAbstractPaginatedStorage.java:2519)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.validatedPutIndexValue(OAbstractPaginatedStorage.java:2498)
    at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:82)
    at com.orientechnologies.orient.core.index.OIndexUnique.put(OIndexUnique.java:35)
    at com.orientechnologies.orient.core.index.OIndexAbstract.populateIndex(OIndexAbstract.java:940)
    at com.orientechnologies.orient.core.index.OIndexAbstract.indexCluster(OIndexAbstract.java:1040)
    at com.orientechnologies.orient.core.index.OIndexAbstract.fillIndex(OIndexAbstract.java:539)
    at com.orientechnologies.orient.core.index.OIndexAbstract.create(OIndexAbstract.java:250)
    at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:129)
    at com.orientechnologies.orient.core.index.OIndexOneValue.create(OIndexOneValue.java:40)
    at com.orientechnologies.orient.core.index.OIndexManagerShared.createIndex(OIndexManagerShared.java:171)
    at com.orientechnologies.orient.core.index.OIndexManagerProxy.createIndex(OIndexManagerProxy.java:87)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importIndexes(ODatabaseImport.java:1307)
    at com.orientechnologies.orient.core.db.tool.ODatabaseImport.importDatabase(ODatabaseImport.java:186)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.importDatabase(OConsoleDatabaseApp.java:2214)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:405)
    at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:260)
    at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:131)
    at com.orientechnologies.orient.console.OConsoleDatabaseApp.main(OConsoleDatabaseApp.java:145)
    at org.sonatype.nexus.orient.console.Main.main(Main.java:63)

Error: com.orientechnologies.orient.core.db.tool.ODatabaseExportException: Error on importing database ‘component’ from file: component-export.json.gz

Error: com.orientechnologies.orient.core.storage.ORecordDuplicatedException: Cannot index record #19:43334: found duplicated key ‘OCompositeKey{keys=[#16:2, #17:11446, com/]}’ in index ‘asset_bucket_component_name_idx’ previously assigned to the record #19:36509
DB name=“component” INDEX=asset_bucket_component_name_idx RID=#19:36509

orientdb {db=component}> repair database --fix-links

Repairing database…

  • Removing broken links…
    – Done! Fixed links: 1, modified documents: 1
    Repair database complete (0 errors)
    orientdb {db=component}> rebuild index *

Rebuilding index(es)…
Rebuilt index(es). Found 33820 link(s) in 1.676000 sec(s).

Index(es) rebuilt successfully
orientdb {db=component}> CHECK DATABASE

Checking storage.Check of storage is started…Check of storage completed in 235ms. without errors.
Checking indexes.
Checking cluster component for index component_bucket_group_name_version_idx
cluster component (17) || 100%
Checking cluster component_1 for index component_bucket_group_name_version_idx
cluster component_1 (18) || 100%
Total errors found on indexes: 0
orientdb {db=component}> disconnect

Disconnecting from the database [component]…OK
orientdb> exit

orientdb {db=component}> list indexes

INDEXES
±—±--------------------------------------±--------------------±------±--------±------±----------------------+
|# |NAME |TYPE |RECORDS|CLASS |COLLATE|FIELDS |
±—±--------------------------------------±--------------------±------±--------±------±----------------------+
|0 |___exportImportRIDMap |DICTIONARY_HASH_INDEX|527436 | |default| |
|1 |bucket_repository_name_idx |UNIQUE |33 |bucket |default|repository_name(STRING)|
|2 |component_bucket_group_name_version_idx|UNIQUE |33814 |component| | |
|3 |component_bucket_name_version_idx |NOTUNIQUE |33784 |component| | |
|4 |component_ci_name_ci_idx |NOTUNIQUE |4606 |component|ci |ci_name(STRING) |
|5 |dictionary |DICTIONARY |0 | |default| |
|6 |ORole.name |UNIQUE |3 |ORole |ci |name(STRING) |
|7 |OUser.name |UNIQUE |3 |OUser |ci |name(STRING) |
±—±--------------------------------------±--------------------±------±--------±------±----------------------+
| |TOTAL | |599679 | | | |
±—±--------------------------------------±--------------------±------±--------±------±----------------------+

Pl. help to create require indexes for Nexus oss 3.22.0 version.

2

Created Jira ticket to get details of list command to create remaining indexes on component db.

https://issues.sonatype.org/browse/NEXUS-25031

Thanks
Ravi

3

Component db as below list of index,

±—±-----------------------------------------------±----------------±------±--------------------±------±----------------------+
|# |NAME |TYPE |RECORDS|CLASS |COLLATE|FIELDS |
±—±-----------------------------------------------±----------------±------±--------------------±------±----------------------+
|0 |asset_bucket_component_name_idx |UNIQUE |0 |asset | | |
|1 |asset_bucket_name_idx |NOTUNIQUE |0 |asset | | |
|2 |asset_component_idx |NOTUNIQUE |0 |asset |default|component(LINK) |
|3 |asset_name_ci_idx |NOTUNIQUE |0 |asset |ci |name(STRING) |
|4 |browse_node_asset_id_idx |UNIQUE |0 |browse_node |default|asset_id(LINK) |
|5 |browse_node_component_id_idx |NOTUNIQUE |0 |browse_node |default|component_id(LINK) |
|6 |browse_node_repository_name_parent_path_name_idx|UNIQUE |0 |browse_node | | |
|7 |bucket_repository_name_idx |UNIQUE |7 |bucket |default|repository_name(STRING)|
|8 |component_bucket_group_name_version_idx |UNIQUE |0 |component | | |
|9 |component_bucket_name_version_idx |NOTUNIQUE |0 |component | | |
|10 |component_ci_name_ci_idx |NOTUNIQUE |0 |component |ci |ci_name(STRING) |
|11 |component_group_name_version_ci_idx |NOTUNIQUE |0 |component | | |
|12 |dictionary |DICTIONARY |0 | |default| |
|13 |docker_foreign_layers_digest_idx |UNIQUE |0 |docker_foreign_layers|default|digest(STRING) |
|14 |OFunction.name |UNIQUE_HASH_INDEX|0 |OFunction |default|name(STRING) |
|15 |ORole.name |UNIQUE |3 |ORole |ci |name(STRING) |
|16 |OUser.name |UNIQUE |3 |OUser |ci |name(STRING) |
|17 |statushealthcheck_node_id_idx |UNIQUE |1 |statushealthcheck |ci |node_id(STRING) |
±—±-----------------------------------------------±----------------±------±--------------------±------±----------------------+
| |TOTAL | |14 | | | |
±—±-----------------------------------------------±----------------±------±--------------------±------±----------------------+

4

Belows are OrientDB create index command to restore index in database

create index asset_bucket_component_name_idx on asset (bucket, component, name)UNIQUE
create index asset_bucket_name_idx on asset (bucket, name) NOTUNIQUE
create index asset_component_idx on asset (component collate default) NOTUNIQUE
create index asset_name_ci_idx on asset (name collate ci) NOTUNIQUE
create index browse_node_asset_id_idx on browse_node (asset_id collate default) UNIQUE
create index browse_node_component_id_idx on browse_node (component_id collate default) NOTUNIQUE
create index browse_node_repository_name_parent_path_name_idx on browse_node (repository_name, parent_path, name) UNIQUE
create index bucket_repository_name_idx on bucket (repository_name collate default) UNIQUE
create index component_bucket_group_name_version_idx on component (bucket, group, name, version) UNIQUE
create index component_bucket_name_version_idx on component (bucket, name, version) NOTUNIQUE
create index component_ci_name_ci_idx on component (ci_name collate ci) NOTUNIQUE
create index component_group_name_version_ci_idx on component (group, name,version) NOTUNIQUE
create index docker_foreign_layers_digest_idx on docker_foreign_layers (digest collate default) UNIQUE
create index OFunction.name on OFunction (name collate default) UNIQUE_HASH_INDEX
create index ORole.name on ORole (name collate ci) UNIQUE
create index OUser.name on OUser (name collate ci) UNIQUE
create index statushealthcheck_node_id_idx on statushealthcheck (node_id collate ci) UNIQUE