Nexus3 on docker giving java.net.UnknownHostException

Sonatype Nexus Repository
,
1

Hi all,

I have a weird situation in that I have a docker container running Nexus v3.50 and I am getting the java.net.UnknownException errors.

The above issues means that the nexus home page does not load with the default info. Also, I have a docker repository that is also not working.

2023-04-05 08:13:35,022+0000 WARN  [qtp1511057130-1553]  *UNKNOWN com.sonatype.nexus.plugins.outreach.internal.outreach.SonatypeOutreach - Could not download page bundle
java.net.UnknownHostException: links.sonatype.com: Name or service not known
        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:866)
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
        at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:815)
        at java.net.InetAddress.getAllByName0(InetAddress.java:1282)
        at java.net.InetAddress.getAllByName(InetAddress.java:1140)
        at java.net.InetAddress.getAllByName(InetAddress.java:1064)
        at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at com.sonatype.nexus.plugins.outreach.internal.outreach.OutreachConnector.get(OutreachConnector.java:136)
        at com.sonatype.nexus.plugins.outreach.internal.outreach.SonatypeOutreach.remote(SonatypeOutreach.java:280)
        at com.sonatype.nexus.plugins.outreach.internal.outreach.SonatypeOutreach.getOutreachBundle(SonatypeOutreach.java:164)
        at com.sonatype.nexus.plugins.outreach.internal.outreach.SonatypeOutreach.getPageBundle(SonatypeOutreach.java:153)
        at com.sonatype.nexus.plugins.outreach.Outreach$getPageBundle$1.call(Unknown Source)
.......


2023-04-05 08:35:28,952+0000 WARN  [quartz-10-thread-20]  *SYSTEM org.sonatype.nexus.quartz.internal.task.QuartzTaskJob - Task 547007b3-5d33-492c-b8e8-01a20954a358 : 'System - Repository Health Check: maven-central' [healthcheck] execution failure
java.net.UnknownHostException: rhc.sonatype.com: Name or service not known
        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:866)
        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
        at java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:815)
        at java.net.InetAddress.getAllByName0(InetAddress.java:1282)
        at java.net.InetAddress.getAllByName(InetAddress.java:1140)
        at java.net.InetAddress.getAllByName(InetAddress.java:1064)
        at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at com.sonatype.nexus.plugins.healthcheck.service.impl.HdsServiceImpl.execute(HdsServiceImpl.java:246)
        at com.sonatype.nexus.plugins.healthcheck.service.impl.HdsServiceImpl.doGet(HdsServiceImpl.java:194)
        at com.sonatype.nexus.plugins.healthcheck.service.impl.HdsServiceImpl.getNextRunDeltas(HdsServiceImpl.java:99)
        at com.sonatype.nexus.plugins.healthcheck.task.HealthCheckTask.doDeltaCheck(HealthCheckTask.java:243)
        at com.sonatype.nexus.plugins.healthcheck.task.HealthCheckTask.doRunSafe(HealthCheckTask.java:139)
        at com.sonatype.nexus.plugins.healthcheck.task.HealthCheckTask.execute(HealthCheckTask.java:102)
        at com.sonatype.nexus.plugins.healthcheck.task.HealthCheckTask.execute(HealthCheckTask.java:1)
        at org.sonatype.nexus.scheduling.TaskSupport.call(TaskSupport.java:100)
        at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.doExecute(QuartzTaskJob.java:143)
        at org.sonatype.nexus.quartz.internal.task.QuartzTaskJob.execute(QuartzTaskJob.java:106)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
........


2023-04-05 08:22:09,105+0000 WARN  [qtp1511057130-1564]  anonymous org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl - Exception java.net.UnknownHostException: index.docker.io: Name or service not known checking remote for update, proxy repo docker-hub failed to fetch v2/jenkins/jenkins/manifests/latest, content not in cache.
2023-04-05 08:22:09,204+0000 WARN  [qtp1511057130-1562]  anonymous org.sonatype.nexus.repository.docker.internal.orient.DockerProxyFacetImpl - Exception java.net.UnknownHostException: index.docker.io checking remote for update, proxy repo docker-hub failed to fetch v2/jenkins/jenkins/manifests/latest, content not in cache.

I have tested the hosts using curl and even wrote a small test java program both run fine on the container.

I am at a loss why this is happening all other containers are working fine.

Finally, I have spun up a new docker container - same problem.

All help will be greatly appreciated!

2

By way of an update I installed the tcpdump on the docker container and see that there is no outgoing traffic for nexus

However, when I run a standalone test java app on the same container I can see outgoing traffic.

Am I missing something really obvious?

3

So, it seems the last time this was working was in version 3.40.1. All versions after that have this problem

4

SOLVED!

The reason was that I switched the docker driver from the old devicemapper to overlay2. This screwed up the network layer. Had to backup the docker folder and re-create with overlay2 and them migrate the volumes across.