The Good:
We successfully setup an ALB in AWS routing traffic to a target group hosting our nexus server.
This allows traffic coming in for nuget, npm and pypi hosted and proxy repositories without any issues.
The Bad:
We attempted to setup a docker hub proxy and followed the documentation. Note that we have tried both http and https in the setup and both seem to timeout and error.
https:
traffic is getting fed down to the target group without issues and then it seems to be throwing target errors and 504 gateway timeouts.
http:
traffic seems to get fed to the target group appropriate then the command threw an error about http coming from https as the docker pull command attempted to use https. As a work around we added the server to the list of insecure registries and set it to be the mirror for docker.io and tried a pull that way.
At that point the behavior went back to the timeout but instead of throwing a 504 it fell back to docker.io and successfully pulled container image.
So it looks like from our logs the request hits the alb and then routes appropriately to the target group. The target group then attempts to send to nexus on the same port the connector for the proxy is set to and that’s where things seem to break down. It seems like the repository connector port isnt open like we would expect it to be. We didn’t have to do anything special to get nexus running or the other repositories in terms of ports so I am not really sure why this is causing such a headache.
Any advice on this? everything else seems to work appropriately
EDIT - as an update it looks like the nexus logs see the traffic coming in and throws a 404 looking for a pretty generic image. In this case ubuntu. So it seems like its not hitting the index properly or something. I thought perhaps this was a timing issue where the timeout for searching is longer than the timeout for the pull command from the dockerdaemon or the alb timeout (300 seconds). Which would potentially result in the 504s coming from the AWS side but the log on the proxy side showing the 404 error as it continued its search? Thats really just a guess. That being said that begs the question as to why it cant get out to the actual dockerhub index
I verified the SG the instance is in allows all outbound traffic. further verified by the other proxy repos pulling and caching packages correctly.
ALB listens on a port using https → perm redirects traffic to 443 and changes the path to be “/repository/dockerproxy” which then gets forwarded to the normal target group hosting nexus on the same http port it normally listens on. This does NOT make use of the repository connector
Update - after re-examining the documentation for option 2 above (Docker Repository Reverse Proxy Strategies – Sonatype Support) I realized the reverse proxy was not setting the path correctly which was part of the issue.
so now the path is set to “/repository/dockerproxy/#{path}”
So that path actually looks correct and if I run it through postman or a browser it pulls things and gives me back some json. But for some reason when I run it through docker pull it throws the error. Which makes me think docker is doing something with headers that postman or the browser is not doing
Eventually got solution 1 working
ALB / reverse proxy listens on port A using https
Traffic is forwarded to target group on port B using https
traffic is forwarded to the nexus instance on port B which is the http connector on the proxy repo group
All security groups were updated appropriately and the pulls started working at this point