Hey guys, I have a question about an automated python file to perform execution to download dependencies in nexus and send it back to the physical computer, how do i do that?
This is about the bare minimum to get you started. You’ll need error handling and other things, and I’m sure there are more elegant/pythonic ways of writing this code (I’m not really a developer, I just like to automate parts of my job). Hopefully it’s helpful.
#!/usr/bin/env python3 import json import urllib.request # change baseurl and downloadRepository to your NXRM server and pypi repo baseurl = 'http://nxrm.myorg.com:8080/service/rest/v1/' downloadRepository = 'pypi-proxy' url = baseurl + 'components?repository=' + downloadRepository go = 1 while go response = urllib.request.urlopen(url) data = json.load(response) for item in data["items"]: for asset in item["assets"]: fileurl = asset["downloadUrl"] filename = downloadPath + fileurl.split('/')[-1] # '\' for Windows outfile = open(filename, "w") outfile.write(str(urllib.request.urlopen(fileurl).read())) outfile.close() if data["continuationToken"] is None: break else: # construct pagination url and loop url = baseurl + 'components?continuationToken=' + data["continuationToken"] + '&repository=' + downloadRepository
Sorry for the late reply, could you explain what does these lines of code do? For loop of asset , items and the last line. Appreciate the help!
The nexus server returns the data from the REST query in JSON format as a dictionary. You can see it if you do
after this line
data = json.load(response)
items is the key with all of the python packages as values. Inside items, you’ll see an
assets key. One of the values in assets is the download URL for the package. This python program downloads the packages using those URLs.
Some repositories may be large, so the Nexus repository manager server API only returns 10 items for each REST call, and it also returns a pagination token to let you know there are more items. You can construct another REST call and include the pagination token to see the next page of items. If
data["continuationToken"]=none, then there are no more items in the repository.
I have completed coding my scripts from internal to server to list down the dependencies list needed for downloading, right now I needed to establish the API towards nexus repository I would require your assistance from you with this one, do you have discord maybe we could chat from there?
That would be very helpful
I’m sorry, I don’t have discord. It sounds like you have a list of dependencies you want to download from the repository. If that’s true, you could do the following:
for each of the
item in data["items"] in the example loop I gave, search through your list and compare
data to components in your list, then go into the
for asset in item["assets"] to get the URL to download the component/package. That might look like this inserted into the code above…
while go response = urllib.request.urlopen(url) data = json.load(response) for item in data["items"]: <insert code here to compare your dependency name to item["name"] and your dependency version to item["version"]> <if item matches something in your dependencies, then do the following> for asset in item["assets"]: fileurl = asset["downloadUrl"] filename = downloadPath + fileurl.split('/')[-1] # '\' for Windows outfile = open(filename, "w") outfile.write(str(urllib.request.urlopen(fileurl).read()))
I have the following code that brings in the dependencies list,
loggin.debug("dependencies list %s", str(dList))
<insert code here to compare your dependency name to item[“name”] and your dependency version to item[“version”]>
how should I compare it with this, I’m quite new with this, pardon me.
Appreciate the help!
Hello Austin, I have experienced errors while executing,
File “/usr/lib64/python3.6/json/init.py”, line 299, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File “/usr/lib64/python3.6/json/init.py”, line 354, in loads
File “/usr/lib64/python3.6/json/decoder.py”, line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “/usr/lib64/python3.6/json/decoder.py”, line 357, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
any clue on what has happened?
I saw your other post about the URL. It would be good for you to review the API documentation that can be accessed from the web GUI after logging into your Nexus repository manager server as an administrator. I’ve attached a screenshot of mine. The Base URL is given at the top, and the additional pieces of the URL are given in the API doc.
You have to construct a URL for your script. For example, something like
will list all of your blob stores. To list all of the components in a particular repository, you should construct a URL like
If you compare the URLs above to the API docs in the screenshot, do you see how the URL you construct is made up of three parts: your server’s IP address, the Base URL, and the other part of the URL specific to what you want?
What I call “base URL” in my script doesn’t match the “base URL” in the API docs because I include my server’s IP address in my base URL. But I think this should be enough information for you to be able to construct the URLs for your urllib requests.
I also think you may experience problems using 127.0.0.1 in your URL. That IP address means localhost, so your script will only work if you’re running it on the Nexus repository manager server. You should replace 127.0.0.1 with the IP address assigned to your Nexus server. It might be something like 192.168.x.x
How to hide credentials while importing python packages from Nexus to EMR notebook.
Need to hide credentials**