Error creating manufacturers - netbox v3.7.1

[howardtopher]

My guess here is that this script doesn't support netbox v3. But this is the error I get when trying to run the script to do an initial import on a new system. There are currently 0 manufacturers existing on this system yet.

Error creating manufacturers
Error during manufacturer creation. - {"error": "duplicate key value violates unique constraint \"dcim_manufacturer_slug_key\"\nDETAIL:  Key (slug)=(watchguard) already exists.", "exception": "IntegrityError", "netbox_version": "3.7.1", "python_version": "3.9.18"}

I tried changing requirements.txt to bump pynetbox up to 7.3.3 and I still get the same error.

[cbka]

+1

[howardtopher]

I did determine that the import works if you specify a vendor to install. I only encounter the error when running the script to install everything at once.

[Konraden]

The device library itself has "Watchguard" and "WatchGuard" directories containing devices. I'm going guess it breaks when it tires to create a slug the second time. The index is likely case insensitive but the script isn't.

[gurubert]

netbox-community/devicetype-library#1978 fixes this issue.

[netravnen]

Used the master branch of the repository here for the importer. Tried limiting the vendors to different ones. Without any real success.

# /opt/Device-Type-Library-Import/venv/bin/python3 /opt/Device-Type-Library-Import/nb-dt-import.py --vendors Generic 
Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Traceback (most recent call last):
  File "/opt/Device-Type-Library-Import/nb-dt-import.py", line 54, in <module>
    main()
  File "/opt/Device-Type-Library-Import/nb-dt-import.py", line 17, in main
    netbox = NetBox(settings)
             ^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 29, in __init__
    self.existing_manufacturers = self.get_manufacturers()
                                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in get_manufacturers
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in <dictcomp>
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/response.py", line 127, in __next__
    next(self.response), self.endpoint.api, self.endpoint
    ^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 280, in get
    req = self._make_call(add_params=add_params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 247, in _make_call
    raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error but more specific details were not returned in json. Check the NetBox Logs or investigate this exception's error attribute.

# ./nb-dt-import.py --vendors apc
Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Traceback (most recent call last):
  File "/opt/Device-Type-Library-Import/./nb-dt-import.py", line 54, in <module>
    main()
  File "/opt/Device-Type-Library-Import/./nb-dt-import.py", line 17, in main
    netbox = NetBox(settings)
             ^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 29, in __init__
    self.existing_manufacturers = self.get_manufacturers()
                                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in get_manufacturers
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in <dictcomp>
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/response.py", line 127, in __next__
    next(self.response), self.endpoint.api, self.endpoint
    ^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 280, in get
    req = self._make_call(add_params=add_params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 247, in _make_call
    raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error but more specific details were not returned in json. Check the NetBox Logs or investigate this exception's error attribute.

[danner26]

v3 is supported, but a newer version must have broke this functionality. We will look into it ASAP, or anyone is welcome to submit a PR if the issue is found

[gurubert]

The PR is already merged: netbox-community/devicetype-library#1978

[netravnen]

v3 is supported, but a newer version must have broke this functionality. We will look into it ASAP, or anyone is welcome to submit a PR if the issue is found

Now double-checking, after your response, and ensuring I have pulled the latest import and repo commits. It works... To a degree.

Now I am hitting different kind of errors during the device type import.

1. conversion between UTF8 and SQL_ASCII is not supported ⟵ Unsure how to got about this one. Installed the netbox in an LXC container according to the manual. Debian 15.6-0+deb12u1.
   1. Postgresql defaults to encoding SQL_ASCII if not specified otherwise.
   2. Followed this blog post.
   3. Stopped netbox.
   4. Took a database-dump of database netbox.
   5. Dropped and recreated template1 to new default encoding (UTF8).
   6. Created new database, netbox1.
   7. Restored database-dump to netbox1.
   8. Updated the configuration.py to point to database netbox1.
   9. Started netbox.
   10. Rerun device importer.
   11. No more conversion errors.
2. power_port":["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: PS1" ⟵ Seems easy to resolve. As the device type needs to be updated Stuck on what device is affected here
3. Error {"u_height":["Child device types must be 0U."]} creating device type: Mellanox SX6012T-2BFS ⟵ Add u_height value to yaml definition. Fix missing u_height for Mellanox/SX6012* devicetype-library#2014
4. Could not find Rear Port for Front Port: Port/18 - st - 1594 Error '[{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{"rear_port":["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: Port/18"]},{},{},{},{},{},{}]' creating Front Port ⟵ ... 🤔 💭 Incorrect rear_port ST-24-port-fiber-patch-panel-rear-splice devicetype-library#2015