Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for socks connection to Git backend #17640

Merged
merged 13 commits into from
Oct 1, 2024
Merged

Conversation

arthanson
Copy link
Collaborator

@arthanson arthanson commented Sep 27, 2024

Fixes: #17639

Note: will also need to install the python_socks library along with the Dulwich library to use this:

pip install dulwich
pip install python_socks

To test locally, start a socks proxy in a docker container. Make a HTTP request and observe it using the proxy:

$ docker run -d --name socks5 -p 1080:1080 serjs/go-socks5-proxy
$ docker logs -f socks5

Specify the local proxy in Netbox configuration.py:

HTTP_PROXIES = {
   'http': 'socks5h://localhost:1080',
   'https': 'socks5h://localhost:1080',
}

Add a remote GIT datasource and you should see the traffic logs in the terminal. Setup a DataSource in Netbox, I was using the following in the form:

Name = GitSync
Type = Git
URL = https://github.com/bogdancordos/netbox_scripts.git

The URL just has a selection of scripts so you can see that it actually syncs the data source. Press the [Sync] button and make sure it actually syncs.

For SOCKs auth scenario it can be setup as follows:

docker run -d --name socks5 -p 1080:1080 -e PROXY_USER=aaa -e PROXY_PASSWORD=bbb  serjs/go-socks5-proxy

Then in configuration.py:

HTTP_PROXIES = {
    'http': 'socks5h://aaa:bbb@localhost:1080',
    'https': 'socks5h://aaa:bbb@localhost:1080',
}

netbox/core/data_backends.py Outdated Show resolved Hide resolved
docs/features/synchronized-data.md Outdated Show resolved Hide resolved
@arthanson arthanson changed the title DRAFT: Add support for socks connection to Git backend Add support for socks connection to Git backend Sep 30, 2024
@arthanson arthanson marked this pull request as ready for review September 30, 2024 16:56
Copy link
Member

@jeremystretch jeremystretch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested this both with and without authentication, and syncing worked as expected both times. Nice work @arthanson!

docs/features/synchronized-data.md Outdated Show resolved Hide resolved
netbox/core/data_backends.py Outdated Show resolved Hide resolved
netbox/utilities/socks.py Outdated Show resolved Hide resolved
netbox/utilities/socks.py Outdated Show resolved Hide resolved
netbox/utilities/socks.py Outdated Show resolved Hide resolved
netbox/utilities/socks.py Outdated Show resolved Hide resolved
arthanson and others added 4 commits September 30, 2024 10:54
Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
netbox/utilities/constants.py Outdated Show resolved Hide resolved
netbox/utilities/socks.py Outdated Show resolved Hide resolved
@jeremystretch jeremystretch merged commit 92d8aa5 into develop Oct 1, 2024
6 checks passed
@jeremystretch jeremystretch deleted the git-datasource branch October 1, 2024 16:11
bctiemann pushed a commit that referenced this pull request Oct 11, 2024
* Add support for socks connection to Git backend

* cleanup socks detection

* add documentation for installing python_socks

* dont need lower()

* cleanup

* refactor Socks to utilities

* fix imports

* fix missing comma

* Update docs/features/synchronized-data.md

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>

* review feedback

* Update docs/features/synchronized-data.md

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>

* review changes

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
jeremystretch added a commit to alehaa/netbox that referenced this pull request Oct 11, 2024
* Add support for socks connection to Git backend

* cleanup socks detection

* add documentation for installing python_socks

* dont need lower()

* cleanup

* refactor Socks to utilities

* fix imports

* fix missing comma

* Update docs/features/synchronized-data.md

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>

* review feedback

* Update docs/features/synchronized-data.md

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>

* review changes

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add SOCKS support to proxy settings for Git remote data source
2 participants