Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor OpenSockets to provide cross OS compatibility #180

Merged
merged 26 commits into from
Sep 10, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6333b33
Remove connections vector from OpenSockets, use common OpenSockets im…
Aug 16, 2020
f09ad10
Replace termion backend with crossterm, which works on Windows as well.
Aug 16, 2020
c9e748f
Merge branch 'crossterm' into windows
Aug 17, 2020
56d47ea
More fixes for windows build.
Aug 17, 2020
8c07e77
Remove tui default-features (termion), update unit tests for crossterm.
Aug 17, 2020
d6bdcae
Merge remote-tracking branch 'origin/crossterm' into windows
Aug 17, 2020
9f0f0d9
Windows compilation fixes.
Sep 1, 2020
20f7c9c
Remove unused get_open_sockets implementations for linux and mac. Fix…
Sep 2, 2020
1772bdb
Add build.rs for windows to download and extract Packet.lib from npca…
Sep 2, 2020
f7d9a65
Merge remote-tracking branch 'upstream/main' into windows
Sep 2, 2020
743150f
Resolve Cargo.lock after merging main.
Sep 2, 2020
ae5d79d
Merge branch 'main' into windows
imsnif Sep 3, 2020
336fde4
fix(tests): adjust snapshots new location of the dns resolution
imsnif Sep 3, 2020
3fda7ef
style(clippy): clippy
imsnif Sep 3, 2020
db1966a
style(clippy): remove dead code
imsnif Sep 3, 2020
9acb8a3
style(clippy): use write_all in build.rs
Sep 5, 2020
77704d2
style(clippy): remove unused import added by Intellij
Sep 5, 2020
2c7edfb
style(review): use String instead of str
Sep 7, 2020
e40e88a
fix(build): run build.rs only once
Sep 7, 2020
9688d80
fix(regression): skip iface_is_up() filter only for Windows
Sep 7, 2020
785b8aa
fix(review): restore per os implementation of get_open_sockets()
Sep 9, 2020
db98c8a
fix(cargo): add missing os specific packages
Sep 9, 2020
5c02531
fix: conditional compilation of windows module
Sep 9, 2020
e199a79
fix: compilation errors
Sep 9, 2020
2e17c4b
fix: missing Protocol::from_str() implementation
Sep 9, 2020
1a35f1e
style(clippy): remove unused methods
imsnif Sep 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
568 changes: 496 additions & 72 deletions Cargo.lock

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ trust-dns-resolver = "0.18.1"
async-trait = "0.1.21"
unicode-width = "0.1.8"

[target.'cfg(target_os="windows")'.dependencies]
netstat2 = "0.9.0"
sysinfo = "0.15.1"

[target.'cfg(target_os="linux")'.dependencies]
procfs = "0.7.4"

Expand All @@ -43,3 +47,8 @@ pnet_base = "0.26.0"
cargo-insta = "0.11.0"
packet-builder = "0.5.0"
regex = "1"

[build-dependencies]
#[target.'cfg(target_os="windows")'.build-dependencies]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct, those crates should not be downloaded on unixes.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this is correct. They were not downloaded locally on my linux.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imsnif actually that's not the case: I had to uncomment this line to make sure that http_req doesn't get downloaded.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oi, I'm sorry about that! You are very right and I managed to miss this. :) I'm fixing this now and will push a patch. Thanks bringing this up (and catching it so quickly)!

http_req = "0.7.0"
zip = "0.5.6"
54 changes: 54 additions & 0 deletions build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
fn main() {
#[cfg(target_os = "windows")]
download_winpcap_sdk()
}

#[cfg(target_os = "windows")]
fn download_winpcap_sdk() {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this correctly, this happens every time we build bandwhich, right? Even if the file already exists. Not terrible, but might make development a little slow? What do you think about using the rerun-if-changed flag on the resulting Packet.lib file? That way if it got deleted we'll redownload it, but otherwise skip this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea, thanks. I thought that this flag was for C sources only, gotta check it out.

use http_req::request;
use std::env;
use std::fs::File;
use std::io::prelude::*;

println!("cargo:rerun-if-changed=build.rs");

let out_dir = env::var("OUT_DIR").unwrap();

let mut reader = Vec::new();
let _res = request::get(
"https://nmap.org/npcap/dist/npcap-sdk-1.05.zip",
&mut reader,
)
.unwrap();

let mut pcapzip = File::create(format!("{}{}", out_dir, "/npcap.zip")).unwrap();
pcapzip.write_all(reader.as_slice()).unwrap();
pcapzip.flush().unwrap();

pcapzip = File::open(format!("{}{}", out_dir, "/npcap.zip")).unwrap();

let lib_name = "Packet.lib";
#[cfg(target_arch = "x86_64")]
let lib_dir = "Lib/x64";
#[cfg(target_arch = "x86")]
let lib_dir = "Lib";

let lib_path = format!("{}/{}", lib_dir, lib_name);
let mut zip_archive = zip::ZipArchive::new(pcapzip).unwrap();
let mut pcaplib = match zip_archive.by_name(lib_path.as_str()) {
Ok(pcaplib) => pcaplib,
Err(err) => {
panic!(err);
}
};

let mut pcaplib_bytes = Vec::new();
pcaplib.read_to_end(&mut pcaplib_bytes).unwrap();

std::fs::create_dir_all(format!("{}/{}", out_dir, lib_dir)).unwrap();
let mut pcaplib_file = File::create(format!("{}/{}", out_dir, lib_path)).unwrap();
pcaplib_file.write_all(pcaplib_bytes.as_slice()).unwrap();
pcaplib_file.flush().unwrap();

println!("cargo:rustc-link-search=native={}/{}", out_dir, lib_dir);
}
17 changes: 5 additions & 12 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ mod tests;
use display::{elapsed_time, RawTerminalBackend, Ui};
use network::{
dns::{self, IpTable},
Connection, LocalSocket, Sniffer, Utilization,
LocalSocket, Sniffer, Utilization,
};
use os::OnSigWinch;

Expand Down Expand Up @@ -75,9 +75,6 @@ fn main() {
}

fn try_main() -> Result<(), failure::Error> {
#[cfg(target_os = "windows")]
compile_error!("Sorry, no implementations for Windows yet :( - PRs welcome!");

use os::get_input;
let opts = Opt::from_args();
let os_input = get_input(&opts.interface, !opts.no_resolve)?;
Expand All @@ -101,7 +98,6 @@ fn try_main() -> Result<(), failure::Error> {

pub struct OpenSockets {
sockets_to_procs: HashMap<LocalSocket, String>,
connections: Vec<Connection>,
}

pub struct OsInputOutput {
Expand Down Expand Up @@ -190,19 +186,16 @@ where
while running.load(Ordering::Acquire) {
let render_start_time = Instant::now();
let utilization = { network_utilization.lock().unwrap().clone_and_reset() };
let OpenSockets {
sockets_to_procs,
connections,
} = get_open_sockets();
let OpenSockets { sockets_to_procs } = get_open_sockets();
let mut ip_to_host = IpTable::new();
if let Some(dns_client) = dns_client.as_mut() {
ip_to_host = dns_client.cache();
let unresolved_ips = connections
.iter()
let unresolved_ips = utilization
.connections
.keys()
.filter(|conn| !ip_to_host.contains_key(&conn.remote_socket.ip))
.map(|conn| conn.remote_socket.ip)
.collect::<Vec<_>>();

dns_client.resolve(unresolved_ips);
}
{
Expand Down
39 changes: 20 additions & 19 deletions src/os/linux.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@ use ::std::collections::HashMap;

use ::procfs::process::FDTarget;

use crate::network::{Connection, Protocol};
use crate::network::{LocalSocket, Protocol};
use crate::OpenSockets;

pub(crate) fn get_open_sockets() -> OpenSockets {
let mut open_sockets = HashMap::new();
let mut connections = std::vec::Vec::new();
let mut inode_to_procname = HashMap::new();

if let Ok(all_procs) = procfs::process::all_processes() {
Expand All @@ -28,14 +27,15 @@ pub(crate) fn get_open_sockets() -> OpenSockets {
tcp.append(&mut tcp6);
}
for entry in tcp.into_iter() {
let local_port = entry.local_address.port();
let local_ip = entry.local_address.ip();
if let (connection, Some(procname)) = (
Connection::new(entry.remote_address, local_ip, local_port, Protocol::Tcp),
inode_to_procname.get(&entry.inode),
) {
open_sockets.insert(connection.local_socket, procname.clone());
connections.push(connection);
if let Some(procname) = inode_to_procname.get(&entry.inode) {
open_sockets.insert(
LocalSocket {
ip: entry.local_address.ip(),
port: entry.local_address.port(),
protocol: Protocol::Tcp,
},
procname.clone(),
);
};
}
}
Expand All @@ -45,19 +45,20 @@ pub(crate) fn get_open_sockets() -> OpenSockets {
udp.append(&mut udp6);
}
for entry in udp.into_iter() {
let local_port = entry.local_address.port();
let local_ip = entry.local_address.ip();
if let (connection, Some(procname)) = (
Connection::new(entry.remote_address, local_ip, local_port, Protocol::Udp),
inode_to_procname.get(&entry.inode),
) {
open_sockets.insert(connection.local_socket, procname.clone());
connections.push(connection);
if let Some(procname) = inode_to_procname.get(&entry.inode) {
open_sockets.insert(
LocalSocket {
ip: entry.local_address.ip(),
port: entry.local_address.port(),
protocol: Protocol::Udp,
},
procname.clone(),
);
};
}
}

OpenSockets {
sockets_to_procs: open_sockets,
connections,
}
}
24 changes: 9 additions & 15 deletions src/os/lsof.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
use ::std::collections::HashMap;

use crate::network::Connection;
use crate::network::LocalSocket;
use crate::OpenSockets;

use super::lsof_utils;
use std::net::SocketAddr;

#[derive(Debug)]
struct RawConnection {
Expand All @@ -17,26 +16,21 @@ struct RawConnection {

pub(crate) fn get_open_sockets() -> OpenSockets {
let mut open_sockets = HashMap::new();
let mut connections_vec = std::vec::Vec::new();

let connections = lsof_utils::get_connections();

for raw_connection in connections {
let protocol = raw_connection.get_protocol();
let remote_ip = raw_connection.get_remote_ip();
let local_ip = raw_connection.get_local_ip();
let remote_port = raw_connection.get_remote_port();
let local_port = raw_connection.get_local_port();

let socket_addr = SocketAddr::new(remote_ip, remote_port);
let connection = Connection::new(socket_addr, local_ip, local_port, protocol);

open_sockets.insert(connection.local_socket, raw_connection.process_name.clone());
connections_vec.push(connection);
open_sockets.insert(
LocalSocket {
ip: raw_connection.get_local_ip(),
port: raw_connection.get_local_port(),
protocol: raw_connection.get_protocol(),
},
raw_connection.process_name.clone(),
);
}

OpenSockets {
sockets_to_procs: open_sockets,
connections: connections_vec,
}
}
34 changes: 0 additions & 34 deletions src/os/lsof_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,6 @@ impl RawConnection {
Protocol::from_str(&self.protocol).unwrap()
}

pub fn get_remote_ip(&self) -> IpAddr {
self.remote_ip.parse().unwrap()
}

pub fn get_remote_port(&self) -> u16 {
self.remote_port.parse::<u16>().unwrap()
}

pub fn get_local_ip(&self) -> IpAddr {
self.local_ip.parse().unwrap()
}
Expand Down Expand Up @@ -203,19 +195,6 @@ com.apple 590 etoledom 204u IPv4 0x28ffb9c04111253f 0t0 TCP 192.168.1.
assert!(connection.is_none());
}

#[test]
fn test_raw_connection_parse_remote_port_ipv4() {
test_raw_connection_parse_remote_port(LINE_RAW_OUTPUT);
}
#[test]
fn test_raw_connection_parse_remote_port_ipv6() {
test_raw_connection_parse_remote_port(IPV6_LINE_RAW_OUTPUT);
}
fn test_raw_connection_parse_remote_port(raw_output: &str) {
let connection = RawConnection::new(raw_output).unwrap();
assert_eq!(connection.get_remote_port(), 2222);
}

#[test]
fn test_raw_connection_parse_local_port_ipv4() {
test_raw_connection_parse_local_port(LINE_RAW_OUTPUT);
Expand All @@ -229,19 +208,6 @@ com.apple 590 etoledom 204u IPv4 0x28ffb9c04111253f 0t0 TCP 192.168.1.
assert_eq!(connection.get_local_port(), 1111);
}

#[test]
fn test_raw_connection_parse_ip_address_ipv4() {
test_raw_connection_parse_ip_address(LINE_RAW_OUTPUT, "198.252.206.25");
}
#[test]
fn test_raw_connection_parse_ip_address_ipv6() {
test_raw_connection_parse_ip_address(IPV6_LINE_RAW_OUTPUT, "fe80:4::aede:48ff:fe33:4455");
}
fn test_raw_connection_parse_ip_address(raw_output: &str, ip: &str) {
let connection = RawConnection::new(raw_output).unwrap();
assert_eq!(connection.get_remote_ip().to_string(), String::from(ip));
}

#[test]
fn test_raw_connection_parse_protocol_ipv4() {
test_raw_connection_parse_protocol(LINE_RAW_OUTPUT);
Expand Down
3 changes: 3 additions & 0 deletions src/os/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ pub(self) mod lsof;
#[cfg(any(target_os = "macos", target_os = "freebsd"))]
mod lsof_utils;

#[cfg(target_os = "windows")]
pub(self) mod windows;

mod errors;
mod shared;

Expand Down
26 changes: 26 additions & 0 deletions src/os/shared.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,16 @@ use ::tokio::runtime::Runtime;
use ::std::time;

use crate::os::errors::GetInterfaceErrorKind;
#[cfg(not(target_os = "windows"))]
use signal_hook::iterator::Signals;

#[cfg(target_os = "linux")]
use crate::os::linux::get_open_sockets;
#[cfg(any(target_os = "macos", target_os = "freebsd"))]
use crate::os::lsof::get_open_sockets;
#[cfg(target_os = "windows")]
use crate::os::windows::get_open_sockets;

use crate::{network::dns, OsInputOutput};

pub type OnSigWinch = dyn Fn(Box<dyn Fn()>) + Send;
Expand Down Expand Up @@ -63,6 +67,7 @@ fn get_interface(interface_name: &str) -> Option<NetworkInterface> {
.find(|iface| iface.name == interface_name)
}

#[cfg(not(target_os = "windows"))]
fn sigwinch() -> (Box<OnSigWinch>, Box<SigCleanup>) {
let signals = Signals::new(&[signal_hook::SIGWINCH]).unwrap();
let on_winch = {
Expand All @@ -82,6 +87,15 @@ fn sigwinch() -> (Box<OnSigWinch>, Box<SigCleanup>) {
(Box::new(on_winch), Box::new(cleanup))
}

#[cfg(any(target_os = "windows"))]
fn sigwinch() -> (Box<OnSigWinch>, Box<SigCleanup>) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a better way to return sigwinch stub on Windows?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, are we not handling window size changes on windows? Or did I miss it?
If we're not doing this, isn't the experience a little wonky when you change the terminal window size? Something like: change size, see a broken interface which gets fixed after ~1 second?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows does not provide SIGWINCH, and its not that easy to get the console resize event. https://stackoverflow.com/questions/10856926/sigwinch-equivalent-on-windows

I've just checked and yes,

Something like: change size, see a broken interface which gets fixed after ~1 second?

that is how it behaves, however I noticed that only after you described that.
I think it is a cosmetical issue, and we could leave that for later.

let on_winch = { move |_cb: Box<dyn Fn()>| {} };
let cleanup = move || {
println!("Fake signal cleanup");
};
(Box::new(on_winch), Box::new(cleanup))
}

fn create_write_to_stdout() -> Box<dyn FnMut(String) + Send> {
Box::new({
let mut stdout = io::stdout();
Expand Down Expand Up @@ -180,6 +194,12 @@ pub fn get_input(
datalink::interfaces()
};

#[cfg(any(target_os = "windows"))]
let network_frames = network_interfaces
.iter()
.filter(|iface| !iface.ips.is_empty())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a regression, but I had to do that as on Windows libpnet does not read interface flags which causes iface.is_up() to fail.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a bug in libpnet? Are the interface flags there, or is it not a thing in windows?
For now, can we hide this behind conditional compilation? Have the old way in not(windows) and this one in windows?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, its a TODO in libpnet, I guess conditional is a way to go for now.

         all_ifaces.push(NetworkInterface {
                name: name_str,
                index: (*cursor).Index,
                mac: Some(mac),
                ips: ips,
                // flags: (*cursor).Type, // FIXME [windows]
                flags: 0,
            });

.map(|iface| (iface, get_datalink_channel(iface)));
#[cfg(not(target_os = "windows"))]
let network_frames = network_interfaces
.iter()
.filter(|iface| iface.is_up() && !iface.ips.is_empty())
Expand Down Expand Up @@ -257,3 +277,9 @@ fn eperm_message() -> &'static str {
`cap_sys_ptrace,cap_dac_read_search,cap_net_raw,cap_net_admin+ep`
"#
}

#[inline]
#[cfg(any(target_os = "windows"))]
fn eperm_message() -> &'static str {
"Insufficient permissions to listen on network interface(s). Try running with administrator rights."
}
Loading