Stdout from launcher processes doesn't get redirected to stdout_file provided

[vringar]

Reported on abcb2bf

Describe the bug
I'm not 100% sure this is a bug, but if it isn't this is a documentation issue. The documentation for stdout_file states:

A file name to write all client output to

To Reproduce
Steps to reproduce the behavior:

1. Set stdout_file on LaucherBuilder
2. In run_client write to stdout
3. Observe that stdout_file doesn't exist

Expected behavior
I'd expect the file to be created and contain all the output from all run_client instances

Additional context

Reproducer WARNING: THIS MIGHT WRITE TO /tmp/stdout.log (if it works for you):

use std::{
    cell::RefCell,
    fs::File,
    io::{self, Write},
    os::fd::{AsRawFd, FromRawFd},
    thread::sleep,
    time::Duration,
};

use libafl::{
    corpus::{CachedOnDiskCorpus, InMemoryCorpus},
    events::{EventConfig, Launcher, LlmpRestartingEventManager},
    inputs::BytesInput,
    monitors::MultiMonitor,
    state::StdState,
    Error,
};
use libafl_bolts::{
    core_affinity::Cores,
    os::dup,
    rands::RomuDuoJrRand,
    shmem::{ShMemProvider, StdShMemProvider},
};

fn main() {
    {
        // The shared memory allocator
        let shmem_provider = StdShMemProvider::new().expect("Failed to init shared memory");

        // Launcher config
        let broker_port = 1337;
        let num_cores = 0;
        let cores = Cores::from_cmdline(&format!("0-{num_cores}")).unwrap();

        let mut run_client = |_: Option<_>,
                              _: LlmpRestartingEventManager<
            (),
            StdState<_, InMemoryCorpus<_>, RomuDuoJrRand, CachedOnDiskCorpus<BytesInput>>,
            _,
        >,
                              _| {
            let mut i = 0;
            loop {
                println!("Hello world {i}");
                i += 1;
                sleep(Duration::from_secs(10))
            }
        };
        let stdout_cpy = RefCell::new(unsafe {
            let new_fd = dup(io::stdout().as_raw_fd()).unwrap();
            File::from_raw_fd(new_fd)
        });
        let monitor = MultiMonitor::new(|s| {
            writeln!(stdout_cpy.borrow_mut(), "Monitor says: {s}").unwrap();
        });
        // Build and run a Launcher
        match Launcher::builder()
            .shmem_provider(shmem_provider)
            .broker_port(broker_port)
            .configuration(EventConfig::from_build_id())
            .monitor(monitor)
            .run_client(&mut run_client)
            .cores(&cores)
            .stdout_file(Some("/tmp/stdout.log"))
            .build()
            .launch()
        {
            Ok(()) => (),
            Err(Error::ShuttingDown) => println!("Fuzzing stopped by user. Good bye."),
            Err(err) => panic!("Failed to run launcher: {:?}", err),
        }
    }
}

[tokatoka]

I can't reproduce it even with your reproducer. /tmp/stdout.log does exist for me

toka@DESKTOP-PNU39Q6:/tmp$ cat stdout.log
Hello world 0
Hello world 1

are you sure you didn't export LIBAFL_DEBUG_OUTPUT?
set this env will disable redirecting the stuff

[vringar]

Hmm, this is unfortunate. I verified that LIBAFL_DEBUG_OUTPUT output was not set, retried on the latest master and still got nothing.
Any other ideas, what might be causing this?

[tokatoka]

i don't know.
can you
start the process,
check the fuzzer process's id, (the process that you launched via launcher not the broker)
run ls -la /proc/PID/fd

then you see where these guys points to

[tokatoka]

For example, if I run the exact same code that you pasted.
It should look like this. (and this result makes sense)

toka@DESKTOP-PNU39Q6:/tmp$ ls -la /proc/652036/fd
total 0
dr-x------ 2 toka toka  0 Apr 30 18:14 .
dr-xr-xr-x 9 toka toka  0 Apr 30 18:14 ..
lrwx------ 1 toka toka 64 Apr 30 18:14 0 -> /dev/pts/5
l-wx------ 1 toka toka 64 Apr 30 18:14 1 -> /tmp/stdout.log
l-wx------ 1 toka toka 64 Apr 30 18:14 2 -> /tmp/stdout.log
lrwx------ 1 toka toka 64 Apr 30 18:14 3 -> /dev/pts/5
l-wx------ 1 toka toka 64 Apr 30 18:14 4 -> /tmp/stdout.log

[vringar]

ls -la /proc/152736/fd
total 0
dr-x------ 2 stefan stefan  7 Apr 30 18:33 .
dr-xr-xr-x 9 stefan stefan  0 Apr 30 18:33 ..
lrwx------ 1 stefan stefan 64 Apr 30 18:33 0 -> /dev/pts/4
lrwx------ 1 stefan stefan 64 Apr 30 18:33 1 -> /dev/pts/4
lrwx------ 1 stefan stefan 64 Apr 30 18:33 2 -> /dev/pts/4
lrwx------ 1 stefan stefan 64 Apr 30 18:33 3 -> /dev/pts/4
lrwx------ 1 stefan stefan 64 Apr 30 18:33 4 -> 'socket:[332936]'
lr-x------ 1 stefan stefan 64 Apr 30 18:33 63 -> anon_inode:inotify

And the subprocesses also point to /dev/pts/4, so I'm guessing the redirect in the parent process doesn't get set up correctly?

[vringar]

I think I've found the issue: My cargo.toml is

[dependencies]
libafl = { path = "../LibAFL/libafl/", default-features = false, features = [
    "prelude",
] }
libafl_bolts = { path = "../LibAFL/libafl_bolts/"}
libafl_qemu = { path = "../LibAFL/libafl_qemu/", features = [
    "arm",
    "systemmode",
], default-features = false }

so I'm getting this launch_with_hooks:

LibAFL/libafl/src/events/launcher.rs

Lines 326 to 437 in b231803

	pub fn launch_with_hooks(&mut self, hooks: EMH) -> Result<(), Error> {
	use libafl_bolts::core_affinity;
	let is_client = std::env::var(_AFL_LAUNCHER_CLIENT);
	let mut handles = match is_client {
	Ok(core_conf) => {
	let core_id = core_conf.parse()?;
	// TODO: silence stdout and stderr for clients
	// let debug_output = std::env::var(LIBAFL_DEBUG_OUTPUT).is_ok();
	// the actual client. do the fuzzing
	let (state, mgr) = RestartingMgr::<EMH, MT, S, SP>::builder()
	.shmem_provider(self.shmem_provider.clone())
	.broker_port(self.broker_port)
	.kind(ManagerKind::Client {
	cpu_core: Some(CoreId(core_id)),
	})
	.configuration(self.configuration)
	.serialize_state(self.serialize_state)
	.hooks(hooks)
	.build()
	.launch()?;
	return (self.run_client.take().unwrap())(state, mgr, CoreId(core_id));
	}
	Err(std::env::VarError::NotPresent) => {
	// I am a broker
	// before going to the broker loop, spawn n clients
	#[cfg(windows)]
	if self.stdout_file.is_some() {
	log::info!("Child process file stdio is not supported on Windows yet. Dumping to stdout instead...");
	}
	let core_ids = core_affinity::get_core_ids().unwrap();
	let num_cores = core_ids.len();
	let mut handles = vec![];
	log::info!("spawning on cores: {:?}", self.cores);
	let debug_output = std::env::var("LIBAFL_DEBUG_OUTPUT").is_ok();
	//spawn clients
	for (id, _) in core_ids.iter().enumerate().take(num_cores) {
	if self.cores.ids.iter().any(|&x| x == id.into()) {
	let stdio = if self.stdout_file.is_some() {
	Stdio::inherit()
	} else {
	Stdio::null()
	};
	std::env::set_var(_AFL_LAUNCHER_CLIENT, id.to_string());
	let mut child = startable_self()?;
	let child = (if debug_output {
	&mut child
	} else {
	child.stdout(stdio)
	})
	.spawn()?;
	handles.push(child);
	}
	}
	handles
	}
	Err(_) => panic!("Env variables are broken, received non-unicode!"),
	};
	// It's fine to check this after the client spawn loop - since we won't have spawned any clients...
	// Doing it later means one less check in each spawned process.
	if self.cores.ids.is_empty() {
	return Err(Error::illegal_argument(
	"No cores to spawn on given, cannot launch anything.",
	));
	}
	if self.spawn_broker {
	#[cfg(feature = "std")]
	log::info!("I am broker!!.");
	RestartingMgr::<EMH, MT, S, SP>::builder()
	.shmem_provider(self.shmem_provider.clone())
	.monitor(Some(self.monitor.clone()))
	.broker_port(self.broker_port)
	.kind(ManagerKind::Broker)
	.remote_broker_addr(self.remote_broker_addr)
	.exit_cleanly_after(Some(NonZeroUsize::try_from(self.cores.ids.len()).unwrap()))
	.configuration(self.configuration)
	.serialize_state(self.serialize_state)
	.hooks(hooks)
	.build()
	.launch()?;
	//broker exited. kill all clients.
	for handle in &mut handles {
	handle.kill()?;
	}
	} else {
	log::info!("Not spawning broker (spawn_broker is false). Waiting for fuzzer children to exit...");
	for handle in &mut handles {
	let ecode = handle.wait()?;
	if !ecode.success() {
	log::info!("Client with handle {handle:?} exited with {ecode:?}");
	}
	}
	}
	Ok(())
	}
	}

where the whole redirect thing doesn't get set up.
Is this only supported when using the forking launcher?

[vringar]

Sorry for not including the Cargo.toml in the initial report.

[tokatoka]

that's the problem
https://github.com/AFLplusplus/LibAFL/blob/main/libafl/src/events/launcher.rs#L190
no fork then no fd inheritance from the parent process

[vringar]

Should I open a PR to hide the fields if the wrong feature is selected? This way they can't be set when they don't actually do anything?

[tokatoka]

yes! please 👍