Completing a CLI with <mask>... <file> never offers <file> completions with Rust native completions

[windsource]

Please complete the following tasks

• I have searched the discussions
• I have searched the open and rejected issues

Rust Version

rustc 1.80.1 (3f5fd8dd4 2024-08-06)

Clap Version

clap 4.5.16, clap_complete 4.5.23

Minimal reproducible code

use std::ffi::OsStr;

use clap::{CommandFactory, Parser, ValueHint};
use clap_complete::{ArgValueCompleter, CompleteEnv, CompletionCandidate};

fn mask_completer(_: &OsStr) -> Vec<CompletionCandidate> {
    vec![
        CompletionCandidate::new("foo"),
        CompletionCandidate::new("baz"),
    ]
}

#[derive(Parser, Debug)]
#[command(name = "cmdtest")]
#[command(version, about, long_about = None)]
struct Args {
    #[arg(required = true, add = ArgValueCompleter::new(mask_completer))]
    mask: Vec<String>,

    #[arg(required = true, value_hint = ValueHint::FilePath)]
    file: String,
}

fn main() {
    CompleteEnv::with_factory(Args::command).complete();

    let args = Args::parse();

    for m in args.mask {
        println!("Mask: {}", m);
    }
    println!("File: {}", args.file)
}

Steps to reproduce the bug with the above code

cmdtest foo AND PRESSING TAB

Actual Behaviour

As completion candidates I only get "foo" and "baz"

Expected Behaviour

I was expecting to "foo", "baz" and the files in the current folder as completion candidates.

Additional Context

After the first argument for mask is passed, the next argument could either be a mask or a file so I was expecting to get completions for both but I only get completions for mask.

I am using zsh.

Debug Output

No response

[epage]

This is parsing two positionals in a row with the first positional's length being unbounded. This is a very complicated case in the parser that we've not covered yet with the new completions. Currently, our quality bar is "is it as good as the old completions". If you happened to use the old completions before and can report back on its behavior in this scenario, that can help us prioritize this.

See

clap/clap_builder/src/parser/parser.rs

Lines 288 to 358 in fe81090

	// Correct pos_counter.
	pos_counter = {
	let is_second_to_last = pos_counter + 1 == positional_count;
	// The last positional argument, or second to last positional
	// argument may be set to .multiple_values(true) or `.multiple_occurrences(true)`
	let low_index_mults = is_second_to_last
	&& self.cmd.get_positionals().any(|a| {
	a.is_multiple() && (positional_count != a.get_index().unwrap_or(0))
	})
	&& self
	.cmd
	.get_positionals()
	.last()
	.map(|p_name| !p_name.is_last_set())
	.unwrap_or_default();
	let is_terminated = self
	.cmd
	.get_keymap()
	.get(&pos_counter)
	.map(|a| a.get_value_terminator().is_some())
	.unwrap_or_default();
	let missing_pos = self.cmd.is_allow_missing_positional_set()
	&& is_second_to_last
	&& !trailing_values;
	debug!("Parser::get_matches_with: Positional counter...{pos_counter}");
	debug!("Parser::get_matches_with: Low index multiples...{low_index_mults:?}");
	if (low_index_mults || missing_pos) && !is_terminated {
	let skip_current = if let Some(n) = raw_args.peek(&args_cursor) {
	if let Some(arg) = self
	.cmd
	.get_positionals()
	.find(|a| a.get_index() == Some(pos_counter))
	{
	// If next value looks like a new_arg or it's a
	// subcommand, skip positional argument under current
	// pos_counter(which means current value cannot be a
	// positional argument with a value next to it), assume
	// current value matches the next arg.
	self.is_new_arg(&n, arg)
	|| self
	.possible_subcommand(n.to_value(), valid_arg_found)
	.is_some()
	} else {
	true
	}
	} else {
	true
	};
	if skip_current {
	debug!("Parser::get_matches_with: Bumping the positional counter...");
	pos_counter + 1
	} else {
	pos_counter
	}
	} else if trailing_values
	&& (self.cmd.is_allow_missing_positional_set() || contains_last)
	{
	// Came to -- and one positional has .last(true) set, so we go immediately
	// to the last (highest index) positional
	debug!("Parser::get_matches_with: .last(true) and --, setting last pos");
	positional_count
	} else {
	pos_counter
	}
	};

What will be interesting about this case is that clap definitively knows when mask is done and file starts because it can peek ahead on the command-line. While the command-line is being written, we can't guarantee there will be anything to peek ahead to. Effectively, we'll need to just assume that any positional after the minimum possible is either mask or file and complete it as such.

To help in showing either completion, we should probably group completions by their argument by default (see #5651).

[windsource]

@epage not sure what you mean with "old completions"? Is there any older clap_complete release which I should try to see if the completions for my scenario work with?

[epage]

CompleteEnv is an in-work completion system that is unstable. clap_complete::aot is what we have historically provided.