fixed Replacer trait to work more like BufRead::read_line

[eminence]

I'm trying to write something like the following:

struct MyReplacer;

impl Replacer for MyReplacer {
    fn reg_replace(&mut self, caps: &regex::Captures) -> Cow<str>  {
        let cap: &str = caps.at(1).unwrap();

        Cow::Borrowed(cap)

    }
}

fn main() {
    let re = regex::Regex::new("foo(.*)bar").unwrap();
    let my_data = "foocatsbar";
    let result = re.replace_all(my_data, MyReplacer);
}

But I'm running into the following lifetime error:

src/lib.rs:11:30: 11:35 error: cannot infer an appropriate lifetime for lifetime parameter `'t` due to conflicting requirements [E0495]
src/lib.rs:11         let cap: &str = caps.at(1).unwrap();
                                           ^~~~~

There are some suggestions about adding explicit lifetime parameters, but it seems like nothing I tried compiled.

What I'm trying to do is replace some parts of a String, but it's not possible to write a regex that will exactly represent what I want. So in my MyReplacer, I want to extract the capture group, and based on the value of the capture group, either build a new string to return as a Cow::Owned(String), or return the capture group as Cow::Borrowed(&str) as to avoid an allocation.

Do you have any hints about how to do this? I'm on rust nightly (1.7)

Thanks!

[BurntSushi]

This is a tricky one. I think the fundamental problem you're running into is in the definition of the Replacer trait:

pub trait Replacer {
    fn reg_replace(&mut self, caps: &Captures) -> Cow<str>;

    fn no_expand(&mut self) -> Option<Cow<str>> { None }
}

If we apply lifetime elision rules, then I think it looks like this:

pub trait Replacer {
    fn reg_replace<'a, 'b, 'c>(&'a mut self, caps: &'b Captures<'c>) -> Cow<'a, str>;

    fn no_expand<'a>(&'a mut self) -> Option<Cow<'a, str>> { None }
}

This means that the returned string has a lifetime tied to self, but your code is trying to return something with a lifetime tied to the return value of caps.at, which translates to the lifetime 'c in this code. I claim that this indeed what I intended when I initially wrote the trait definition. Namely, 'a refers to the lifetime of the string that is used as the replacement, while 'c in this definition refers to the lifetime of the text that is being modified (i.e., the original search text). This configuration is important with respect to replacement expansion. For example, if your replacement string is "$name", then a new copy must be made for each match since $name is replaced with the named sub-capture match name (e.g., (?P<name>[0-9]+)). If a replacement string is wrapped in NoExpand, then it never needs to have expansion done, so a copy of it is unnecessary. Hence, the reason why Cow is returned with a lifetime tied to self.

This means that, given the Replacer definition, it is impossible to return a Cow with a lifetime tied to the search text. That's unfortunate, because it does seem like something you might occasionally want to do. I'm not sure whether that's easy to fix or not. I think we would have to at least abandon Cow for something with three variants and two parameterized lifetimes:

enum Replacement<'r, 't> {
    /// A borrow of the replacement string. Matches current semantics.
    ReplaceBorrowed(&'r str),
    /// A borrow of the input text string.
    TextBorrowed(&'t str),
    /// An owned string.
    Owned(String),
}

This would let you return one of three types of strings: one tied to the replacement string (available today), one tied to the original search text (unavailable today) or one that is owned (available today).

Of course, this doesn't help you today. Fortunately, the replacement functions are relatively straight-forward to implement. Here's your example expanded with the above definition, a better trait definition and tweaked replacement functions to make everything work:

extern crate regex;

use std::borrow::Cow;
use std::ops::Deref;

use regex::{Captures, Regex};

trait BetterReplacer {
    fn reg_replace<'r, 't>(
        &'r mut self,
        caps: &Captures<'t>,
    ) -> CowReplacement<'r, 't>;

    fn no_expand(&mut self) -> Option<Cow<str>> { None }
}

struct MyReplacer;

impl BetterReplacer for MyReplacer {
    fn reg_replace<'r, 't>(
        &'r mut self,
        caps: &regex::Captures<'t>,
    ) -> CowReplacement<'r, 't>  {
        CowReplacement::BorrowedSearchText(caps.at(1).unwrap())
    }
}

#[allow(dead_code)]
enum CowReplacement<'r, 't> {
    /// A borrow of the replacement string. Matches current semantics.
    BorrowedReplacement(&'r str),
    /// A borrow of the searched string.
    BorrowedSearchText(&'t str),
    /// An owned string.
    Owned(String),
}

impl<'r, 't> Deref for CowReplacement<'r, 't> {
    type Target = str;

    fn deref(&self) -> &Self::Target {
        use self::CowReplacement::*;
        match *self {
            BorrowedReplacement(s) => s,
            BorrowedSearchText(s) => s,
            Owned(ref s) => &**s,
        }
    }
}

fn replace_all<R: BetterReplacer>(re: &Regex, text: &str, rep: R) -> String {
    replacen(re, text, 0, rep)
}

fn replacen<R: BetterReplacer>(
    re: &Regex,
    text: &str,
    limit: usize,
    mut rep: R,
) -> String {
    let mut new = String::with_capacity(text.len());
    let mut last_match = 0;

    if rep.no_expand().is_some() {
        // borrow checker pains. `rep` is borrowed mutably in the `else`
        // branch below.
        let rep = rep.no_expand().unwrap();
        for (i, (s, e)) in re.find_iter(text).enumerate() {
            if limit > 0 && i >= limit {
                break
            }
            new.push_str(&text[last_match..s]);
            new.push_str(&rep);
            last_match = e;
        }
    } else {
        for (i, cap) in re.captures_iter(text).enumerate() {
            if limit > 0 && i >= limit {
                break
            }
            // unwrap on 0 is OK because captures only reports matches
            let (s, e) = cap.pos(0).unwrap();
            new.push_str(&text[last_match..s]);
            new.push_str(&rep.reg_replace(&cap));
            last_match = e;
        }
    }
    new.push_str(&text[last_match..]);
    new
}

fn main() {
    let re = Regex::new("foo(.*)bar").unwrap();
    let my_data = "foocatsbar";
    println!("{}", replace_all(&re, my_data, MyReplacer));
}

Thanks very much for this report. Hopefully we'll be able to get a fix into the library proper! (Although it will be a breaking change.)

[eminence]

Thanks for the clear and detailed reply! I suspected something like this was going on, but without rigorously applying the lifetime elision rules, I fell short of a complete understanding.

I look forward to seeing this use-case handled in the future!

[BurntSushi]

I came up with a much better solution to this problem that both avoids the lifetime issue and gives more explicit control over allocation. Instead of controlling allocation indirectly via borrows, we should just have the implementor write to the destination buffer explicitly. Here's the new trait definition:

pub trait Replacer {
    fn replace_append(&mut self, caps: &Captures, dst: &mut Vec<u8>);

    fn no_expansion<'r>(&'r mut self) -> Option<Cow<'r, [u8]>> {
        None
    }
}

In your case, all you'd need to do is write the contents of cap.at(1) directly to dst and then never worry about whether the borrowing is right or not.

This removes the need for an extra type and makes implementations of Replacer quite a bit simpler!

[BurntSushi]

(Sorry, that code should read &mut String. I'm working on the bytes API at the moment.)

[BurntSushi]

Note that this is fixed in the bytes::Regex API (using the replace_append variant of the Replacer trait).