Unstable benchmark excluding setup feature

[fereidani]

This PR tries to resolve a missing feature by adding a new benchmarking method iter_excluding_setup.

This idea came from working on vec::retain algorithm when I noticed that benchmark has about 15% noise of preparing the benchmark data:

    let mut v = Vec::with_capacity(100000);

    b.iter(|| {
        v.clear();
        v.extend(black_box(1..=100000));
        v.retain(|x| x & 1 == 0)
    });

This includes time to clear and extend in benchmark time, which is not our benchmark target.
As an algorithm improves through development this preparation noise increases and benchmark time becomes less reliable.

With this feature we can rewrite it as following:

    b.iter_excluding_setup(
        Vec::with_capacity(100000),
        |v| {
            v.clear();
            v.extend(1..=100000);
        },
        |v| v.retain(|x| x & 1 == 0),
    );

In contrary, iter_excluding_setup times exactly the retain part and ignores preparation phase.

    vec::bench_retain_100000      43740.52ns/iter +/- 7667.35
    vec::bench_retain_100000_original 50763.95ns/iter +/- 6600.93

To achieve this we need to account for Instant::now syscall overhead. I tested with both average and minimum sample of 1000 runs. I think worst case min is more reliable than average. an empty functions benchmarks around 0ns-3ns on my laptop's 5850U cpu which is IMHO acceptable.

If we account that Instant::now() is 20ns, 1000 runs will be calculated in 20 microseconds once and will be reused after that, which is not a concern for any benchmark.

I also removed single Instant::now() sample from current iter benchmarks to try to slightly improve its accuracy for single runs or small benchmarks.

This feature is also applicable and needed in current string benchmarks in alloctests/benches when allocation time is included in insert benchmarks resulting in about 100% noise:

    string::bench_insert_char_long      28.12ns/iter  +/- 4.51
    string::bench_insert_char_long_original  55.20ns/iter  +/- 6.21
    string::bench_insert_char_short     27.04ns/iter  +/- 5.25
    string::bench_insert_char_short_original 48.79ns/iter  +/- 6.16
    string::bench_insert_str_long       29.96ns/iter  +/- 7.37
    string::bench_insert_str_long_original   54.21ns/iter +/- 29.19
    string::bench_insert_str_short      30.72ns/iter  +/- 6.36
    string::bench_insert_str_short_original  48.78ns/iter  +/- 8.15

Rewrite with excluding setup sample:

#[bench]
fn bench_insert_char_short_original(b: &mut Bencher) {
    let s = "Hello, World!";
    b.iter(|| {
        let mut x = String::from(s);
        black_box(&mut x).insert(black_box(6), black_box(' '));
        x
    })
}

#[bench]
fn bench_insert_char_short(b: &mut Bencher) {
    let s = "Hello, World!";
    b.iter_excluding_setup(
        String::new(),
        |x| *x = String::from(s),
        |x| black_box(x).insert(black_box(6), black_box(' ')),
    )
}

We can also use this feature to benchmark exactly allocation and deallocation of an object, for example:

#[bench]
fn deallocate(b: &mut Bencher) {
    b.iter_excluding_setup(
        None,
        |v| *v = Some(Arc::new(0usize)),
        |v| {
            _ = black_box(v).take();
        },
    );
}

#[bench]
fn allocate(b: &mut Bencher) {
    b.iter_excluding_setup(
        None,
        |v| *v = None,
        |v| {
            *v = black_box(Some(Arc::new(0usize)));
        },
    );
}

    arc::allocate    7.68ns/iter +/- 3.11
    arc::deallocate 10.11ns/iter +/- 2.51

This is currently impossible to do with current available features, It will be also useful in benchmarking more complex data structures like hashmaps and linked lists too. (hash collisions, empty hashmap insertion, deallocation, allocation, etc.)

I only need this features to improve rust library itself, and there is no need for stabilization.

[rustbot]

r? @JohnTitor

rustbot has assigned @JohnTitor.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: libs
• libs expanded to 12 candidates
• Random selection from JohnTitor, Mark-Simulacrum, clarfonthey, nia-e, tgross35