Hi Rust experts,

My team has been porting our serverless API to Rust, being the Lambda functions conversions quite smoothly. Then came the time to replace an in-memory buffer that was written in Java, deployed as ECS task. Although it works quite efficiently, I hate those occasional GC issues that happen before instances scale-out. I know that Rust's standard library provides a very reasonable LinkedList implementation but, as a learning experiment, I've been trying to put in place a simplified version of the very known Persistence Stack.

Sadly, I'm running into Stack Overflow when trying to populate it with 10k entries - a side effect that I couldn't notice using LinkedList. I've dug into the LinkedList code and noticed it uses very clever strategies to swap head and tail elements - I certainly wouldn't get the same results without further research. But it's not clear why I got SO in my dummy implementation. Sadly I couldn't set up a convenient way to plot the memory usage in a way that I could easily identify which variables were fulfilling the stack.

Perhaps you guys could point me:

1. Where was my mistake?
2. Which tool could I have used to identify this by myself?

Below you can find a minimal sample that simulates the error. After that, you can also find the output of its only test.

    use std::rc::Rc;

    pub struct Buffer<T> where T: Debug   Clone {
        pub head: BufferEntry<T>,
        pub tail: BufferEntry<T>
    }

    type BufferEntry<T> = Option<Rc<Entry<T>>>;

    pub struct Entry<T> where T: Debug   Clone {
        pub value: T,
        pub next: Option<Rc<Entry<T>>>
    }

    impl<T> Buffer<T> where T: Debug   Clone {

        pub fn new(value: T) -> Self {
            let head = Some(Rc::new(Entry { value, next: None }));
            Buffer { tail: head.clone(), head }
        }

        pub fn append(&mut self, value: T) {
            let next = self.tail.take();
            self.tail.replace(Rc::new(Entry {
                value, next
            }));
        }
    }


    #[cfg(test)]
    mod tests {
        //use std::collections::LinkedList;

        use super::*;

        #[test]
        fn should_create_a_massive_list_on_reasonable_period_of_time() {

            // let mut i = 10_000_000;
            let mut max = 10_000;

            let mut buffer = Buffer::new(max   1);
            // let mut buffer = LinkedList::new();

            while max > 0 {
                buffer.append(max);
                // buffer.push_back(max);
                max-= 1;
            }

            println!("Done")
        }
    }

The test output:

> Executing task: cargo test --package stdlib-playground --lib -- experiment::buffer_entry::tests --nocapture <

    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on package cache
    Blocking waiting for file lock on build directory
   Compiling stdlib-playground v0.1.0 (/Users/miere/Development/Sandbox/rust/stdlib-playground)
    Finished test [unoptimized   debuginfo] target(s) in 0.78s
     Running target/debug/deps/stdlib_playground-5e02c37c1e37950d

running 1 test
Done

thread 'experiment::buffer_entry::tests::should_create_a_massive_list_on_reasonable_period_of_time' has overflowed its stack
fatal runtime error: stack overflow
error: test failed, to rerun pass '-p stdlib-playground --lib'

Caused by:
  process didn't exit successfully: `/Users/miere/Development/Sandbox/rust/stdlib-playground/target/debug/deps/stdlib_playground-5e02c37c1e37950d 'experiments::buffer_entry::tests' --nocapture` (signal: 6, SIGABRT: process abort signal)
The terminal process "/bin/zsh '-c', 'cargo test --package stdlib-playground --lib -- experiments::buffer_entry::tests --nocapture'" terminated with exit code: 101.