The intuition behind Rust’s borrowing rules and ownership

The concept of ownership and borrowing was not a very obvious one for me when I first looked at it. After spending some time with Rust and a lot of thinking it started to become clear.

In this blog I will try to capture the thoughts that finally made this concept click for me in the hope that it might help others also. I will go over my thoughts on why memory management in Rust works the way it does and my interpretation of the borrowing rules. You can copy and run the examples at the Rust Playground as you follow the blog.

What is the borrow checker trying to do?

Let’s first take a look at some Python code that demonstrates a common bug that happens in reference based languages

a = [1, 2, 3]

b = a
a.append(4)

print(b)  # prints [1, 2, 3, 4]

If you have spent enough time with python, you expected it to be [1, 2, 3, 4] instead of [1, 2, 3] at line 6. So what is happening here? At line 3, the statement b = a is not actually copying the value of a and assigning it to b, instead it is assigning a reference of a to b. Assigning a reference means that you point to an existing value instead of having your own value.

So because b does not have its own value and is pointing to an existing value, modifying that existing value is the same as modifying b, in this case the existing value is a. One important property that we can note here is that the line that modifies the value (a.append(4)) does not have the letter b in it, in other words the modification was done behind b’s back, unknown to b. When we printed b on line 6, we saw this unexpected modification.

The borrow checker tries to prevent you from using variables which are pointing to an existing value that could have been potentially unexpectedly modified behind the variable’s back.

Let’s look at a roughly¹ equivalent Rust program to really understand what this statement means

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;
    a.push(4);

    println!("{:?}", b);
}

The mut keyword in line 2 lets us modify a and do a.push(4) at line 5. At line 4, we do b = &mut a, which is equivalent to doing b = a in the Python example. Which means we are assigning the reference of a to b. The default b = a in Rust does something else, which we will look into later.

The borrow checker prevented us from using b at line 7 because the value it was pointing to was unexpectedly modified behind its back at line 5.

Let’s try to make this error go away. One way is to make the unexpected modification an expected one. You can do that by changing a.push(4) to b.push(4). The line that does the modification now has the letter b in it and the modification is not done behind b’s back. Make the change and run the program, you can see it compiles successfully and prints [1, 2, 3, 4]!

Another way to make the error go away is to use b before doing any unexpected change. We will move the last line that prints b to before the a.push(4) line, before the unexpected modification behind b’s back. This program prints [1, 2, 3] instead of [1, 2, 3, 4], but the error goes away.

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;
    println!("{:?}", b);    // this line was moved up

    a.push(4);
}

Although the last line a.push(4) is modifying the value that b is pointing to behind its back, we don’t use b after the modification, so this is okay. The borrow checker prevents only the later use of variables whose values have been unexpectedly modified behind their back, not the modification itself.

Before we go further, let’s establish some Rust terminology. In all of the above examples a is called the owner and b is borrowing a’s value. Borrowing is done through either using &mut a or &a. The data that the borrower b is pointing to is being borrowed from the owner of the data a.

The difference between &mut borrowing and just & borrowing is that & mut borrowing lets the borrower mutate the data it is pointing to. The below set of examples will demonstrate this

fn main() {
    let mut a = vec![1, 2, 3];  // owner of the data

    let b = &a;                 // plain & borrowing
    b.push(4);                  // try and modify a via b

    println!("{:?}", a);        // print a
                                //  the modification
                                //  will reflect here
}

You will get a compile error at line 5 saying you can’t mutate using just a plain & reference, change to an &mut reference. So let’s exactly do that

fn main() {
    let mut a = vec![1, 2, 3];  // owner of the data

    let b = &mut a;             // &mut borrowing
    b.push(4);                  // try and modify a via b

    println!("{:?}", a);        // print a
                                //  the modification
                                //  will reflect here
}

This program should compile, and it should print [1, 2, 3, 4]. The &mut at line 4 allowed the borrower b to modify the data it was borrowing from a. We saw the modification when we printed the owner a at line 7. Hold on a second! Something seems a little off about this program? At line 5 b is modifying data behind a’s back, and that modification is later used at line 7. So why does this program compile? Because a is an owner and not a borrower.

The borrowing rules apply to the borrowers and not for the owners. The borrow checker prevents you from using borrowers whose borrowed data could have been potentially unexpectedly modified behind its back.

What does the word “potentially” mean here? Let’s look at another example to understand that

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;
    let c = &a;


    println!("{:?} {:?}", b, c);     // print b and c
}

There was no modification here at all, so why does this program not compile? By printing or using b at line 8, you have established the fact that b is relevant at lines 5 to 8. So you are likely in the future add a line like b.push(4) between these lines and modify the data behind c’s back. The thing b that can be used to change the data behind another borrowers’s back c is still relevant, and it could be potentially used to do so and you can potentially see that modification at line 8.

Of course there is no modification here and you might think this error message is too strict but you can think of this as being similar to getting a lint error for using let instead of const in Javascript. We will see why you can think of it that way by making the error go away, by removing mut from line 4.

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &a; // change &mut to plan &
    let c = &a;


    println!("{:?} {:?}", b, c);     // print b and c
}

This program compiles and prints [1, 2, 3] [1, 2, 3]. By removing mut from line 4, you are declaring that b cannot change the content it is borrowing. So statements like b.push(4) are not allowed. So even if b is relevant at lines 5 to 8, you can’t use it to modify data behind c’s back.

Another way to make the error go away instead of removing mut would be to remove b from the print statement at line 8

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;
    let c = &a;


    println!("{:?}", c);     // remove b, print only c here
}

By removing b from line 8, you have told the compiler it is not relevant anymore from lines 5 to 8. And hence you are not likely in the future to add a statement like b.push(4) between these lines and make a modification behind c’s back.

I have been using the word relevant a lot, this is really a crude substitute for the concept of lifetimes. We will improve our model of the borrow checker later in the lifetimes section and revisit some of the examples using the improved model. The current model that we have developed so far is a little different from how it is actually implemented in the Rust compiler, but it does explain the reasoning behind the existence of the borrow checker and the kinds of error it was designed to catch. The borrow checker was designed to catch “data races”.

The Rust Book mentions “data races”, the word “unexpected” and “the modification behind X’s back” is my interpretation of the word “data races”. More often the borrowing rules are explained using the below statement from the Rust Book

Let’s try to understand this statement using what we have discussed so far. The only way to prevent data being modified behind a borrower’s back is to make sure when a borrower is a writer, that borrower is the only relevant borrower. If there are multiple borrowers and at least one of them can write, then one of the writer borrowers can change the data behind another borrowers back. Hence the statement “you can have either one mutable reference or any number of immutable references”. “Mutable reference” means a borrower that can write and “immutable reference” means a borrower that can’t write, only read.

The “at any given time” part of the statement at the beginning is the confusing one here, what I think it means is at any given line, for a given data, get all of the borrowers of that data that are relevant at that line and make sure that “either one mutable reference or any number of immutable references” is followed.

Lifetimes

In Rust Lifetimes are a construct that is used to refer to a span of lines or region in which a variable is relevant. Lifetimes in Rust are used to prevent dangling references and is also used by the borrow checker to prevent data races. Let’s start by taking an example from the Rust Book

fn main() {
    let r;                // --------------+-- life of
                          //               |   r
    {                     //               |
        let x = 5;        // -+-- life of  |
        r = &x;           //  |   x        |
    }                     // -+            |
                          //               |
    println!("r: {}", r); //               |
}                         // --------------+

In this example, there is a borrower r borrowing data from the owner x. The borrower r has to live from the point it is declared at line 2 till the it goes out of scope at line 10. The owner x has to live from the point it is declared at line 5 till it goes out of scope at line 7.

The issue here is that the owner x is invalidated (dead) at line 7, since it went out of scope, but the borrower continues to live on beyond the owner, which means the borrower is pointing to invalid data! Borrowers pointing to invalid data are called “dangling references” and Rust will not allow such references to exist.

Rust treats this as a type mismatch and won’t compile the program. The owner must always live as long as or outlive the borrower. Here the owner x does not live as long as the borrower r, hence the error. You could also say, r wants to live longer but x won’t allow it to do so, because x is forced to get invalidated at line 7 because of the scope.

fn main() {
    let r;                // --------------+-- life of
                          //               |   r
                          //               |
    let x = 5;            // -+-- life of  |
    r = &x;               //  |   x        |
                          //  |            |
                          //  |            |
    println!("r: {}", r); //  |            |
}                         // --------------+

Removing the scope allows the compiler to make x live longer, as long as r and so r still points to valid data and the program compiles.

Coming back to the discussion of data races, Rust will also not allow a writer borrower to live along side any other borrower (does not matter if the other borrower is a writer or not). Because if a writer borrower is allowed to live along side another borrower, it can modify data behind the other borrower’s back. This will violate the “you can have either one mutable reference or any number of immutable references” rule.

Let’s revisit a previous example and look at it from the perspective of lifetimes

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;                // -------------+- life of
    let c = &a;                    // -+- life of  |  b
                                   //  |  c        |
                                   //  |           |
    println!("{:?} {:?}", b, c);   //  |           |
}                                  // -+-----------+

Our previous explanation to why this won’t compile was that the borrower b that could be used to used to modify data behind borrower c’s back was still relevant at lines 5 to 8, so there is potential for the unexpected modification to be seen at line 8.

Let’s now try to explain this using lifetimes, we can see the writer borrower b is alive while another borrower c is also alive, and Rust will not allow a writer borrower live along side any other borrowers, so this won’t compile.

If we remove mut from line 4, then b becomes just another read only borrower, and two read only borrowers can live along side each other, so the program compiles.

We saw earlier that another way to make the error go away instead of removing mut would be to remove b from the print statement at line 8

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;        // --------------- life of b
    let c = &a;            // -+- life of     b
                           //  |  c
                           //  |
    println!("{:?}", c);   //  |
}                          // -+

When we remove b from the print statement at line 8, the Rust compiler is now able to invalidate (kill) b at the same line it is created. So life of b lasts only for a single line, line 4. Then c lives from lines 5 to 8. Now the writer borrower b does not live along side another borrower c, so the program compiles.

You can see although b did not go out of scope, the Rust compiler invalidated b much earlier in the program, so the program can compile. The compiler analyzes your program and always tries to come up with lifetimes to make your program compile, it can be invalidating borrowers earlier or it can be making owners live longer so borrowers still point to valid data. If it can’t it will throw a compile error. Rust will always ensure that borrowers never outlive an owner and a writer borrower never lives along side any other borrower.

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;                // -------------+- life of
                                   //              |  b
    println!("{:?} {:?}", b, a);   //              |
}                                  // -------------+

This program does not compile. At first glance it looks like there is only one writer borrower b that lives from lines 4 to 7, but there is actually another temporary borrower created by the println! statement for passing a as an argument that lives only for a single line, line 6. Since there is a writer borrower b that lives along side another temporary borrower created by println! for, this program won’t compile.

Similar reason also applies to modifying the owner. Let’s revisit another previous example to see this

fn main() {
    let mut a = vec![1, 2, 3];

    let b = &mut a;        // -------------+- life of
    a.push(4);             //              |  b
                           //              |
    println!("{:?}", b);   //              |
}                          // -------------+

Our previous explanation to why this won’t compile was that a.push(4) at line 5 was modifying data behind b’s back and that unexpected modification was seen later at line 7 in the println! statement.

Looking at it from the perspective of lifetimes, at first glance it looks like there is only one writer borrower b the lives from lines 4 to 8, but there is actually another temporary writer borrower created at line 5 by the a.push(4) statement so it can modify a. So at line 5, there is actually a temporary writer borrow that is alive along side another (writer) borrower b, hence this program won’t compile.

Ownership and how memory is freed in Rust

In all of the examples we have seen so far, we have never seen the statement b = a, only b = &a or b = &mut a. So what does b = a in Rust mean? Let’s find out from the next example

fn main() {
    let a = vec![1, 2, 3, 4];

    let b = a;

    println!("{:?}", a);
}

You can see a new kind of error at line 4, “value moved here”. What this means is the ownership of the data vec![1, 2, 3, 4] was transferred to b when we did let b = a in line 4, so a is no longer valid, b becomes the new owner. So you can’t print a at line 6 because a was invalidated at line 4. So the default b = a in Rust means to transfer ownership.

You can also see the compiler suggests changing line 4 to let b = a.clone() instead of let b = a, what this does is copies all of the data, and make b the owner of the new copy of the data, a will still remain the owner of the old data and will remain valid, and the program will compile and run. Rust will actually copy by default for primitive values² such as integers and floats since they are cheap to copy unlike something like a vec! (list). So the below program will still work

fn main() {
    let a = 1;

    let b = a;  // a is copied here because it is an integer

    println!("{:?}", a);  // a is still valid here
}

Another way owners can get invalidated is if they go out of scope, as we saw earlier in the lifetimes section

fn main() {
    {
        let a = vec![1, 2, 3, 4];
    } // end of scope, a is no longer valid

    println!("{:?}", a);   // compile error, a is not valid
}

What happens to all the borrowers if the owner from which they are borrowing from gets invalidated? The borrowers will also get invalidated because they are not allowed to outlive the owner. Let’s look at an example to understand this

fn main() {
    let a = vec![1, 2, 3];
    let b = &a;

    let x = a;    // a along with the borrower b
                  //    becomes invalid here

    println!("{:?}", b);
}

This program won’t compile. The borrower b wants to live from line 3 till is goes out of scope at line 9, but the owner a lives only till line 5, because when we did let x = a at line 5, we invalidated the owner a and transferred the ownership to x. Since owner is not able to outlive the borrower, this program won’t compile.

Let’s note down some observations on ownership from what we have discussed so far

The first three points are slight rephrasing of the ownership rules as mentioned in the Rust Book. These properties that we have observed are very critical to how Rust does memory management. These properties are the reason why Rust does not need something like a garbage collector or a runtime. This is because since a given piece of data can only have one owner and all of the borrowers get invalidated if the owner is invalidated, Rust knows at compile time exactly when to delete and free memory occupied by that data: when the owner stops being alive.

a = [1, 2, 3]
b = a
del a
print(b)

This program will work and print [1, 2, 3]. The variable a was invalidated at line 3, and b still continues to work. In python, there is no concept of ownership, instead every variable shares data. When we did a = [1, 2, 3] at line 1, a was the only one using the data [1, 2, 3]. When we did b = a, b started sharing the data with a. So python has to keep track of all the variables sharing a particular piece of data at runtime, it can only free the data once the count of all variables sharing the data becomes zero. This is usually done with Reference counting or Garbage collection.

In Rust when the owner of the data stops being alive, you know no one else is using the data, because when the owner stops being alive and gets invalidated, the borrowers are also invalidated. There are also no other owners, every value or data has only one owner. So there should be no variable using the data that was owned by the owner after the owner stops being alive. So Rust can insert code at compile time to free memory when the owner stops being alive. You can see this code insertion in action in the below example

struct Foo {
    x: i32
}

impl Drop for Foo {
    fn drop(&mut self) {
        println!("Dropping Foo with x {}!", self.x);
    }
}

fn main() {
    let a = Foo{x: 100};

    // compiler automatically inserts code to call drop
    //  because a goes out of scope and stops being alive here
}

This program will print Dropping Foo with x 100!. We create a struct (or class) called foo, then we add a method³ called drop to it. The drop method is a special one, the compiler automatically inserts code to call this method on owners when they stop being alive. The Vec types (created by calling vec!) in Rust use this method to free memory that was allocated for the items.

Lifetime annotations

Rust lifetimes annotations usually appear when you define a function that takes one or more borrowers and returns a borrower or when you define a struct that holds one or more borrowers. When a function takes one or more borrowers and returns a borrower, the borrow checker needs to know the relationship between lifetimes of input borrowers and lifetimes of output returned borrowers so it can look for errors at the call site. This relationship is specified using the 'a annotation syntax.

It is important to remember that these are just annotations that specify and define a relationship as part of the function signature that has to be followed both by the function body and at the call site. They don’t modify the arguments or the return value in any way. They are just like return types and argument types, types that has to be followed both by the function body and at the call site.

fn foo<'a>(i: i32, x: &'a Vec<i32>, y: &'a Vec<i32>) -> &'a Vec<i32> {
    if i <= 10 {
        return x;
    }

    return y;
}

fn main() {
    let p = vec![1, 2, 3];
    let q = vec![4, 5, 6];

    let r = foo(4, &p, &q);

    println!("{:?}", r);
}

If you ignore the 'a, you can see the function foo takes in three arguments, i: i32, x: &Vec<i32> and y: &Vec<i32>. The function foo takes in two borrowers, x and y, and returns one of them depending on the value i. At line 13, the call site, we call foo with 4 and pass borrowed data from the owners p and q and assign the returned borrower to r. So how do we know from which owner r is borrowing from? The function foo could return a borrower that is borrowing from either of the owners depending on what value was passed to i. So how does the compiler know the borrower r is not outliving the owner?

You might think the compiler can analyze the function and figure out that i is 4, so foo must be returning borrowed data from p and hence r is borrowing from the owner p, and r’s lifetime should not outlive p’s lifetime, but this is not possible all the time. What if i was coming from the user as a command line argument?

The compiler needs some more information, that information is the relationship between lifetimes of input borrowers x and y and lifetime of output returned borrower. By adding 'a to input borrowers in the argument types and output borrower in the return type, you are linking and defining a relationship between the lifetimes of input borrowers and the lifetime of output returned borrower. The relationship is “lifetime of each input borrower should be at least as long as the output borrower”. Let’s see this in action by slightly modifying the example

fn foo<'a>(i: i32, x: &'a Vec<i32>, y: &'a Vec<i32>) -> &'a Vec<i32> {
    if i <= 10 {
        return x;
    }

    return y;
}

fn main() {
    let p = vec![1, 2, 3];

    let r;

    {
        let q = vec![4, 5, 6];
        r = foo(4, &p, &q);
    }   // q goes out of scope here and gets invalidates


    println!("{:?}", r);
}

This program does not compile, let’s find out why. In this example, in the signature of the function foo the arguments x and y have the 'a, and hence their lifetimes are linked to the lifetime of output returned borrower which also has a 'a. At line 16, where foo is called, you establish a relationship between r and the borrowed data from p and q. The relationship as mentioned before is that the lifetime of both p and q should be at least as long as the lifetime of r. The borrower r wants to live from line 12 till it goes out of scope at line 21. The owner p upholds the relationship by living from line 10 to line 21. But the owner q only lives till line 17 and goes out of scope and breaks the relationship, so the program won’t compile.

One way to fix this error would be to redefine the relationship. Here this program failed to compile because the lifetime of r was linked to the lifetime of q. If we remove this link, then this program should compile. How to remove the link? By not giving the 'a to q (or y from the function’s perspective). Let’s modify the foo function to do this

fn foo<'a>(i: i32, x: &'a Vec<i32>, y: &'_ Vec<i32>) -> &'a Vec<i32> {
    return x;
}

Notice that the second argument y has '_ instead of 'a. So the lifetime of input borrower y has no relationship with the lifetime of output returned borrower. Since only x and the output returned borrower has the 'a, only their lifetimes are linked and the lifetime of argument y does not matter. Since lifetime of y and the lifetime of output returned borrower are not related, we are not allowed to return y, so we also change the body to always return x. Make this change and the program will compile (ignoring the warning about unused arguments in i and y in foo).

The key take away here is that Rust allows you to specify lifetime annotations as part of the function signature, so it can check for two things

Lifetimes can also appear in a struct as mentioned before. When a struct is passed to or returned from a function, you can think of it being the same as passing or returning multiple arguments or values, with each argument or value being the individual fields of the struct from a function, with one caveat: the struct cannot exist if the lifetime relationship for one of the borrower fields break. Let’s look at an example to understand this

struct Baz<'a, 'b> {
    k: &'a Vec<i32>,
    l: &'a Vec<i32>,
    m: &'b Vec<i32>,
}

fn bar<'u, 'v>(x: &'u Vec<i32>, y: &'v Vec<i32>) -> Baz<'u, 'v> {
    Baz {
        k: x,
        l: x,
        m: y,
    }
}

fn main() {
    let p = vec![1, 2, 3];

    let test: Baz;

    {
        let q = vec![4, 5, 6];
        test = bar(&p, &q);
    }   // q gets invalidated here,

    println!("{:?} {:?}", test.k, test.l); // compile error
}

If you look at the signature of bar, you can see the lifetime of output of the function has a relationship with the lifetimes of the input arguments x and y. Specifically, the lifetime of the fields k and l of the return type Baz is linked to the input argument x. This is because x has a 'u and in the return type Baz also has a 'u substituted for 'a in the angle brackets and 'a appears for the fields k and l in the definition of Baz. Take a minute to pause and understand the statement if it seems a bit complex. It helps to think of the lifetimes annotations 'a and 'b in the Baz struct as generics type parameters. Similarly the lifetime of the input argument y is linked to lifetime of the m field of the return type with 'v.

In the function body, we return a Baz with k and l being equal to the input argument x (hence they all have the same 'u and their lifetimes are linked), and m being equal to the input argument y (hence they have the same 'v).

So the relationship that has been defined here is the input lifetime of x must live at least as long as the lifetime of k and l fields of the output return and the input lifetime of y must live at least as long as the lifetime of m field of the output return.

In the call site at line 22, the function call forms a relationship between test and the borrowed data from the owners p and q. The lifetimes of k and l fields of test are linked to borrowed data from the owner p. So the owner p has to outlive or live as long as the k and l fields of test. The fields k and l of test wants to live from line 22 till the point it goes out of scope at line 26. The owner p upholds the relationship by living from line 18 till 26.

Similarly the lifetime of m field of test is linked to borrowed data from the owner q. So the owner q has to outlive or live as long as the m field of test. The m field of the struct test is never used so it could be invalidated at the same line it was created, line 22, but this field must live at least as long as its parent struct test. So the m field also wants to live from line 22 to line 26 even if it is not used anywhere because of its parent test. This was the caveat mentioned before. The owner q fails to uphold the relationship and only lives from line 21 to line 23 and the program fails to compile.

struct Baz<'a, 'b> {
    k: &'a Vec<i32>,
    l: &'a Vec<i32>,
    m: &'b Vec<i32>,
}

fn bar<'u, 'v>(x: &'u Vec<i32>, y: &'v Vec<i32>) -> Baz<'u, 'v> {
    Baz {
        k: x,
        l: x,
        m: y,
    }
}

fn main() {
    let p = vec![1, 2, 3];

    let test: Baz;
    let a;
    let b;
    let c;

    {
        let q = vec![4, 5, 6];
        test = bar(&p, &q);
        a = test.k;
        b = test.l;
        c = test.m;
    }   // q gets invalidated here,

    println!("{:?} {:?}", a, b);
}

Here test only want to lives till line 29 and does not appear in the println! statement at the end. So the owner q can now uphold the relationship by living till line 29. The borrowers a and b inherit the lifetime relationship of k and l fields of test that was formed with the owner p.

This above example was actually a complicated example of lifetimes, it is not common to see two lifetimes in a struct definition like this, most Rust code you write will only have one. Rust also implicitly declares this lifetimes for you based on elision rules if your function signature is simple.

Conclusion

I hope this blog post has captured some of the reasons for why Rust’s borrowing and ownership works the way it does. I did not cover examples with control flow like if statements or for loops, so please do modify the examples and experiment. This blog post is not meant to be complete resource on learning borrowing and ownership, so I highly recommend reading the Rust Book and other resources also.