CS 6983 Special Topics: The Technology of Lambda -*- Outline -*- Olin Shivers Project 5: Closure conversion -- Analysis 2: Environment Analysis Due 2026/3/17 midnight by git commit * 1. Project description ------------------------ This is the analysis pass that works out how to pass data around the first-order code in the registers. Unlike prior analyses, it's *not* a recursive tree walk. As you'll see it's more of a backwards flow analysis that is essentially a form of liveness analysis (like the kind you would write to do graph-coloring register allocation). The information we get is the heart of your compiler's ability to use registers instead of the stack or the heap to communicate values from one part of the program to another -- it's how we get C-level performance for the C-like parts of the program. As with the previous project, you may want to refer to the summary chart of all the analysis annotations that is provided at the end of the project 6 description. ------------------------------------------------------------------------------- * 2. Environment analysis ------------------------- The point of closure conversion is deciding how environment structure will be implemented. This pass is the heart of that work. In this analysis, we determine the key issue of what the total variable "needs" of a lambda are, an extension of the idea of "free variables." The big idea is that when we closure-convert our program, we will transform the code so that all first-order lambdas are passed their free variables from their call sites as extra parameters -- which means that, after closure conversion, no first-order lambda will have any free variables. Closure lambdas (after conversion) won't have free variables, either -- all their free-variable references will be turned into loads from their closure record, which will be passed to these lambdas by the caller. Only open lambdas will have free variables after conversion, and they don't really count, being inlined where they appear. When we're done with the (forthcoming) transform, every variable is just a register. (Well, a virtual register -- we'll pack these down to physical registers later.) (Wait, won't passing all these extra parameters around... and around... from call to call to call involve a lot of work? No. Again, post-closure-conversion, all these variables are just registers. So "passing them as parameters" in first-order function calls are just register/register copies. These "copies" are really just coalescing requests to the register allocator.) But if we unpack this idea a little bit, we discover that a given lambda can require access to more of its in-scope variables than just the ones in its free-variable set. This is due to the way we're going to implement first-order lambdas when we convert them. To motivate the idea of "total needs," let's consider an example, where we call a first-order lambda that itself calls another first-order lambda. Suppose call-site C1 calls first-order lambda L1, and inside L1's body, there is a call C2 to *another* first-order lambda L2: C1: (f x y k1) ; Call C1 jumps to F, which is let-bound to 1st-order L1. L1: (fun (a b k2) ; First-order lambda L1 has free vars G, C, Z (if z ; Here's a free ref to Z. C2:(g b c k2) ; Call C2 jumps to G (which was let-bound to L2) ...)) L2: (fun (p q r k4) ; Here's first-order lambda L2, which ... s ... w ...) ; contains references to free vars S & W. Since L1 is a first-order function, we are going to transform it and its call sites (when we do closure conversion), so that it is passed its free variables (G, C, Z) as extra parameters. Except we don't have to pass G, right? G is a first-order function, so we won't need to have G passed around as a *value*. We'll just compile its use in call C2:(G B C K2) as a *direct* jump to a *known* code location. So, we just need for C1 to pass L1 the extra parameters C & K3, right? Not so fast. L1 needs more -- because L1 has to pass to L2 (from call C2, inside L1) the free variables that L2 needs. That's the contract -- L2 is a first-order lambda, so it is going to be passed its free variables S & W from its call site, C2. But that means that L1 needs access to S & W itself, so it can, in turn, pass them to along to L2. Bottom line: C1 needs to pass to L1 the variables C & Z (as L1 needs these) *and* the variables S & W (so that L1 will have them to pass on to L2). Put another way: we don't want to pass *every* in-scope variable visible at C1 to L1. That is... we *could* do it, but why bother? L1 probably doesn't need all of them. We just want to pass to it the variable bindings that it needs: its free variables, which are a subset of its in-scope variables. That's easier and less work. But this subset of the in-scope variables that C1 needs to supply to L1 *transitively* includes the needs of all the first-order functions L1 calls, even though L1 doesn't use them directly. So we need the transitive closure of these "what we need" sets. We want some way of saying that the call site requires / uses / needs these extra variables. That's the EXTENDED set for a lambda -- it's a subset of its in-scope variables... but possibly a super-set of its free variables. Every lambda and LETREC L is annotated with a set of variables called its "extended free variables." These are (a) its free variables, plus (b) the variables that are *also* needed by L because they are needed by some first-order lambda L' that is *called from* L. Because LETREC lets us have cycles in our environment structure, these sets are mutually dependent; we want to find the smallest sets that satisfy these dependencies. That is to say, we are looking for the *least fixed point* of some generating function. To summarise: Binder L (a lambda or LETREC) needs - all the variables in its free-variable set -- except those bound to first-order lambdas -- and - all the variables needed by all the first-order lambdas called by L. (See how "needs" occurs recursively in this definition?) When we do simple free-var analysis, the rules of lexical scope say that sets propagate *upwards* in the AST. For example, the free-var sets of the test, consequent and alternate sub-terms of an IF form propagate up into the free-var set of the IF itself. Likewise, the free-var set of some argument in a call form propagates upward to the free-var set of the call itself. But when we consider the extra needs of a first-order lambda L, however, growth in this set of variables propagates from callee L *back* to each call C that calls L. When variable needs propagate from some first-order lambda L back to a call C that is the body of binder B, that increases B's "needs." (That is, it increases B's EXTENDED set.) - If B is a LETREC, these extra needs propagate up into B's parent LETREC or lambda in the AST. (How convenient that B has a label annotation allowing you to move up in the AST: this is why we have the PARENT-BINDER annotation.) - If B is a closure or open lambda, again, these extra needs likewise propagate *up* into B's parent-binder in the AST. - But if B is a first-order lambda... then these extra needs propagate *back* to each call site C that calls B, and from there up into C's containing binder. (Again, you have just the right annotation to help you navigate along this backwards control-flow path, right? These are just the binders in B's CALLERS annotation.) Notice that in the case of an open lambda, propagating to its lexical parent and its caller is exactly the same thing, so the two different propagations collapse together: the "back" link (control) is the same as the "up" link (environment). If you think about it, you'll realise that this "extended" free variable analysis is the same backwards flow analysis and the same problem as determining live-variable sets in the CFG of, say, a single function in a C compiler. This is where we address that issue, in our lambda-based compiler. You job here is to write an analysis that will take a program and determine the EXTENDED annotation for every binding form (lambda or LETREC) in the program. This analysis will hop around the AST in non-tree-walking ways, following callee/caller control links to propagate informations as EXTENDED sets grow. So, your analysis works like this: - Make a dictionary mapping labels to the AST nodes they name. This will allow you to hop around the program following various links. (In a more highly engineered compiler, you'd have real pointers in the AST nodes for these links.) - Make a set of all the first-order variables in the program (that is, all the variables that are bound to first-order lambdas). - Make another dictionary mapping the label of a binding form (lambda or LETREC) to the extended free-variable set for that form. The table should have an entry for every binding form in the program; initialise each entry to the simple free variable set for that form, minus the variables that are first-order. - Propagate information around the program until you reach a fixed point. This is *not* a simple tree-walk traversal of the AST. You can do this with a worklist algorithm, or with an equivalent recursive one. Every binder in your table should go on the initial worklist. (It's called a "work list," but it's really a *set* -- in this case, a set of labels for binders whose sets have grown and therefore for whom we need to propagate the possible consequences of that growth.) - Walk the AST installing the final extended free-variable sets for every binding form as an annotation.