1. Meaning.
   1. In this class we will focus upon the meaning of programming languages.
   2. For example, we will ask and answer such questions as
      1. What does it mean for a programming language to be functional?
      2. What does it mean for a programming language to be strongly typed?
      3. What does it mean for a programming language to be object oriented?
      4. What does it mean for a programming language to be more powerful than another language?
      5. What does it mean for a programming language to be evil and dangerous?
   3. In order to answer these and other such questions, we must first investigate how the meaning of programming languages can be clearly, concisely, and formally expressed.
      1. This requires that we study notations for the formal semantics of programming languages.
      2. At their best, these notations provide the means to define semantics in a manner comparable to how BNF to defines syntax.
      3. That is, a good notation for formal semantics expresses meaning in a concise and formal way, without ambiguity.
      4. As we shall see, there are a number of approaches for expressing semantics, suitable for a number of purposes ranging from compiler writing to formal program verification.

2. How is meaning defined?
   1. In general, how do we define the meaning of a language?
   2. Consider, for example, how we define the meaning of English.
      1. In a dictionary, we define English in terms of itself.
      2. Anthropological linguists define English in terms of the historical languages upon which it is based.
      3. Structural linguists define English by building formal models of its syntax and semantics.
   3. For defining the meaning of programming languages, we use similar techniques.
      1. We can define the meaning of a language in terms of itself, by writing a compiler or interpreter for the language in itself.
      2. We can define programming languages historically in terms of the generations of languages that have come before.
      3. Finally, we can define programming languages by building formal models of their syntax and semantics.
   4. The specific forms of semantic definition we will study in this class are the following:
      1. Operational semantics -- defining the meaning of a language by writing a compiler or interpreter for it.
      2. Attribute grammars -- defining meaning with a formal BNF grammar plus semantic equations.
      3. Denotational semantics -- a formal BNF grammar plus functional definitions.
      4. Axiomatics semantics -- a formal BNF plus plus logical axioms.
      5. Algebraic semantics -- an algebraic definition consisting of semantic primitives and reduction rules.

3. Programming languages as religion.
   1. Computer scientists are particularly fond of heated discussion about programming languages.
      1. Such discussion often takes on the character of religious debate.
      2. I.e., discussants argue more on the basis of what they believe about programming languages, rather than what they can prove definitively.
   2. In a fundamental sense, much of the debate about programming languages is moot.
      1. While computer scientists can argue ad nauseum about one language being "better" than another, most computer scientists know that in a certain theoretical sense, there is no such thing as a "better" programming language.
      2. That is, it can be shown that the fundamental computational power is provably the same for all programming languages.
      3. Hence, what is left to debate is how conveniently, efficiently, and/or aesthetically languages express certain forms of computation.
   3. Despite what computer scientists know about languages, they continue to debate most about what they believe.
   4. We will join the debate in this class.

4. This class will be taught using a particular belief system.
   1. Specifically, it will be taught from the "applicative" (aka, "functional") point of view.
      1. This is in general contrast to the "imperative" point of view.
      2. We will define both "applicative" and "imperative" more precisely in a moment.
   2. From a pedagogical standpoint, teaching with a strong point of view can be unsound, unless "opposing" viewpoints are given fair treatment.
      1. Part of the applicative belief system is that applicative languages are a better vehicle for investigation of programming language principles than are imperative languages.
      2. In particular, using an applicative approach, we can learn about both applicative and imperative concepts more readily than with an imperative approach.
   3. Other aspects of teaching with a strong point of view include:
      1. Keeping the facts straight.
      2. Keeping the opinions pointed, provocative, and sometimes outlandish.

5. Some initial definitions.
   1. An applicative language is one in which the fundamental model of computation is based on the application of side-effect-free functions.
      1. A side-effect-free function is one in which the entire result of computation is produced by the return value(s) of the function.
      2. Side-effect-free functions can only access explicit input parameters; there are no global variables in a fully applicative language.
      3. And, in the purest form of applicative language, there are no assignment statements.
   2. An imperative language is one in which the fundamental model of computation is based on the execution of a sequence of instructions that modify a state memory.
      1. The imperative model of computation is familiar to programmers who use C, Java, and most other common programming languages
      2. In contrast to an applicative language, functions in imperative languages often have side effects, and the assignment statement is a fundamentally important part of the language.

6. The foundations of computing and programming languages.
   1. The distinction between applicative and imperative languages is one of the most fundamental distinctions in all of computer science.
   2. To examine the distinction fully, we will start by going all the way back to the pre-history of computing.
      1. In so doing, we will see that applicative and imperative languages stem from two fundamentally different foundations.
      2. Viz., applicative languages stem from a foundation in mathematical logic, in which computing is based on a theory of recursive functions.
      3. In contrast, imperative languages stem from a machine-based foundation, in which computing is based on a stored-program computational device. ¹

7. Turing machines and the imperative model of computation.
   1. Founders: Alan Turing, John von Neumann, and others.
   2. As you probably know, a Turing Machine (TM) is a model of effective computability.
      1. That is, the TM model of computation accounts for anything that is computable.
      2. Hence, the TM model can be (and is) used as a foundation for both computing machines and the languages in which machine programs are expressed.
   3. Formally, a TM is a state machine, consisting of the following components:
      1. An infinite memory tape, divided into individual memory slots
      2. A movable head, that can perform three functions:
         1. Read a symbol from the tape slot immediately below the head.
         2. Write a symbol onto the tape slot immediately below the head.
         3. Move one slot to the right or the left on the tape.
   4. A TM program consists of a set of quintuples of the form:

      (current state, symbol read, new state, symbol written, move direction)
      

   5. As a very simple example, here is a TM program that computes the constant value 4, expressed in unary notation (i.e., 1111), given an initially blank tape:

      (0, ,1,1,R)
      (1, ,2,1,R)
      (2, ,3,1,R)
      (3, ,4,1,R)
      

   6. As another example, here is a TM program to add two positive integers, expressed in unary notation:

      (0,1,1,X,R)
      (0,,,0,,,R)
      (0,:,3,:,R)
      
      (1,1,1,1,R)
      (1,,, 1,,,R)
      (1,:, 1,:,R)
      (1, , 2,1,L)
      
      (2,1,2,1,L)
      (2,:,2,:,L)
      (2,,,2,:,L)
      (2,X,0,X,R)
      

      1. Here is a sample input tape (to add 2+3):

         11,111:
         ^
         0
         

      2. Here is, the resulting output tape (showing the unary sum of 2+3 to the right of the ':'):

         XX,XXX:11111
                ^
                3
         

      3. Here is a description of what each state does:

         State	Description
         0	check for 1, ',', or ':' and goto state 1, 0, or 3, resp; if a 1, replace it with a 'X' to indicate that it's been handled as an operand
         1	carry a 1 over to the far right end of the tape and write it
         2	go back to the rightmost X to get the next 1
         3	halt

8. Recursive function theory and the applicative model of computation.
   1. Founders: Stephen Kleene, Alonso Church, and others.
   2. Recursive function theory (RFT) is an alternative (and equivalent) model of effective computability.
   3. As with the TM model, RFT accounts for anything that is computable.
   4. Formally, a recursive function is defined as one of the following:
      1. The Zero function: Z(x) = 0
      2. The Successor function: S(x) = x + 1
      3. A function defined as a composition of recursive functions:

         f(x0,...,xn) = h(g0(x0,...,xn),...,gk(x0,...,xn))
         

      4. A function defined using an inductive recursion scheme:

         f(0,x1,...,xn) = g(x1,...,xn)
         f(S(n),x1,...,xn) = h(f(x1,...,xn),n,x1,...,xn)
         

         where g and h are defined recursively ²
   5. Here is the recursive function definition of the constant function that computes 4 (the first TM example above):

      Four(x) = S(S(S(S(Z(x)))))
      

   6. Here is the recursive function definition of addition on natural numbers (the second TM example above):

      Add(0,y) = y
      Add(S(n),y) = S(Add(n,y))
      

      1. The first equation is the inductive base for addition -- adding 0 to anything returns the original number (and stops any recursion)
      2. The second equation is the inductive step -- for any n, to add (n+1) to y, recursively add n to y, and then add 1 to the result.

9. The formal equivalence of TMs and RFT.
   1. It can be proved formally that the (infinite) set of all computations definable by a TM can be defined using recursive functions, and vice versa.
   2. This is in fact an extremely important (and comforting) result (and it's the basis for the earlier assertion that no programming language is "better" than any other, if "better" is taken to mean a language's fundamental power of computational expression).
   3. While TM's and RFT are equivalent models of computation, they are also equivalently unsuited to use for practical programming.
      1. In particular, both lack adequate control and data representations.
      2. Encoding even the simplest computations is usually quite tedious using either formalism.
   4. What is important about the two formalisms is the fundamental models of computation that they represent, and how these models affect the course of computation.
   5. Another important piece of theory is the Church Hypothesis
      1. Not only are TM's and RFT equivalent, each in its own right captures the essence of effective computability.
      2. That is, there is no devisable system of computation that is fundamentally more powerful than TMs (and therefore RFT).
      3. This hypothesis is clearly unprovable, but it is generally believed by all.
         1. The belief is strengthened by the existence of several other models of computation, different from both TMs and RFT.
         2. All such models so far developed have been proved formally equivalent to TMs.

10. A practical comparison of the two models.
    1. In the TM model:
       1. The computation is defined as a sequence of instructions (the program).
       2. Data are stored in a sequential memory, which changes state as the computation proceeds.
       3. Computation is carried out by executing the program instructions sequentially.
    2. In the RFT model:
       1. The computation is defined by a set of functions (the program).
       2. Data are passed as parameters to functions and returned as function values (there is no state memory at all).
       3. Computation is carried out by invoking functions, not necessarily sequentially.
    3. Summary:
       1. The TM model is the fundamental basis for imperative languages.
       2. The RFT model is the fundamental basis for applicative languages.

11. Some compelling motivations for applicative programming
    1. Concurrency
    2. Verifiability
    3. Referential transparency
    4. We'll discuss each of these in detail in upcoming lectures.

12. A question for the applicative zealot
    1. One question that should be asked of applicative language promoters is the following:

       If the advantages of applicative languages are so compelling, why is their use not more widespread?

    2. Answer 1: Programmers are inherently lazy and weak-willed.
       1. Writing programs in an applicative language feels a lot like doing mathematics.
       2. Real programmers don't do mathematics (even though they should).
    3. Answer 2: Present-day hardware isn't any good.
       1. Almost all hardware platforms are designed based on an imperative model of computation, making them ill-suited to execute applicative programs efficiently.
       2. We need radically new architectures, suitable for applicative computation (and they're coming).
    4. Answer 3: We are at an unhappy point in the natural evolution of programming languages.
       1. Natural selection does in fact work, including for programming languages.
       2. However, the process of natural selection sometimes conducts unsuccessful experiments -- like dinosaurs and imperative languages.
       3. At present, we are in the age of programming language dinosaurs, awaiting the cosmic event that will lead to rapid dinosaur demise, and the subsequent evolution of intelligent languages.
          1. Ada is the Brontosaurus of languages; small brain, large unwieldy body, slow grazing behavior.
          2. C++ is the T Rex of the languages; we watch with morbid fascination as it devours all in its path and wreaks havoc on the planet; we wait anxiously for its extinction.
          3. Java is the pterodactyl, seemingly more lithe and agile than the other dinos, but destined for extinction with the rest.
          4. ML is the proto-mammalian successor to the dinosaurs, waiting to spring forth and evolve.
    
    
    
    

    IHaving introduced the basic historical concepts, we now proceed to examine
    applicative languages in detail, beginning with Pure Lisp.

13. A thought experiment -- assignmentless programming
    1. The most distinctive feature of a purely applicative language is its lack of assignment statements.
    2. So, take your favorite imperative programming language and throw out assignment statements.
       1. What kind of programming could you do?
       2. Would any non-trivial programming even be possible?
       3. In case you're thinking that call-by-var parameters and/or pointers might help, these get thrown out as well, as do for-loops, while-loops, and all other iteration constructs.
    3. While a language completely devoid of any form of assignment is extreme, such a language represents the essence of applicative programming.
    4. A fundamental tenet of applicative programming is that data do not change.
       1. Rather, data are manipulated and constructed as they pass through functions.
       2. Data are never mutated once having been constructed and a single datum is never accessed by more than one function.
       3. Data values are like electrons flowing through hardware wires.
    5. In order for useful and efficient computation to be performed in an applicative language, it cannot be constructed simply by removing assignment statements from some imperative language.
       1. Rather, we must provide applicative language primitives that allow us to effectively program without assignment statements, and without other imperative features.
       2. Typically, imperative languages do not have the necessary applicative primitives to allow us to get by without assignment.

14. The necessary evil of imperative constructs
    1. Very few real programming languages are completely applicative (or completely imperative, for that matter).
    2. In the practical world, languages are primarily one category or the other.
       1. For example, almost all imperative languages have constructs to define and apply recursive functions.
       2. Similarly, most applicative languages have constructs for some form of assignment and/or data mutation.
    3. We will begin our study of applicative languages from a pure standpoint -- no assignment and no other applicative constructs.
    4. Subsequently, we will see how imperative features can fit into an applicative framework (but not without some penalty).

15. Motivation for the study of Pure Lisp
    1. It defines the most fundamental aspects of all programming languages in a very simple and elegant way.
    2. It is useful as a vehicle to introduce purely applicative programming.
    3. It is also useful as a vehicle to describe the operational semantics of both applicative and imperative programming, for which purpose we will use it in the programming assignments.
    4. Related to the previous point, Lisp is a good tool for the rapid prototyping of language translators, to investigate how new language features can be used to write actual programs.

16. General features of Pure Lisp, vis a vis Pascal-class languages ³.
    1. Lisp has syntax different from Pascal-class languages, but this difference is profoundly unimportant.
    2. (Pure) Lisp is an untyped language.
       1. Unlike most Pascal-class languages, the types of Lisp objects are not declared explicitly.
       2. For example, a single variable can be used to hold a numeric value at one point in a computation, and a list value at some other point in the same computation
    3. Lisp is an expression language
       1. It does not have declarations or statements in the conventional sense of Pascal-class languages.
       2. Every construct in the language returns a value.
    4. The overall style of Lisp programming is recursive, not iterative.
    5. Lisp is built on a few simple and orthogonal primitives.
       1. It's simplicity helps those who study programming languages focus on fundamental concepts rather than unimportant details.
       2. The orthogonality of its primitives means that each primitive provides a single form of functionality, and there is no unnecessary overlap with other primitives (cf. the orthogonal bases of a vector space).

17. The basis of a functional language -- the function definition
    1. Here's a simple example

       (defun APlusB (a b)
           (+ a b)
       )
       

    2. For a conventional frame of reference, here's the equivalent definition in C:

       int APlusB(int a,b) {
           return a + b;
       }
       

    3. There are a number of observations we can make from this simple example.
       1. The basic concept of function definition is the same in Lisp as in an imperative language, such as C.
          1. A function has a declared name (e.g., APlusB).
          2. There are formal parameters (e.g., a and b).
          3. There is a computational body (e.g., (+ a b)).
       2. Note again that Lisp is an untyped language
          1. Neither the return value nor formal parameters for the APlusB function have declared types.
          2. Evidently, the parameters must be numeric (or a least addable), since the body of the function adds them together.
          3. However, the Lisp translator does not enforce any static type requirements on the formal parameters; any addition errors will be caught at runtime.
          4. It should be noted that Lisp's untypedness is not a necessary feature of an applicative language; for example, ML is both strongly typed and applicative.
       3. In Lisp's syntax, all expressions are given in prefix notation.
          1. In most modern programming languages, built-in functions (such as arithmetic) can be invoked using infix notation (e.g., "a+b") instead of prefix notation (e.g., "+(a,b)").
          2. In Lisp, all functions, whether built-in or user-defined, are invoked in prefix notation.
          3. Notice further the rather quirky function invocation syntax.
             1. Viz., the name of a function appears inside the parens, rather than outside, and the actual function parameters are separated by spaces rather than the more typical commas.
             2. Here's a side-by-side look at Lisp calling forms and the equivalent C forms:

                Lisp Form	CJ Form	Remarks
                (+ a b)	a + b	Call the built-in addition function
                (APlusB 10 20)	APlusB(10, 20)	Call the user-defined function APlusB

          4. It should be noted that Lisp's unusual invocation syntax has nothing whatsoever to do with applicativeness or imperativeness, though there is a good reason for it (and it's a reason you'll certainly appreciate in all of the programming assignments).
       4. Notice the lack of a return statement in the Lisp function definition.
          1. This owes to Lisp being an expression language -- i.e., every construct returns a value.
          2. In the case of a function definition, the value that the function returns is whatever value its expression body returns.
          3. No explicit "return" is necessary (this takes some getting used to).
          4. This notion of no explicit returns is a fundamental feature of applicative languages, not just another Lisp quirk.

18. The one and only applicative control construct -- cond (a.k.a., if-then-else)
    1. Lisp has a conditional control construct comparable to the if-then-elsif-else in imperative languages.
    2. The general form is the following:

       (cond  ( (test-expression1) (expression11) ... (expression1j) ) )
                     . . .
                  ( (test-expressionn) (expressionn1) ... (expressionnk) )
       

    3. The test-expressions are evaluated in order until the first true case is encountered (say the ith), whereupon the ith expression sequence is executed and the cond is finished; if no test expression is true, no expression sequence is executed.
    4. This conditional form takes some getting used to, but once mastered it is quite handy.
    5. Examples of its use are coming up.

19. A suitable data structure for an applicative language -- the heterogeneous list.
    1. A Lisp list is a collection or zero more elements.
       1. Since Lisp is an untyped language, list elements can be any type of data, including (recursively) lists themselves.
       2. Lists are a superset of arrays, and can easily be used to represent records.
       3. Further, the recursive nature of lists allows them to represent data structures that require pointers in imperative languages.
       4. Hence, the list is a single data structure that can easily represent most of the typical built-in data structures of imperative languages.
    2. The precise definition of a Lisp list is as follows:
       1. A list is a collections of zero or more elements, enclosed in matching parentheses.
       2. List elements can be atoms or lists.
       3. An atom is a number, string, boolean, or identifier.
    3. There are only three fundamental list operations, from which all others can be derived:

       Operation	Meaning
       car	return the first element of a list
       cdr	return everything except the first element of a list
       cons	construct a new list, given an atom and another list

    4. The following fundamental relationships exist between the three list primitives:
       1. (car (cons X Y)) = X
       2. (cdr (cons X Y)) = Y
    5. Examples to follow shortly.

20. A completely Lisp-specific function -- quote
    1. If one carefully examines the Lisp syntax we have discussed thus far, there is an interesting potential problem.
    2. Viz. there is no syntactic distinction between a function invocation and a list datum.
       1. E.g, consider the following two function definitions:

          (defun f (x) ... )
          (defun a (x) ...)
          

       2. Given these definitions, what does the following Lisp form represent?

          (f (a b))
          

       3. Is it
          1. A call to function f, with the list argument (a b)?
          2. A call to function f, with an argument that is the result of a call to function a with argument b?
       4. The answer is (b).
       5. That is, the default meaning for a plain list form in Lisp is a function call.
       6. To obtain the alternate meaning (a) above, we must use the Lisp quote function (usually abbreviated as a single quote character) to indicate that we want to treat a list as a literal datum.
       7. I.e., the following form produces meaning (a) above:

          (f '(a b))
          

21. Iteration through recursion
    1. In applicative languages, the iterative control constructs found in imperative languages are replaced by recursion.
       1. Hence, a purely applicative language has no while, for, repeat, or the like.
       2. In fact, a while loop is fundamentally unsuited for inclusion in an applicative language, for reasons that we'll discuss a little later (you might want to think about this now, however).
    2. As an example, consider the following equivalent Lisp and C programs that compute the average of a list (array) of integers:
       1. Lisp:

          (defun avg (l)
              (/ (sum l) (length l))
          )
          
          
          (defun sum (l)
              (cond ((null l) 0)
                    (t (+ (car l) (sum (cdr l))))
              )
          )
          
          
          (defun main ()
              (avg '(1 2 3 4 5))
          )
          

       2. C:

          int avg(int l[], int length) {
              int i, sum;
              for (i=0, sum=0; i<length; i++)
                  sum += l[i];
              return sum/length;
          }
          
          
          main() {
              int l[] = {1,2,3,4,5};
              printf("%d0, avg(l, 5));
          }
          

    3. Here are some key observations about these two definitions of the averaging program:
       1. In the Lisp version of sum, a technique commonly called tail recursion is used to effect iteration.
       2. To better understand how tail recursion is working, here is a transliteration of the Lisp sum function into C, assuming that normal Lisp built-in functions are still available (i.e., null, car, and cdr) and there is a built-in list datatype:

          int sum(list l) {
              if (null(l))          /* if the list l is empty */
                  return 0;         /*   then return 0 as the summing result */
              else                  /* otherwise, */
                  return car(l) +   /*   return the sum of the 1st list element */
                      sum(cdr(l);   /*     with the sum of the rest of the elements */
          

       3. The idea of this tail recursion is that each recursive invocation of a function handles the first element of a list, and passes on the rest of the list (i.e., the tail) to another recursive invocation of the same function.
       4. The recursion stops when the list has been exhausted by the tail having been reduced one element at a time.

22. Another illustrative list-processing example.
    1. In all real Lisp environments, there are many more list processing functions than just the three basic primitives.
    2. A fundamentally important feature of Lisp is that any list operation can be (easily) built using the three primitives.
    3. As an example, here is the implementation ⁴ of the built-in function nth, that returns the nth element of a list (starting from 0), if it exists.

       (defun my-nth (n l)
           (cond ( (< n 0) nil )
                 ( (eq n 0) (car l) )
                 ( t (my-nth (- n 1) (cdr l)) )
           )
       )
       

    4. This function uses a similar form of tail recursion as used in the sum function discussed above.
       1. The reader is invited to perform a careful dissection of my-nth.
       2. The dissection should help reveal precisely how the recursion is working.
       3. Such use of recursion is fundamental to successful functional programming.

Footnotes:

¹ This is not to imply that the machine-based approach is "unmathematical". The point here is to contrast a foundation of mathematical logic versus a foundation on state machines.

² The definition given here that ofprimitive recursive functions rather than general recursive; primitive recursion is adequate for the discussion at hand.

³ "Pascal-class language" is a somewhat dated term used to refer to the general class of imperative languages that came after FORTRAN. These days, the "CJ" family of languages is probably a better term, where "CJ" stands for the class of languages consisting of C, C++, and Java.

⁴ It's named "my-nth" since "nth" is built-in to Lisp, and we don't want to change its definition.