CPS 343/543 Lecture notes: Introduction




Abridged transcript of lecture

This is CPS 343/543 Comparative Languages. My name is Saverio Perugini and I am the course instructor.

Required textbook: [EOPL2] (e-book free through library; used heavily for 3 of 7 modules; about 70% of course).

Course policies:

  • no cell phones or laptops
  • honor code; there is no group work in this course; undroppable, doubly-weighted zero penalty

Rather than following standard procedure and discussing course description and objectives, I prefer to discuss my approach to this course and what I expect from you. Perhaps at the end of this lecture we will discuss course objectives. We will also discuss these (again) at the end of course so you can reflect on where you have been and what you have learned.

  • This is not a required course. If you are taking this course only to fulfill a breadth requirement and you do not have a scientific curiosity or passion for languages, I recommend against taking the course as this will not be a fun experience for you.

  • This is an extremely challenging course and the workload is tremendous (because understanding the philosophy, psychology, and organization of PLs, like anything good in life, does not comes easy). It takes work (see evaluation table). There are intense programming assignments in the course; you will build interpreters for full-blown programming languages in this course.

    Exams require a significant amount of critical thinking; you will be asked to solve problems which you have never seen in class. What I am really trying to teach and evaluate in this course is how well you can apply what you have learned/studied in a completely new domain. This is really the hallmark of a smart person --- how well one can think independently on one's own.

    This course is widely considered the most-difficult CPS course. However, it is also considered one of the most exciting and eclectic.

  • There is nothing you are going to learn in this course that is going to help you get a job. This course is not learn C# in 21 days or ASP.NET for Dummies. However, this course can be your claim to fame. A guy named Paul Graham built a product called Viaweb based on the ideas presented in this course and sold it to Yahoo! for several million dollars. It became the Yahoo! store.

    The best analogy I can make is: this course is like learning Latin; it is not spoken anywhere, but most smart people know it and it will help you become stronger in your own language as well as better able to learn new languages with ease.

  • I not telling you any of this to scare you. I am telling you this so you know exactly what you are getting yourself into and so there are no misconceptions about the course and my expectations of you. The situation I am trying to avoid at all costs (which generally happens every year) is a student takes this course with no interest in languages and hacks their way through the first 2-3 assignments with Cs or Ds. Then that student finds thyself in mid-February when it is too late to drop the course and when all the assignments start to build on each other, and the student does not have a clue how to use LISP and has no interest in learning it. This is a recipe for disaster. Assignments 2-8 build on each other, and I do not release solutions.

  • So then why take the course? Because the course is awesome! I love teaching this course because programming languages ise my area of research and I am eager to share my expertise with you. This course is for the student who enjoys being challenged to look at the world/computing in radically different ways. It is for the student who enjoys exploring new ways of thinking and new systems which support those views.

    I integrate aspects of philosophy, psychology, and computer science in this course. I am going to expose you to a very different view of computation (we are going to write programs in languages which do not have variables!). My goal is to topple your whole view of your field, examine the pieces, and put them back together for you in a different way. Like learning anything, in the beginning you often learn things the wrong way just to get started (e.g., basketball analogy). I am not saying what you learned in CPS 151 and CPS 350 is not important. Nay, those topics are very important or CPS 350 would not be a pre-requisite for this course. However, what I am saying is that in this course we are going to look at those topics from different perspectives (e.g., see conduits diagram below). And in that sense this course will be a microcosm of your entire CPS curriculum; it will tie several loose ends together to give you a more holistic view of your discipline. I hope there will be several epiphanies for you in this course. And I hope you are as mystified as I was the first time I learned this material. This course has the potential to be the most exciting and interesting course you have ever taken.

    Our field is constantly evolving, and at a pace much more rapid than other fields. We constantly have to re-invent ourselves. If you have a solid foundation, you can do that with ease. My goal is to give you that strong, holistic foundation in this course. However, if your foundation is brittle, you will find adaptation to be a challenge.

    `If you give someone fish, he can eat for a day. If you teach someone to fish, he can eat for a lifetime.'

The Little Lisper, now in the 4th ed. called The Little Schemer.

Q&A protocol: the goal is to have no quizzes the entire semester.

So why explore new ways of thinking?

Scientific discoveries are usually not incremental. This is especially important for startups. If you are doing the same thing as the next guy, you both are going to be crushed by Microsoft.

In psychology it is widely believed that one's capacity to think is limited by the language in which one communicates one's thoughts.

`I can't think genetic algorithms in C++.'

Mergesort non-recursively anyone?

Because languages influence thought and capacity for thought, programming languages are (perhaps, so far) your only conduit into the field of computing, which encompasses a lot more than just languages. Thus, the study of languages can envelop the entire field of computing.

Fundamental questions:

  • what is a language?
  • what is a program?
  • what is a programming language?

This is a course on PL concepts.

What is a PL concept?

A PL concept usually has options (e.g., automobiles analogy).

Static vs. dynamic bindings in languages (e.g., the times of our life analogy, birth, life, and so on).

Our approach to learning PL concepts is to implement the concepts in real languages using LISP (in this case, Scheme).

We are going to demonstrate how to add recursion to a language which does not support recursion.
We are going to study how to add threads to a language which does not support multiple threads of control.

Why Scheme?

`If you give someone Fortran, he has Fortran. If you give someone LISP, he has any language he pleases' (afterword to [TSS]).

Before we do this we have to learn the fundamentals of languages. We have to build a vocabulary for comparative study (we will spend about a month on this). We will start heavily studying PL concepts and building interpreters in the third module of the course, though we will study some PL concepts (and even implement some) in the first two modules of the course.

UD house diagram.

Along the way we will study PL paradigms which are the CS analogs to families of natural languages. If you think in one and write in another, you are doing what I do when I speak Italian. The languages we are going to study, especially LISP and Smalltalk, encompass the foundational blocks for the concepts we study.

Course objectives.

Course themes. These will recur often. I will ask you often of which themes in-class examples are an instance.

Advantages to this course of study.

What influence language design?

Languages evolve. Based on what?

Oil painting example from student:

``If this looks like I just barfed code across half a dozen files, it's probably because I did. Which is why I chose Python, as Python is a great language for just throwing down pseudocode without getting everything exactly planned out right ahead of time.''

Throwaway prototypes (e.g., interviewing example --- use it to figure out what we are looking for in a candidate).

There is shift in modern times toward more dynamic languages. Why?

Writing generates thoughts (e.g., grocery list).

Assignment: Write a 2,000 word essay answering the following questions:

  • Why do you want to take this class?
  • What excites you about programming languages?
  • What compelled or excited you about the ideas presented in the first lecture? What do you agree/disagree with?
  • What are your goals for this course?
  • What questions do you have?
Your essay will be graded on the independence, thoughtfulness, and creativity in your responses to these questions.


Overview of lecture

This is a course in programming language concepts.

The approach involves studying language concepts, such as scope and parameter passing, by implementing a series of interpreters in Scheme, for purposes of its combined simplicity and power, and assessing the differences in the resulting languages.

Students can also expect a comparative survey of programming language paradigms and the use of representative languages, such as Haskell, PROLOG, and Smalltalk.

What is a PL paradigm?

What are the four main PL paradigms?

Purity in language paradigms.

    Is Java a pure OO language?

    Is C++ a pure OO language?

    Is Scheme a pure functional language?

Evolution of PL's ... blurred distinctions

We need to develop a vocabulary for comparative study:

    What is a program, language, programming language, PL concept?

    What is a static/dynamic property?

    What is a side effect? (expressions vs. statements)

    What is referential transparency?

    What is a first-class entity?

    Are objects first-class in C++? Java?

    Are functions first-class in functional languages?

Language evaluation: readability, writability, reliability, cost

Course themes:

    relationship between languages and the capacity to express ideas about computation

    static (rigid and fast) vs. dynamic (flexible and slow) properties

    cost of execution vs. cost of development

    speed of execution vs. speed of development

    simple, small, consistent, and powerful languages (LISP, Smalltalk) vs. complex, large, irregular, and weak languages (so called kitchen-sick languages (e.g., C++))

    languages evolve: the influence of language design and implementation options on current trends in programming practice and vice versa (iPod, GPS in car)

    shift in modern times towards more functional, dynamic languages (speed of execution is less important as a design goal as it once was)

Why bother?

    improve background for choosing appropriate languages and learning new languages and technologies

    some things cannot be expressed as easily in certain languages (e.g., déjà vu in English, courtesy Charlie Suer, Fall 2010)

    increase capacity to express ideas about computation (thinking out of the box)

    to become a well-rounded computer scientist and a better programmer, in any language

This class is for the student who enjoys problem solving and exploring new ways of thinking and new systems which supports those views.


Introduction to CPS 343/543: Comparative Languages

  • logistics, evaluation, and policies
  • misconceptions or `what this course is not about'
    • you will not learn <insert hottest language of the month here> in this course
    • you will not learn a language to help you get a job (but this course can be your claim to fame and fortune (e.g., Paul Graham))
    • `If you give someone fish, he can eat for a day. If you teach someone to fish, he can eat for a lifetime.'
  • this is a class in programming language concepts (and comparing and contrasting languages according to broad concepts); we will develop a vocabulary for comparative study
  • this class is for the student who enjoys problem solving and exploring completely new systems and ways of thinking
  • I am going to ask you to study and program in languages which are well beyond anything you have ever seen
  • why? relationship between languages and the capacity to express ideas about computation; `it is widely believed that the depth at which we can think is influenced by the expressive power of the language [though] which we communicate our thoughts' [COPL6]
  • like taking a course in Latin
  • programming team example (`I can't think genetic algorithms in C++')


Some fundamental questions

  • what is a language? a medium of communication
  • what is a program? a set of instructions which a computer understands and follows
  • what is a programming language? a notational system for describing computation in human-readable and machine-readable form
    • is HTML a programming language?
    • `a programming language is Turing complete provided it has integer variables and arithmetic and sequentially executes statements, which include assignment, selection (if), and loop (while) statements' [PLPP] p. 13
    • or alternatively, `a programming language is Turing complete if it has integer values, arithmetic functions on those values, and if it has a mechanism for defining new functions using existing functions, selection, and recursion' [PLPP] p. 17
  • what is a programming language concept?
    • parameter passing (by-value, by-reference)
    • typing (static or dynamic)
    • support for abstraction (procedural, data)
    • scope (static or dynamic)
    • interpreted or compiled?
    • as you can see, often has options
  • on which criteria can we evaluate programming languages?
    • four main criteria (desiderata for computer languages)
      • readability
      • writability
      • reliability (safety)
      • cost (efficiency) of execution or development or both?
    • how are readability and writability related?
    • others?


Static vs. dynamic bindings

  • a static binding happens before run-time (i.e., usually at compile-time) (e.g., type of x)
  • a dynamic binding happens (and is changable) during run-time (e.g., the value of x)


Automobile analogs

  • make: Honda or Toyota
  • engine: gas or diesel
  • transmission: manual or automatic
  • steering: manual or power
  • braking system: manual or power
  • options: A/C or no-A/C


Approach

  • study language concepts by implementing a series of interpreters in Scheme (otherwise know as operational semantics) and assess the differences in the resulting languages
  • why Scheme (a functional language with imperative features)? for purposes of its elegance and power
    • ideally situated between formal mathematics and natural language (preface to [TSS])
    • `If you give someone Fortran, he has Fortran. If you give someone LISP, he has any language he pleases' (afterward to [TSS])
    • it is the most powerful programming language
    • LISP is the second oldest programming language (which is the oldest?)
    • it is a metalanguage: a language for creating languages [BITL]


Programming language paradigms

  • What is a paradigm? a model or world view (contributed by historian of science Thomas Kuhn) [SICP]
  • What is a model? a simplified view of something in the real world
  • What is a programming language paradigm? a pattern for a programming language to follow [PLPP]
  • paradigms
    • imperative
      • `any programming language characterized by
        • sequential execution
        • variables representing memory
        • use of assignment to change the values of variables
        is said to be an imperative programming language, since its primary method of describing computation is through a sequence of commands, or imperatives' [PLPP] p. 13
      • examples: C, FORTRAN, Pascal, Algol
    • functional
      • brings programming closer to mathematics; in functional programming languages, functions are first-class entities (see below)
      • based on λ-calculus: a mathematical theory of functions on which all functional languages are based
      • largely without variables, assignment, and iteration
      • purity: no side-effects (not even for I/O), no variables, no assignment statement (see below)
      • it is possible to program without variables and assignments statements
      • examples: LISP, Scheme, ML, Haskell (pure)
    • declarative/logic
      • describe what you want, not how to get it
      • rule-based
      • support for reasoning about facts and rules
      • rules specified in no particular order
      • examples: PROLOG, CLIPS, POP
      • can you think of another example? how about SQL
        • an SQL query describes what data is desired, not how to answer it
        • travel agents write queries
        • grandma can write a query, but grandma is not going to write a program
        • reason why DBMS's are the runaway success of the software industry
        • usually declarative implies inefficient (several layers of abstraction) (e.g., PROLOG)
        • SQL is declarative and efficient ... take CPS 430!
        • impedance mismatch between database systems and programming languages
          • is also a language paradigm mismatch
          • need interfaces like JDBC
    • object-oriented
      • support for data modeling and abstraction
      • message passing; an object-oriented system is one constructed as a collection of objects passing messages to each other
      • purity: everything is an object, even primitives
      • examples: C++, Java, Smalltalk (pure)
    • others? scripting
      • command- and control-oriented
      • tend to be multi-paradigmed, especially Python (imperative, functional, and object-oriented)
      • examples: sh, AWK, CLOS, Perl, Python, PHP, R, REXX, Ruby, and Tcl/Tk (and earliest example: IBM's JCL)

  • analogies
    • C is to the imperative paradigm, as Spanish is to romance languages (essentially all have the same grammar, only differ in the terminals)
    • C is to the imperative paradigm, as English is to ?
    • C is to the imperative paradigm, as English is to Latin or Greek ?
    • C is to the imperative paradigm, as Hindi is to Sanskrit ?
  • declarative languages are sometimes called very-high-level languages ([PLPP] p. 18 and [SICP] p. 22)
  • will implement imperative languages using Scheme, a functional language
  • object-orientation evolved from the functional paradigm (see below)
  • we will also program in Haskell, PROLOG, and Smalltalk
  • where do functional and logic languages motivate from?
    • LISP machine predecessor to modern single-user workstation as well as important modern technologies such as garbage collection, high-resolution graphics, and computer mice, among others
    • Warren Abstract Machine (or WAM) is a target for PROLOG compilers
  • historically,
    • imperative languages are more static => compiled
    • functional and logic languages are more dynamic => interpreted
  • when to use which language in which paradigm? application-driven? other ideas?


Definitions of terms used above

  • syntax (form of language) vs. semantics (meaning of language)
  • first-class entity
  • side-effect


Why take variables and assignment away from the programmer?

  • seems stoic, but there must be an advantage
  • no assignment in mathematics
  • variable modification through assignment accounts for a large volume of bugs in programs
  • without such facilities we might be able to write less buggy code
  • other reasons? parallelization


First-class entity

  • notion due to British computer scientist Christopher Strachey (1916-1975) ([SICP] p. 76)
  • see [EOPL2] p. 56 or [PLP] p. 143
  • a first-class entity is one that can be
    • stored in a variable (or data structure),
    • passed as an argument, and
    • returned as a value
  • second-class: only passed as an argument
  • third-class: cannot even be passed as an argument
  • are objects first-class in C++? yes. Java? depends how you look at it
  • are closures first-class in Scheme? yes
  • alternate perspective ([PLPP] p. 337): the ability to create new values at run-time


Side effect

  • see [COPL6] p. 299
  • modification of a parameter or something in the external environment (e.g., a global variable or I/O)
  • assignment statement possible?
  • for instance, X = read(); print X + X; vs. print read() + read();
  • a + fun(a) [COPL6] p. 299


Referential transparency

  • expressions and languages are said to be referentially transparent (i.e., independent of evaluation order [PLP] p. 622) if the same arguments to a function yield the same output
  • see [COPL6] p. 583
  • alternate viewpoint (called substitution rule with mathematical expressions): `any two expressions in a program that have the same value may be substituted for each other anywhere in the program -- in other words, their values always remain equal regardless of the context in which they are evaluated' [PLPP] p. 268
  • related to side-effects, but different
  • are all functions without side-effects referentially transparent? no, consider a function which returns time() (courtesy Matthew Hufford)
  • are all referentially transparent functions without side-effects? no, consider a function which prints "hello world"


What influences language design?

  • why did programming languages evolve as they did?
  • computer architecture (e.g., von Neumann architecture gives rise to imperative languages ([PLPP] p. 13-14))
    • memory stores both instructions and data
    • fetch-decode-execute cycle
    • I/O (between processor and memory) is the biggest bottleneck of any computer architecture
    • von Neumann bottleneck: computation need not described as a sequence of instructions operating on a single piece of data. For instace, what about
      • parallel computation,
      • nondeterministic computation, or
      • recursion? [PLPP].
  • programming methodology: top-down design, more data-driven, less procedure-driven, object-oriented
  • surprisingly, not the abilities of programmers
  • read The Psychology of Computer Programming by Gerald M. Weinberg


Course objectives

  • Establish an understanding of fundamental programming language concepts primarily through the implementation of interpreters.
  • Improve your ability to understand new languages and technologies, and expand your background for selecting appropriate languages.
  • Expose students to alternate styles/paradigms of programming and exotic ways of affecting computation so to paint a more holistic picture of computing.


Course themes

  • relationship between languages and the capacity to express ideas about computation
  • static (rigid and fast) vs. dynamic (flexible and slow) properties
  • cost/speed of execution vs. cost/speed of development
  • simple, small, consistent, and powerful languages (LISP, Smalltalk) vs. complex, large, irregular, and weak languages (so called kitchen-sink languages [PLPP] (e.g., C++))
  • languages evolve: the influence of language design and implementation options on current trends in programming practice and vice versa (iPod, GPS in car)
  • shift in modern times towards more functional, dynamic languages (speed of execution is less important as a design goal as it once was)


Advantages of this course of study

(or `why bother with this stuff anyway?')


  • see [COPL6] §1.1 (pp. 2-5)
  • improve background for choosing appropriate languages
  • ability to easily learn new languages and technologies as they arrive (ability to focus on the big picture and not the details)
  • some things cannot be expressed as easily in certain languages (e.g., déjà vu in English, courtesy Charlie Suer, Fall 2010)
  • increase capacity to express ideas about computation (thinking out of the box)
  • why these languages?
  • increase ability to design and implement new languages
  • to better appreciate the historical context
  • to become a well-rounded computer scientist
  • in summary, this course will make you a better programmer, in any language


What will you learn?

  • concepts of programming languages through the implementation of interpreters (i.e., operational semantics)
  • language definition and description methods (e.g., grammars)
  • how to implement interpreters and implementation strategies (e.g., inductive data types, data abstraction and representation, design patterns)
  • programming paradigms and how to program using representative languages in each paradigm (e.g., Scheme, Haskell, ML, PROLOG, Smalltalk)
  • esoteric language concepts (e.g., continuations, currying, lazy evaluation)


Scheme example

  • a closure is a function created by a program at run-time which refers to free variables in its lexical context
  • (code, environment)
  • (behavior, state)
  • ((lambda (x) (+ x 1)) 1)
    
    (define increment (lambda (x) (+ x 1)))
    
    (increment 3)
    
    (define make-adder (lambda (x) (lambda (y) (+ x y))))
    
    (define add8 (make-adder 8))
    
    (define make-counter (lambda ()
       (let ((c 0))
          (lambda ()
             (set! c (+ c 1))
             c))))
    


Smalltalk demo

    Developed by Alan Kay and colleagues at Xerox PARC in the 1970s
    Intended to teach programming to children.

    Squeak:
    Transcript open.
    
    Transcript show: 'hello'
    
    3 + 4 7
    
    1 to: 5 do: [:i | (Transcript show: 'Hello World') cr]
    
    prompt _ FillInTheBlank request: 'How may I help you, Linus?'
    
    N _ 6
    
    N class.
    
    <class name> selectors
    
    <class name> superclass.
    <class name> allSubclasses.
    
    <class name> withAllSubclasses.
    <class name> withAllSuperclasses.
    
    N decrement 6
    
    N _ Dictionary new.
    
    N inspect. "reflection"
    
    N at: '4:30' put: 'cps343'
    
    N at: '5:55' put: 'cps445'
    
    N _ 90.
    
    N inspect. 
    
    "from [QTOL] p. 140"
    Object withAllSubclasses do: 
    
        [:clas | clas selectors do:
    	
             [:sel | 
    	         (sel asLowercase includesSubString: 'random')
    	 
    	          ifTrue: [
    		            Transcript show: clas name.
    		            Transcript show: ' '.
    		            Transcript show: sel.
    		            Transcript cr]]].
    
    Morph new openInWorld.
    


References

    [BITL] G. J. Sussman, G.L. Steele, and R.P. Gabriel. A Brief Introduction to Lisp. ACM SIGPLAN Notices, 28(3), 361-362, 1993.
    [COPL6] R.W. Sebesta. Concepts of Programming Languages. Addison-Wesley, Boston, MA, Sixth edition, 2003.
    [EOPL2] D.P. Friedman, M. Wand, and C.T. Haynes. Essentials of Programming Languages. MIT Press, Cambridge, MA, Second edition, 2001.
    [PLP] M.L. Scott. Programming Language Pragmatics. Morgan Kaufmann, Amsterdam, Second edition, 2006.
    [PLPP] K.C. Louden. Programming Languages: Principles and Practice. Brooks/Cole, Pacific Grove, CA, Second edition, 2002.
    [QTOL] G. Korienek, T. Wrensch, D. Dechow. Squeak: A Quick Trip To ObjectLand. Addison-Wesley, Boston, MA, 2002.
    [SICP] H. Abelson and G.J. Sussman. Structure and Interpretation of Computer Programs. MIT Press, Cambridge, MA, Second edition, 1996.
    [TSS] D.P. Friedman and M. Felleisen. The Seasoned Schemer. MIT Press, Cambridge, MA, 1996.

Return Home