Python
http://markmeretzky.com/python/
mark.meretzky@gmail.com

Sections

1. DHSS-GA.1120 possibly at NYU, 2022.
2. SF19PB1 Python Bootcamp, Summer/Fall 2019.
3. WS19PB02 Python Bootcamp, Spring 2019.
4. INFO1-CE9990 Introduction to Python Programming, Summer 2017 Section 2.

Bibliography

For the current Python 3.10.6 (and the forthcoming 3.11)

1. Python Tutorial.
2. Python Language Reference. Includes control structure keywords such as while, for, and if.
3. Python Standard Library. Includes
   1. built-in functions such as print, input, and round
   2. modules such as sys, math, and random
   3. data types such as
      1. Numeric types: int, float
      2. Sequence types: str, bytes, list, tuple, range
      3. Mapping types: dict
      4. Set types: set
4. Python Glossary, for example EAFP vs. LBYL.
5. the HOWTOs
6. IDLE, the Integrated Development Environment.
7. Learning Python, 5th ed., by Mark Lutz. You don’t have to buy this book.

The course

1. Prologue. A computer is a machine that follows a list of instructions. The list of instructions is called a program. In the future, it will be possible to write programs in English. But that day hasn’t come yet, so for the time being we write programs in simpler languages. One of these simpler languages is Python. This will be a course in writing programs in Python. A short, informal program is often called a script. Test for students who already have Python experience. Teaser.
2. Install Python 3.8.0 and IDLE on macOS, Microsoft Windows, and Amazon AWS Linux. IDLE is the IDE (Integerated Develpment Environment) that comes with Python.
   1. Definition of IDLE in the Python Glossary.
   2. Wikipedia page about IDLE.
   3. Documentation about IDLE.
   4. Eric IDLE.
3. Expressions in the language Python. Play with them using IDLE.
   1. Operators, operands, and variables. The dot operator.
   2. Operator precedence and associativity; overriding them with parentheses.
   3. Function calls, arguments, and return value.
   4. Data types: int, float, string, bool. Conversion between the data types.
4. GitHub
   1. Create a GitHub account.
   2. Create a GitHub organization in your GitHub account.
   3. Create a GitHub repository in your GitHub organization.
5. Standard i/o and variables.
   1. Standard output with the print function and its keyword arguments sep and end. (In Python 2, print was a statement, not a function.) Docstrings, pydoc3.
   2. Exit status: an invisible number returned by the sys.exit in your Python script to the operating system.
   3. Variables and assignment statements. We are about to need a variable to hold the standard input.
   4. Standard input with the input function (was raw_input in Python 2). Raise and catch an exception when something goes wrong with input or type conversion. Indentation. EAFP vs. LBYL.
6. Execute a Python script.
   1. on macOS:
      1. in IDLE: pull down Run → Run Module
      2. in the macOS Finder: drag the Python script onto Applications/Python 3.7/Python Launcher.app
      3. in the macOS Finder: set the script’s Open With.
      4. in the Terminal window: execute python3 from the command line and feed it the script.
         python3 myscript.py
      5. in the Terminal window: make the script executable using the PATH environment variable, #!/usr/bin/env, and chmod,
   2. on Microsoft Windows:
      1. in IDLE: pull down Run → Run Module
      2. in Windows Explorer: take advantage of the “file association” between python.exe and any file whose name ends with .py
      3. in the Command Prompt window: add one directory to the PATH environment variable
   3. Redirect the script’s standard i/o with >, 2>, <, and | (pipe). Standard output and standard error output can be directed to two different destinations.
7. Control structure. By default, each instruction of a script is executed exactly once, and they are executed in the order in which they are written. The control structure statements can cause the statements to be executed in some other order.
   1. while loops, the break statement, infinite loops, nested loops
   2. if/elif/else statements (conditionals)
   3. for loops and the range function as a more localized alternative to while loops. The return value of range is an iterable.
   4. Loops containing if statements:
      1. Validate: keep looping until the user provides valid input.
      2. Guess: a simple guessing game.
      3. Teaser.
      4. Monte Carlo method for computing the value of π.
      5. Converge until we’re close to the value of π.
      6. tkinter graphics, implemented with the Tk from Tcl/Tk.
      7. turtle graphics.
8. Strings of characters. The str data type is an example of a sequence type. See sequence.
   1. Indexing (subscripting), slicing, and the len built-in function. Immutability.
   2. Characters: loop through the characters of a string with a for loop and the enumerate built-in function. A string and the return value of enumerate are iterable.
   3. Search a string with in, not in, index, and a regular expression.
   4. Split a string.
   5. Pig Latin. Translate English to Pig Latin. The operators in, and, or, not. The string methods isupper, isalpha, lower, capitalize.
   6. One big string. Read a text file into one big string.
   7. One big sequence of bytes. Read a text file from the Internet into one big sequence of bytes. Then convert the sequence of bytes into one big string.
9. Other things to loop through with a for loop.
   1. Lines: loop through the lines of an input file or pipe with a for loop. rstrip the newline character and other whitespace.
   2. URL: loop through the lines of a file downloaded from the web. decode each incoming line from a sequence of bytes into a more familiar string of characters.
   3. Binary file: we can input a binary file, but it has no lines to loop through because it does not consist of lines of text. We can, however, loop through the rows and columns of pixels in a binary file that is an image file.
10. Lists. The list data type, like the str data type, is an example of a sequence type.
    1. Basics.
       1. List of numbers. Mutability.
       2. List of strings. Access the individual characters of each string.
       3. Loop through the items of a list with while, for, range, and enumerate. A list is iterable.
       4. zip function for looping through two lists in parallel
    2. Alternative: a list as an alternative for a long chain of if/elif/else statements. Search a list with index.
    3. Barchart: a tkinter graphic of the data in a list.
    4. Sort the items of a list with the functions sort and sorted. Custom sort orders by specifying a key function. Pass a lambda (i.e., nameless) function to another function. Make a shallow copy of a list. Search a list with index.
    5. Max: one more example of a lambda function.
    6. Create a list by splitting a string into a list of strings. Loop through the words on a line.
    7. Create a list by reading the lines of a text file into a list of strings. We can split the file into lines as we input it, or input the file into one huge string and split it up later.
    8. List comprehension: create a copy of an existing list, usually with modifications. See List Comprehensions.
    9. Date: an object of class datetime.date holds a year, month, and day.
    10. CSV: comma-separated values.
    11. CSV database: a database of comma-separated values. Search the New York City Restaurant Inspection results. append to a list.
    12. Street trees. Search a database and display what you find on a Google map.
    13. Pandas: data analysis and scientific computing. See pandas and NumPy.
        pip3 install numpy
        pip3 list
        pip3 show numpy
    14. Summary of the ways to create a list.
    15. Insert, remove, and delete list items. See del in tutorial. Might be simpler to create a new list instead of performing surgery on an old one.
    16. Higher order functions (i.e., functions whose arguments are functions) for looping through a list:
        1. map and itertools.starmap
        2. filter
        3. functtools.reduce and the operator functions operator.add, operator.mul, operator.concat
11. A two-dimensional list is implemented in Python as a list of lists.
    1. Heterogeneous: a list that contains a list, i.e., a list one of whose items is a list. Followed by a list all of whose items are lists.
    2. Tic-tac-toe board.
    3. Plot generator has a list of lists. The random.choice function.
    4. Roman: convert an integer to and from Roman numerals.
    5. Manhattan address algorithm has a list of lists. append to a list. Add a tkinter GUI with buttons, drop-down menus, etc.
    6. JSON: deserialize a JSON string into a Python list.
12. Tuples. A tuple is like an immutable list.
    1. Immutable.
13. Dictionaries. A dictionary is like a list with two parallel columns. The left column contains keys. The right column contains values. We look up a key in order to find the corresponding value. Each key can appear in the dictionary only once.
    1. Two parallel columns, the keys and the values.
    2. Numeric keys: never create a dictionary whose keys are consecutive integers. It’s simpler to create a list instead.
    3. Sparse: use a dictionary instead of a sparse list.
    4. Dictionary comprehension.
    5. Defaultdict is a dictionary whose initial values are created for you by default.
    6. Counter counts how many times an item appears.
    7. Environment variables such as PATH, etc.
    8. Attributes: a dictionary of attributes of different data types.
    9. JSON: deserialize a JSON string into a Python dictionary.
    10. Download JSON: download and parse JSON from OpenWeathermap, Google Maps, and GitHub.
    11. Download XML: download and parse XML from OpenWeathermap.
    12. Facebook: traverse the Facebook graph.
14. Sets. A set is like a dictionary with keys but with no corresponding values. Each key can appear in the set only once.
    1. Test for membership with the in operator.
    2. No duplicates: a set can contain no more than one copy of each item.
    3. Intersect: compute the intersection, union, and difference of two sets.
    4. Limitations on the types of objects that can be contained by a set.
    5. Miscellaneous compound data types: collections.namedtuple and range.
15. Functions defined by the user (i.e., created by you). Package a piece of code as a function and give it a convenient name.
    1. Define a function.
    2. Pass an argument to a function.
    3. Pass list: pass a list or tuple to a function.
    4. Keyword argument: pass an optional keyword argument to a function (like the sep and end arguments of the print function).
    5. Variable number: a function that will accept a variable number of positional arguments.
    6. Variable keyword: a function that will accept a variable number of positional arguments, followed by a variable number of keyword arguments.
    7. Count calls: a function that can count how many times it’s been called, by incrementing a global variable.
    8. Return tuple: a function that returns a tuple of items.
16. Modules for natural (i.e., human) language processing.
    1. Speech to text.
    2. Text to speech.
    3. Translation.
17. Static type checking.
    1. mypy
    2. Annotate a variable (including a function argument) and a function return value.
18. Classes and objects. An object is a value that contains smaller values inside of it, and that has functions that belong to it. The objects we create will include unit tests and iterables.
    1. Attributes: attach an attribute to an object. Call the hasattr, setattr, and getattr functions.
    2. Setattr: the setattr and getattr functions.
    3. __init__: provide a class with an __init__ method to initialize each instance of the class.
    4. Pass an object to a function.
    5. Class Date: create a class containing instance attributes and class attributes, instance methods and class methods. The class definition, the self argument, the __init__ and __str__ methods.
    6. Module: define a class in a separate module, and then import the module into your Python script.
       if __name__ == "__main__":
    7. Dir: list the attributes of an object, and use callable to find out which attributes are also methods.
    8. with statement and context managers.
    9. decimal module as an example of a context manager.
    10. Unit tests for the graph paper function.
    11. Unit tests for class Date.
    12. Life: J. H. Conway’s Game of Life.
    13. Tweepy: the Python interface to the Twitter API. Get the most recent tweets that mention Trump or DeBlasio. Install a Python package (for Twitter, Facebook, etc.) with pip3.
19. Iterators and generators
    1. Iterators: loop through a range of ints, a range of floats, and a range of dates.
    2. Generator: a generator function that returns a generator iterator. The generator function yields a range of floats.
    3. Generator expressions look like list comprehensions, but with (parentheses) instead of [square brackets].
    4. Getitem: apply an index to an object.
20. Inheritance: create a new class with a head start.
    1. Subclass: create a subclass of an existing class.
    2. Leapdate: create a smarter class Date that knows about leap years so it can pass the unit tests.
    3. Abstract base class.
    4. Multiple inheritance. Multiple Inheritance in the Python Tutorial.
21. SQLite: the light version of the Structured Query Language.
    1. Play with SQLite. Put tables into a database, put rows into a table. CRUD, join.
    2. Inspection: create, update, and search an SQLite database on your Mac or PC of restaurant inspection results.
22. Regular expressions for searching a string. This is how you win at Scrabble.
    1. Elementary regular expressions. The re.search function.
    2. Groups in a regular expression.
    3. Match objects tell you what you found.
    4. Substitute different text for what you found. The re.sub function.
    5. Compile a regular expression with the re.compile function.
23. Pandas
24. Recursion. A recursive function is one that calls itself. A recursive data structure is one that contains smaller data structures of the same shape (i.e., a list that contains smaller lists, that in turn contain even smaller lists).
    1. Recursion: a recursive function that prints the integers from 1 to 10.
    2. Quicksort: a recursive function that sorts a list.
    3. Subsets: print all the subsets of a set.
    4. Tree: print and search a simple recursive tree.
    5. Tree of processes: print the tree of currently running processes on macOS.
    6. Maze: find a path through a maze using recursion.
25. Miscellaneous.
    1. io.StringIO: hold a string in memory rather than in a file. Used by pdfminer.
    2. Selenium browser automation.
    3. CGI: the Common Gateway Interface. Connect an HTML form to a Python cgi script. Test it with Selenium.
    4. pdfminer: search a PDF file.
    5. Beautiful Soup HTML parser.
    6. dscl: directory service command line, plistlib, binascii.
26. Summary of the course, and of the distinguishing features of Python.
27. Next time I teach this course:
    1. Catch exception in function that raised it, vs. catch exception in the caller’s code.
    2. Multithreading. Multi-threading, Concurrent execution, threading and queue modules, global interpreter lock.
    3. Specify width for tkinter.OptionMenu as in OpenWeatherMap.
    4. Move ~ interpretation (introduced in Binary) up to first program (for loop)? that reads from a file in home directory (Lines).