Text Processing in Python

Utilizing Higher-Order Functions in Text Processing.
Exercise: More on combinatorial functions.
Specializing Python Datatypes.
Base Classes for Datatypes.
Exercise: Filling out the forms (or deciding not to)
Problem: Working with lines from a large file.

Working with the Python Interpreter.
Working with the Local Filesystem.
Running External Commands and Accessing OS Features.
Special Data Values and Formats.

Serializing and Storing Python Objects.
Platform Specific Operations.
Working with Multi-Media Formats.
Miscellaneous Other Modules.

Problem: Quickly sorting lines on custom criteria.
Problem: Reformatting paragraphs of text.
Problem: Column statistics for delimited or flat-record files.
Problem: Counting characters, words, lines and paragraphs.
Problem: Transmit binary data as ASCII.
Problem: Creating word or letter histograms.
Problem: Reading a file backwards by record, line, or paragraph.

Basic String Transformations.
Strings as Files, and Files as Strings.
Converting Between Binary and ASCII.
Cryptography.
Compression.
Unicode.

Exercise: Many ways to take out the garbage.
Exercise: Making sure things are what they should be
Exercise: Finding needles in haystacks (full-text indexing).

Just What is a Regular Expression Anyway?
Matching Patterns In Text: The Basics.
Matching Patterns In Text: Intermediate.
Advanced Regular Expression Extensions.

Problem: Making a text block flush left.
Problem: Summarizing command-line option documentation.
Problem: Detecting duplicate words.
Problem: Checking for server errors.
Problem: Reading lines with continuation characters
Problem: Identifying URLs and email addresses in texts.
Problem: Pretty printing numbers.

Versions and optimizations.
Simple Pattern Matching.
Regular Expression Modules.

When data becomes deep and texts become stateful.
What is a grammar?
An EBNF grammar for IF/THEN/END structures.
Pencil-and-Paper Parsing.
Exercise: Some variations on the language.

Understanding State Machines.
Text Processing State Machines.
When Not To Use A State Machine.
When to Use a State Machine.
An Abstract State Machine Class.
Processing a Report with a Concrete State Machine.
Subgraphs and State Reuse.
Exercise: Finding other solutions.

Specialized Parsers in the Standard Library.
Low-Level State Machine Parsing.
High-Level EBNF Parsing.
High-Level Programmatic Parsing.

Manipulating and Creating Message Texts.
Communicating with Mail Servers.
Message Collections and Message Parts.

Common Gateway Interface.
Parsing, Creating, and Manipulating HTML Documents.
Accessing Internet Resources.

Standard Internet-Related Tools.
Third Party Internet-Related Tools.

Python Standard Library XML Modules.
Third Party XML-Related Tools