Foundations of Python 3 Network Programming, Second Edition

【实例截图】

【核心代码】

Contents
■
■Contents at a Glance ............................................................................................ iv?
■ ■About the Authors ............................................................................................... xv?
■ ■About the Technical Reviewer ............................................................................. xv?
■ ■Acknowledgments .............................................................................................. xvi?
■ ■Introduction ....................................................................................................... xvii
■ ■Chapter 1: Introduction to Client/Server Networking ............................................ 1?
The Building Blocks: Stacks and Libraries ...................................................................... 1?
Application Layers ........................................................................................................... 4?
Speaking a Protocol ........................................................................................................ 5?
A Raw Network Conversation ......................................................................................... 6?
Turtles All the Way Down ................................................................................................ 8?
The Internet Protocol ....................................................................................................... 9?
IP Addresses ................................................................................................................. 10?
Routing .......................................................................................................................... 11?
Packet Fragmentation ................................................................................................... 13?
Learning More About IP ................................................................................................. 14?
■ ■Chapter 2: UDP .................................................................................................... 15?
Should You Read This Chapter? .................................................................................... 16?
Addresses and Port Numbers ....................................................................................... 16?
Port Number Ranges ..................................................................................................... 17?
Sockets ......................................................................................................................... 19?
■ CONTENTS
vi
Unreliability, Backoff, Blocking, Timeouts .................................................................... 22?
Connecting UDP Sockets ............................................................................................... 25?
Request IDs: A Good Idea .............................................................................................. 27?
Binding to Interfaces ..................................................................................................... 28?
UDP Fragmentation ....................................................................................................... 30?
Socket Options .............................................................................................................. 31?
Broadcast ...................................................................................................................... 32?
When to Use UDP .......................................................................................................... 33?
Summary ....................................................................................................................... 34?
■ ■Chapter 3: TCP ..................................................................................................... 35?
How TCP Works ............................................................................................................. 35?
When to Use TCP ........................................................................................................... 36?
What TCP Sockets Mean ............................................................................................... 37?
A Simple TCP Client and Server .................................................................................... 38?
One Socket per Conversation ........................................................................................ 41?
Address Already in Use ................................................................................................. 42?
Binding to Interfaces ..................................................................................................... 43?
Deadlock ....................................................................................................................... 44?
Closed Connections, Half-Open Connections ................................................................ 48?
Using TCP Streams like Files ........................................................................................ 49?
Summary ....................................................................................................................... 49?
■ ■Chapter 4: Socket Names and DNS ..................................................................... 51?
Hostnames and Domain Names .................................................................................... 51?
Socket Names ............................................................................................................... 52?
Five Socket Coordinates ............................................................................................... 53?
IPv6 ............................................................................................................................... 54?
Modern Address Resolution .......................................................................................... 55?
 ■ CONTENTS
vii
Asking getaddrinfo() Where to Bind .............................................................................. 56?
Asking getaddrinfo() About Services ............................................................................. 56?
Asking getaddrinfo() for Pretty Hostnames ................................................................... 57?
Other getaddrinfo() Flags .............................................................................................. 58?
Primitive Name Service Routines .................................................................................. 59?
Using getsockaddr() in Your Own Code ......................................................................... 60?
Better Living Through Paranoia .................................................................................... 61?
A Sketch of How DNS Works ......................................................................................... 63?
Why Not to Use DNS ...................................................................................................... 65?
Why to Use DNS ............................................................................................................ 66?
Resolving Mail Domains ................................................................................................ 68?
Zeroconf and Dynamic DNS .......................................................................................... 70?
Summary ....................................................................................................................... 70?
■ ■Chapter 5: etwork Data and Network Errors ....................................................... 71?
Text and Encodings ....................................................................................................... 71?
Network Byte Order ...................................................................................................... 73?
Framing and Quoting .................................................................................................... 75?
Pickles and Self-Delimiting Formats ............................................................................. 79?
XML, JSON, Etc. ............................................................................................................ 80?
Compression ................................................................................................................. 81?
Network Exceptions ...................................................................................................... 82?
Handling Exceptions ..................................................................................................... 83?
Summary ....................................................................................................................... 85?
■ ■Chapter 6: TLS and SSL ....................................................................................... 87?
Computer Security ........................................................................................................ 87?
IP Access Rules ............................................................................................................. 88?
Cleartext on the Network .............................................................................................. 90?
■ CONTENTS
viii
TLS Encrypts Your Conversations ................................................................................. 92?
TLS Verifies Identities ................................................................................................... 93?
Supporting TLS in Python .............................................................................................. 94?
The Standard SSL Module ............................................................................................. 95?
Loose Ends .................................................................................................................... 98?
Summary ....................................................................................................................... 98?
■ ■Chapter 7: Server Architecture ........................................................................... 99?
Daemons and Logging .................................................................................................. 99?
Our Example: Sir Launcelot ......................................................................................... 100?
An Elementary Client ................................................................................................... 102?
The Waiting Game ....................................................................................................... 103?
Running a Benchmark ................................................................................................. 106?
Event-Driven Servers .................................................................................................. 109?
Poll vs. Select .............................................................................................................. 112?
The Semantics of Non-blocking .................................................................................. 113?
Event-Driven Servers Are Blocking and Synchronous ................................................ 114?
Twisted Python ........................................................................................................... 114?
Load Balancing and Proxies ........................................................................................ 117?
Threading and Multi-processing ................................................................................. 117?
Threading and Multi-processing Frameworks ............................................................ 120?
Process and Thread Coordination ............................................................................... 122?
Running Inside inetd ................................................................................................... 123?
Summary ..................................................................................................................... 124?
■ ■Chapter 8: Caches, Message Queues, and Map-Reduce ................................... 125?
Using Memcached ...................................................................................................... 126?
Memcached and Sharding .......................................................................................... 128?
Message Queues ......................................................................................................... 130?
 ■ CONTENTS
ix
Using Message Queues from Python .......................................................................... 131?
How Message Queues Change Programming ............................................................. 133?
Map-Reduce ................................................................................................................ 134?
Summary ..................................................................................................................... 136?
■ ■Chapter 9: HTTP ................................................................................................. 137?
URL Anatomy ............................................................................................................... 138?
Relative URLs .............................................................................................................. 141?
Instrumenting urllib2 ................................................................................................... 141?
The GET Method .......................................................................................................... 142?
The Host Header ......................................................................................................... 144?
Codes, Errors, and Redirection ................................................................................... 144?
Payloads and Persistent Connections ......................................................................... 147?
POST And Forms ......................................................................................................... 148?
Successful Form POSTs Should Always Redirect ....................................................... 150?
POST And APIs ............................................................................................................ 151?
REST And More HTTP Methods ................................................................................... 151?
Identifying User Agents and Web Servers ................................................................... 152?
Content Type Negotiation ............................................................................................ 153?
Compression ............................................................................................................... 154?
HTTP Caching .............................................................................................................. 155?
The HEAD Method ....................................................................................................... 156?
HTTPS Encryption ........................................................................................................ 156?
HTTP Authentication .................................................................................................... 157?
Cookies ....................................................................................................................... 158?
HTTP Session Hijacking .............................................................................................. 160?
Cross-Site Scripting Attacks ....................................................................................... 160?
WebOb ......................................................................................................................... 161?
■ CONTENTS
x
Summary ..................................................................................................................... 161
■ ■Chapter 10: Screen Scraping . .......................................................................... 163?
Fetching Web Pages ................................................................................................... 163?
Downloading Pages Through Form Submission ......................................................... 164?
The Structure of Web Pages ....................................................................................... 167?
Three Axes . ................................................................................................................ 168?
Diving into an HTML Document .................................................................................. 169?
Selectors . ................................................................................................................... 173?
Summary ..................................................................................................................... 177?
■ ■Chapter 11: Web Applications .......................................................................... 179?
Web Servers and Python ............................................................................................ 180?
Two Tiers . .................................................................................................................. 180?
Choosing a Web Server .............................................................................................. 182?
WSGI . .......................................................................................................................... 183?
WSGI Middleware ....................................................................................................... 185?
Python Web Frameworks ............................................................................................ 187?
URL Dispatch Techniques ........................................................................................... 189?
Templates ................................................................................................................... 190?
Final Considerations ................................................................................................... 191?
Pure-Python Web Servers ........................................................................................... 192?
CGI . ............................................................................................................................. 193?
mod_python ................................................................................................................ 194?
Summary ..................................................................................................................... 195?
■ ■Chapter 12: E-mail Composition and Decoding ................................................ 197?
E-mail Messages ........................................................................................................ 198?
Composing Traditional Messages ............................................................................... 200?
Parsing Traditional Messages .................................................................................... 202?
Download from www.eBookTM.Com
 ■ CONTENTS
xi
Parsing Dates .............................................................................................................. 203?
Understanding MIME ................................................................................................... 205?
How MIME Works ........................................................................................................ 206?
Composing MIME Attachments ................................................................................... 206?
MIME Alternative Parts ................................................................................................ 208?
Composing Non-English Headers ............................................................................... 210?
Composing Nested Multiparts ..................................................................................... 211?
Parsing MIME Messages ............................................................................................. 213?
Decoding Headers ....................................................................................................... 215?
Summary ..................................................................................................................... 216?
■ ■Chapter 13: SMTP .............................................................................................. 217?
E-mail Clients, Webmail Services ............................................................................... 217?
In the Beginning Was the Command Line .......................................................................................... 218 ?
The Rise of Clients ............................................................................................................................. 218 ?
The Move to Webmail ......................................................................................................................... 220 ?
How SMTP Is Used ...................................................................................................... 221?
Sending E-Mail ............................................................................................................ 221?
Headers and the Envelope Recipient .......................................................................... 222?
Multiple Hops .............................................................................................................. 223?
Introducing the SMTP Library ..................................................................................... 224?
Error Handling and Conversation Debugging .............................................................. 225?
Getting Information from EHLO ................................................................................... 228?
Using Secure Sockets Layer and Transport Layer Security ........................................ 230?
Authenticated SMTP .................................................................................................... 232?
SMTP Tips ................................................................................................................... 233?
Summary ..................................................................................................................... 234?
■ ■Chapter 14: POP ................................................................................................ 235?
Compatibility Between POP Servers ........................................................................... 235?
■ CONTENTS
xii
Connecting and Authenticating ................................................................................... 235?
Obtaining Mailbox Information .................................................................................... 238?
Downloading and Deleting Messages ......................................................................... 239?
Summary ..................................................................................................................... 241?
■ ■Chapter 15: IMAP .............................................................................................. 243?
Understanding IMAP in Python .................................................................................... 244?
IMAPClient ................................................................................................................... 246?
Examining Folders ...................................................................................................... 248?
Message Numbers vs. UIDs ........................................................................................ 248?
Message Ranges ......................................................................................................... 249?
Summary Information ................................................................................................. 249?
Downloading an Entire Mailbox .................................................................................. 250?
Downloading Messages Individually ........................................................................... 252?
Flagging and Deleting Messages ................................................................................ 257?
Deleting Messages ...................................................................................................... 258?
Searching .................................................................................................................... 259?
Manipulating Folders and Messages .......................................................................... 260?
Asynchrony ................................................................................................................. 261?
Summary ..................................................................................................................... 261?
■ ■Chapter 16: Telnet and SSH ............................................................................... 263?
Command-Line Automation ........................................................................................ 263?
Command-Line Expansion and Quoting ...................................................................... 265?
Unix Has No Special Characters .................................................................................. 266?
Quoting Characters for Protection ............................................................................... 268?
The Terrible Windows Command Line ........................................................................ 269?
Things Are Different in a Terminal .............................................................................. 270?
Terminals Do Buffering ............................................................................................... 273?
 ■ CONTENTS
xiii
Telnet .......................................................................................................................... 274?
SSH: The Secure Shell ................................................................................................ 278?
An Overview of SSH .................................................................................................... 279?
SSH Host Keys ............................................................................................................ 280?
SSH Authentication ..................................................................................................... 282?
Shell Sessions and Individual Commands .................................................................. 283?
SFTP: File Transfer Over SSH ...................................................................................... 286?
Other Features ............................................................................................................ 289?
Summary ..................................................................................................................... 290?
■ ■Chapter 17: FTP ................................................................................................. 291?
What to Use Instead of FTP ......................................................................................... 291?
Communication Channels ........................................................................................... 292?
Using FTP in Python .................................................................................................... 293?
ASCII and Binary Files ................................................................................................. 294?
Advanced Binary Downloading ................................................................................... 295?
Uploading Data ............................................................................................................ 297?
Advanced Binary Uploading ........................................................................................ 298?
Handling Errors ........................................................................................................... 299?
Detecting Directories and Recursive Download .......................................................... 301?
Creating Directories, Deleting Things ......................................................................... 302?
Doing FTP Securely ..................................................................................................... 303?
Summary ..................................................................................................................... 303?
■ ■Chapter 18: RPC ................................................................................................ 305?
Features of RPC .......................................................................................................... 306?
XML-RPC ..................................................................................................................... 307?
JSON-RPC ................................................................................................................... 313?
Self-documenting Data ............................................................................................... 315?
■ CONTENTS
xiv
Talking About Objects: Pyro and RPyC ........................................................................ 316?
An RPyC Example ........................................................................................................ 317?
RPC, Web Frameworks, Message Queues .................................................................. 319?
Recovering From Network Errors ................................................................................ 320?
Binary Options: Thrift and Protocol Buffers ................................................................. 320?
Summary ..................................................................................................................... 321
■ ■Index ................................................................................................................. 323