chapter 7 n.
Skip this Video
Download Presentation
Chapter 7

Loading in 2 Seconds...

play fullscreen
1 / 74

Chapter 7 - PowerPoint PPT Presentation

  • Uploaded on

Chapter 7. Scaling up from one client at a time. All server code in the book, up to now, dealt with one client at a time. Except our last chatroom homework. Options for scaling up: event driven: See chatroom example. problem is its restriction to a single CPU or core multiple threads

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Chapter 7' - brock-snider

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
chapter 7
Chapter 7


scaling up from one client at a time
net_pyScaling up from one client at a time
  • All server code in the book, up to now, dealt with one client at a time.
  • Except our last chatroom homework.
  • Options for scaling up:
    • event driven: See chatroom example. problem is its restriction to a single CPU or core
    • multiple threads
    • multiple processes (in Python, this really exercises all CPUs or cores)
load balancing i
net_pyLoad Balancing I
  • Prior to your server code via DNS round-robin:

; zone file fragment

ftp IN A

ftp IN A

ftp IN A

www IN A

www IN A

; or use this format which gives exactly the same result

ftp IN A



www IN A


load balancing ii
net_pyLoad Balancing II
  • Have your own machine front an array of machines with the same service on each and forward service requests in a round-robin fashion.
daemons and logging
net_pyDaemons and Logging:
  • “Daemon” means the program is isolated from the terminal in which it was executed. So if the terminal is killed the program continues to live.
  • The Python program supervisord does a good job in this isolation process and in addition offers the following services:
    • starts and monitors services
    • re-starts a service that terminates and stops doing so if the service terminates several times in a short period of time.
  • supervisord sends stdout and stderr output to a log file system that cycles through log, log.1, log.2, log.3 and log.4.
logging continued
net_pyLogging continued:
  • Better solution is to import your own logging module and save things to a log in that way.
  • logging has the benefit of writing to what you want - files, tcp/ip connection, printer, whatever.
  • It can also be customized from a configuration file called logging.conf by using the logging.fileConfig() method.

import logging

log = logging.getLoger(__name__)

log.error('This is a mistake')

sir launcelot
net_pySir Launcelot:
  • The following is an importable module:

#!/usr/bin/env python

# Foundations of Python Network Programming - Chapter 7 -

# Constants and routines for supporting a certain network conversation.

import socket, sys

PORT = 1060

qa = (('What is your name?', 'My name is Sir Lancelot of Camelot.'),

('What is your quest?', 'To seek the Holy Grail.'),

('What is your favorite color?', 'Blue.'))

qadict = dict(qa)

def recv_until(sock, suffix):

message = ''

while not message.endswith(suffix):

data = sock.recv(4096)

if not data:

raise EOFError('socket closed before we saw %r' % suffix)

message += data

return message

sir launcelot ii
net_pySir Launcelot II:
  • The following is part of an importable module:

def setup():

if len(sys.argv) != 2:

print >>sys.stderr, 'usage: %s interface' % sys.argv[0]


interface = sys.argv[1]

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

sock.bind((interface, PORT))


print 'Ready and listening at %r port %d' % (interface, PORT)

return sock

server simple py

import lancelot

def handle_client(client_sock):


while True:

question = lancelot.recv_until(client_sock, '?')

answer = lancelot.qadict[question]


except EOFError:


def server_loop(listen_sock):

while True:

client_sock, sockname = listen_sock.accept()


if __name__ == '__main__':

listen_sock = lancelot.setup()


  • The server has two nested infinite loops – one iterating over different client/server exchanges and one iterating over the individual client/server exchange until the client terminates.
  • The server is very inefficient; it can only server one client at a time.
  • If too many clients try to attach the connection queue will fill up and prospective clients will be dropped. Hence the #WHS will not even begin; let alone complete.
elementary client
net_pyElementary Client:
  • This client asks each of the available questions once and only once and then disconnects.

#!/usr/bin/env python

import socket, sys, lancelot

def client(hostname, port):

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

s.connect((hostname, port))


answer1 = lancelot.recv_until(s, '.') # answers end with '.'


answer2 = lancelot.recv_until(s, '.')


answer3 = lancelot.recv_until(s, '.')


print answer1

print answer2

print answer3

elementary client ii
net_pyElementary Client II:
  • The rest
  • It seems fast but is it really?
  • To test this for real we need some realistic network latency so shouldn't use localhost.
  • We also need to measure microsecond behaviour.

if __name__ == '__main__':

if not 2 <= len(sys.argv) <= 3:

print >>sys.stderr, 'usage: hostname [port]'


port = int(sys.argv[2]) if len(sys.argv) > 2 else lancelot.PORT

client(sys.argv[1], port)

tunnel to another machine
net_pyTunnel to another machine:

ssh -L

  • See page 289 for this feature
  • Alternatively, here is agood explanation of various possible scenarios


more on sshd port forwarding
net_pyMore on SSHD Port Forwarding:
  • Uses:
    • access a backend database that is only visible on the local subnet
    • your ISP gives you a shell account but expects emails to be sent from their browser mail client to their server
    • reverse port forwarding


ssh -L

ssh -L

ssh -R 8022:localhost:22 username@my.home.ip.address

ssh -p 8022 username@localhost

waiting for things to happen
net_pyWaiting for Things to Happen:
  • So now we have traffic that takes some time to actually move around.
  • We need to time things.
  • If your function, say foo(), is in a file called then the script called will time the running of foo() from
my experiment
net_pyMy Experiment
  • Set up VPN from my home so I have a New Paltz IP address
  • Use as my remote machine
  • Have both server and client run on my laptop

[pletcha@archimedes 07]$ ssh -L 1061: joyous

[pletcha@archimedes 07]$ python handle_client ''

python client localhost 1061

my trace py

#!/usr/bin/env python

# Foundations of Python Network Programming - Chapter 7 -

# Command-line tool for tracing a single function in a program.

import linecache, sys, time

def make_tracer(funcname):

def mytrace(frame, event, arg):

if frame.f_code.co_name == funcname:

if event == 'line':

_events.append((time.time(), frame.f_code.co_filename,


return mytrace

return mytrace

my trace py1

if __name__ == '__main__':

_events = []

if len(sys.argv) < 3:

print >>sys.stderr, 'usage: funcname ...'



del sys.argv[0:2] # show the script only its own name and arguments




for t, filename, lineno in _events:

s = linecache.getline(filename, lineno)

sys.stdout.write('%9.6f %s' % (t % 60.0, s))

my output
net_pyMy Output:

43.308772 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

Δ = 83 μs

43.308855 s.connect((hostname, port))

Δ = 644 μs

43.309499 s.sendall([0][0])

Δ = 41 μs

43.309540 answer1 = lancelot.recv_until(s, '.') # answers end with '.'

Δ = 241 ms

43.523284 while True:

Δ = 8 μs

43.523292 question = lancelot.recv_until(client_sock, '?')

Δ = 149 ms

43.672060 answer = lancelot.qadict[question]

Δ = 9 μs

43.672069 client_sock.sendall(answer)

Δ = 55 μs

43.672124 while True:

Δ = 4 μs

43.672128 question = lancelot.recv_until(client_sock, '?')

Δ = 72 ms

43.744381 s.sendall([1][0])


this iteration

my output1
net_pyMy Output:

43.744381 s.sendall([1][0])

Δ = 80 μs

43.744461 answer2 = lancelot.recv_until(s, '.')

Δ = 133 ms

43.877629 answer = lancelot.qadict[question]

Δ = 10 μs

43.877639 client_sock.sendall(answer)

Δ = 63 μs

43.877702 while True:

Δ = 6 μs

43.877708 question = lancelot.recv_until(client_sock, '?')

Δ = 55 ms

43.932345 s.sendall([2][0])

Δ = 86 μs

43.932431 answer3 = lancelot.recv_until(s, '.')

Δ = 149 ms

44.081574 answer = lancelot.qadict[question]

my output2
net_pyMy Output:

44.081574 answer = lancelot.qadict[question]

Δ = 8 μs

44.081582 client_sock.sendall(answer)

Δ = 47 μs

44.081629 while True:

Δ = 4 μs

44.081633 question = lancelot.recv_until(client_sock, '?')

Δ = 59 ms

44.140687 s.close()

Δ = 88 μs

44.140775 print answer1

Δ = 61 μs

44.140836 print answer2

Δ = 20 μs

44.140856 print answer3

Δ = 146 ms

44.287308 except EOFError:

Δ = 11 μs

44.287317 client_sock.close()

  • Server finds the answer in 10 microseconds (answer =) so could theoretically answer 100000 questions per second.
  • Each sendall() takes ~60 microseconds while each recv_until() takes ~60 milliseconds (1000 times slower).
  • Since receiving takes so long we can't process more than 16 questions per second with this iterative server.
  • The OS helps where it can. Notice that sendall() is 1000 times faster than recv_until(). This is because the sendall() function doesn't actually block until data is sent and ACKed. It returns as soon as the data is delivered to the TCP layer. The OS takes care of guaranteeing delivery.
  • 219 milliseconds between moment when client executes connect() and server executes recv_all(). If all client requests were coming from the same process, sequentially this means we could not expect more than 4 sessions per second.
  • All the time the server is capable of answering 33000 sessions per second.
  • So, communication and most of all, sequentiality really slow things down.
  • So much server time not utilized means there has to be a better way.
  • 15-20 milliseconds for one question to be answered so roughly 40-50 questions per second. Can we do better than this by increasing the number of clients?
  • See page 289 for ssh -L feature
  • Funkload: A benchmarking tool that is written in python and lets you run more and more copies of something you are testing to see how things struggle with the increased load.
test routine
net_pyTest Routine:
  • Asks 10 questions instead of 3

#!/usr/bin/env python

from funkload.FunkLoadTestCase import FunkLoadTestCase

import socket, os, unittest, lancelot

SERVER_HOST = os.environ.get('LAUNCELOT_SERVER', 'localhost')

class TestLancelot(FunkLoadTestCase): # python syntax for sub-class

def test_dialog(self): # In Java & C++, receiver objects are implicit; # in python they are explicit (self == this.

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sock.connect((SERVER_HOST, lancelot.PORT))

for i in range(10):

question, answer =[i % len(]


reply = lancelot.recv_until(sock, '.')

self.assertEqual(reply, answer)


if __name__ == '__main__':


environment variables
net_pyEnvironment Variables:
  • You can set a variable from the command line using SET and make sure it is inherited by all processes run from that command line in the future by EXPORTING it.
  • The authors explain why they are using environment variables - “I can not see any way of to pass actual arguments through to tests via Funkload command line arguments”
  • Run on separate machine.

(BenchMark)[pletcha@archimedes BenchMark]$


config file on laptop
net_pyConfig file on laptop:

# TestLauncelot.conf: <test class name>.conf


title=Load Test For Chapter 7

description=From the Foundations of Python Network Programming

url=http://localhost:1060/ # overridden by environment variable import


log_path = ftest.log

result_path = ftest.xml

sleep_time_min = 0

sleep_time_max = 0


log_to = file

log_path = bench.log

result_path = bench.xml

cycles = 1:2:3:5:7:10:13:16:20

duration = 8

startup_delay = 0.1

sleep_time = 0.01

cycle_time = 10

sleep_time_min = 0

sleep_time_max = 0

testing funkload
net_pyTesting Funkload:
  • Big mixup on Lancelot-Launcelot.

(BenchMark)[pletcha@archimedes BenchMark]$

fl-run-test TestLancelot.test_dialog



Ran 1 test in 0.010s


benchmark run
net_pyBenchmark run:
  • Typical cycle output

Cycle #7 with 16 virtual users


* setUpCycle hook: ... done.

* Current time: 2013-04-11T13:46:34.536010

* Starting threads: ................ done.

* Logging for 8s (until 2013-04-11T13:46:44.187746): .

........................... done.

* Waiting end of threads: ................ done.

  • Since we are sending 10 questions per connection (test) we are answering 1320 questions per second.
  • We greatly outdid the original 16 questions per second in the sequential test example.
  • Adding more than 3 or 4 clients really didn't help.
  • Remember we still only have a single-threaded server. The reason for the improvement is that clients can be “pipelined” with several clients getting something done at the same time.
  • The only thing that can't be in parallel is answering the question.

# Clients


# Questions



# Ques/client













  • Adding clients drags down performance
  • Insurmountable problem: Server is talking to only one client at a time.
  • Adding clients drags down performance
  • Insurmountable problem: Server is talking to only one client at a time.
event driven servers
net_pyEvent-driven Servers:
  • The simple server blocks until data arrives. At that point it can be efficient.
  • What would happen if we never called recv() unless we knew data was already waiting?
  • Meanwhile we could be watching a whole array of connected clients to see which one has sent us something to respond to.
event driven servers1
net_pyEvent-driven Servers:

#!/usr/bin/env python

# Foundations of Python Network Programming - Chapter 7 -

# An event-driven approach to serving several clients with poll().

import lancelot

import select

listen_sock = lancelot.setup()

sockets = { listen_sock.fileno(): listen_sock }

requests = {}

responses = {}

poll = select.poll()

poll.register(listen_sock, select.POLLIN)

event driven servers2
net_pyEvent-driven Servers:

while True:

for fd, event in poll.poll():

sock = sockets[fd]

# Removed closed sockets from our list.

if event & (select.POLLHUP | select.POLLERR | select.POLLNVAL):


del sockets[fd]

requests.pop(sock, None)

responses.pop(sock, None)

# Accept connections from new sockets.

elif sock is listen_sock:

newsock, sockname = sock.accept()


fd = newsock.fileno()

sockets[fd] = newsock

poll.register(fd, select.POLLIN)

requests[newsock] = ''

event driven servers3
net_pyEvent-driven Servers:

# Collect incoming data until it forms a question.

elif event & select.POLLIN:

data = sock.recv(4096)

if not data: # end-of-file

sock.close() # makes POLLNVAL happen next time


requests[sock] += data

if '?' in requests[sock]:

question = requests.pop(sock)

answer = dict([question]

poll.modify(sock, select.POLLOUT)

responses[sock] = answer

# Send out pieces of each reply until they are all sent.

elif event & select.POLLOUT:

response = responses.pop(sock)

n = sock.send(response)

if n < len(response):

responses[sock] = response[n:]


poll.modify(sock, select.POLLIN)

requests[sock] = ''

event driven servers4
net_pyEvent-driven Servers:
  • The main loop callspoll(), which blocks until something/ anything is ready.
  • The difference is recv() waited for a single client and poll() waits on all clients.
  • In the simple server we had one of everything. In this polling server we have an array of everything; one of each thing dedicated to each connection.
  • How poll() works: We tell it what sockets to monitor and what activity we are interested in on each socket – read or write.
  • When one or more sockets are ready with something, poll() returns.
event driven servers5
net_pyEvent-driven Servers:
  • The life-span of one client:

1: A client connects and the listening socket is “ready”. poll() returns and

since it is the listening socket, it must be a completed 3WHS. We accept()

the connection and tell our poll() function we want to read from this connection.

To make sure they never block we set blocking “not allowed”.

2: When data is available, poll() returns and we read a string and append the

string to a dictionary entry for this connection.

3: We know we have an entire question when '?' arrives. At that point we ask

poll() to write to the same connection.

4: Once the socket is ready for writing (poll() has returned) we send as much of

we can of the answer and keep sending until we have sent '.'.

5: Next we swap the client socket back to listening-for-new-data mode.

6: POLLHUP, POLLERR and POLLNOVAL events occur on send() so when recv()

receives 0 bytes we do a send() to get the error on our next poll().

server poll py benchmark benchmark:
  • benchmark
  • So we see some performance degradation.


we got errors
net_pyWe got Errors
  • Some connections ended in errors – check out listen().
  • TCP man page:


Listen for connections made to the socket. The backlog argument specifies the

maximum number of queued connections and should be at least 0; the maximum value

is system-dependent (usually 5), the minimum value is forced to 0.

tcp_max_syn_backlog (integer; default: see below; since Linux 2.2)

The maximum number of queued connection requests which have

still not received an acknowledgement from the connecting

client. If this number is exceeded, the kernel will begin

dropping requests. The default value of 256 is increased to

1024 when the memory present in the system is adequate or

greater (>= 128Mb), and reduced to 128 for those systems with

very low memory (<= 32Mb). It is recommended that if this

needs to be increased above 1024, TCP_SYNQ_HSIZE in

include/net/tcp.h be modified to keep

TCP_SYNQ_HSIZE*16<=tcp_max_syn_backlog, and the kernel be


poll vs select
net_pyPoll vs Select
  • poll() code is cleaner but select(), which does the same thing, is available on Windows.
  • The author's suggestion: Don't write this kind of code; use an event-driven framework instead.
non blocking semantics
net_pyNon-blocking Semantics
  • In non-blocking mode, recv() acts as follows:
    • If data is ready, it is returned
    • If no data has arrived, socket.error is raised
    • if the connection is closed, '' is returned.
  • Why does closed return data and no data return an error?
  • Think about the blocking situation.
    • First and last can happen and behave as above. Second situation won't happen.
    • The second situation had to do something different.
non blocking semantics1
net_pyNon-blocking Semantics:
  • send() semantics:
    • if data is sent, its length is returned
    • socket buffers full: socket.error raised
    • connection closed: socket.error raised
  • Last case is interesting. Suppose poll() says a socket is ready to write but before we call send(), the client sends a FIN. Listing 7-7 doesn't code for this situation.
event driven servers6
net_pyEvent-driven Servers:
  • They are “blocking” and are “synchronous”.
  • poll(), afterall, blocks. This is not essential. You can have poll()timeout and go through a housekeeping loop. Such a server is not entirely event-driven.
  • Since the server reads a question or sends back a reply as soon as it is ready, it is not really asynchronous.
  • “Non-blocking” here, means on a particular client.
  • This is the only alternative to a “busy-loop”, since such a program would grab 100% of CPU. Instead we go into the blocking-IO state of the scheduling state diagram.
  • Author says, “leave the term asynchronous” for hardware interrupts, signals, etc.
alternatives to figure 7 7
net_pyAlternatives to Figure 7-7
  • Twisted Python is a framework for implementing event -driven servers.

#!/usr/bin/env python

from twisted.internet.protocol import Protocol, ServerFactory

from twisted.internet import reactor

import lancelot

class Lancelot(Protocol):

def connectionMade(self): # onConnect()

self.question = ''

def dataReceived(self, data): # onDataReceived

self.question += data

if self.question.endswith('?'):


self.question = ''

factory = ServerFactory()

factory.protocol = Lancelot

reactor.listenTCP(1060, factory)

  • Rewrite (Listing 7-8) to handle the situation in which the client does not wait for the answer to a question before sending out a new question.
  • Rewrite the TestLauncelot.test_dialog() method (Listing 7-5) to send out all questions before it processes any of the answers.
  • Run the funkload test with the same configuration file.
  • In order to do this you will need to set up your own virtual environment and install both funkload (page 106) and twisted.

mkvirtualenv BenchMark

cd to project directory

mkproject BenchMark

workon BenchMark

cp Chapter07code/* .

pip install funkload

pip install twisted

more on twisted python
net_pyMore on Twisted Python
  • Twisted Python handles all the polling, etc of our Listing 7-7.
  • In addition, Twisted can work with actual services that take some time to execute.
  • Hence Twisted can deal with services that require more than the 10 microseconds it takes the program to look up the answer to the question in the dictionary.
  • It does so by allowing you to register deferred methods. These are like callbacks (or callback chain, actually) that are registered and fire off as a separate thread when data is received. They have to terminate with a “send reply” event
load balancing and proxies
net_pyLoad Balancing and Proxies:
  • Our server in Listing 7-7 is a single thread of control. So is Twisted (except for its deferred methods).
  • Single threaded approach has a clear upper limit – 100% CPU time.
  • Solution – run several instances of server (on different machines) and distribute clients among them.
  • Requires a load balancer that runs on the server port and distributes client requests to different server instances.
  • It is thus a proxy since to the clients it looks like the server and to any server instance it looks like a client.
load balancers
net_pyLoad Balancers:
  • Some are built into network hardware.
  • HAProxy is a TCP/HTTP load balancer.
  • Firewall rules on Linux let you simulate a load balancer.
  • Traditional method is to use DNS – one domain name with several IP addresses.
  • Problem with DNS solution is that once assigned a server, the client is stuck if server goes down.
  • Modern load balancers can recover from a server crash by moving live connections to different server instance.
  • DNS still used for reasons of geography.

In Canada, you'll get

load balancers1
net_pyLoad Balancers:
  • Authors feel it is the only approach to server saturation that scales up.
  • Intended to distribute work across different physical machines but could be used to distribute work among server instances on the same physical machine.
  • Threading and forking are special cases of load balancing. If you have several instances of the same file descriptor all executing accept() then the OS will load balance among them.
  • Load balancing is an alternative to writing multi-threaded code.

Open a listening port and then fork the program or create a thread that

shares the same listening port, then all such file descriptors can

execute accept() and wait for a 3WHS completion.

threading and multi processing
net_pyThreading and Multi-processing:
  • Take a simple, one-client-at-a-time server and run several instances of it spawned from same process.
  • Our event-driven example is responsible for deciding what client is ready for service. With threading, the OS does the same work. Each server connection is blocking on recv() and send() but using very few server resources while doing so.
  • The OS wakes them up (removes them from IOBlocking state) when traffic arrives.
  • Apache offers two solutions – prefork and worker.
  • prefork: forks off several instances of httpd
  • worker: runs multiple threads of control in a single httpd instance.
worker example
net_pyWorker Example:

[pletcha@archimedes 07]$ cat

#!/usr/bin/env python

# Foundations of Python Network Programming - Chapter 7 -

# Using multiple threads or processes to serve several clients in parallel.

import sys, time, lancelot

from multiprocessing import Process

from server_simple import server_loop # actual server code

from threading import Thread

WORKER_CLASSES = {'thread': Thread, 'process': Process}


def start_worker(Worker, listen_sock):

worker = Worker(target=server_loop, args=(listen_sock,))

worker.daemon = True # exit when the main process does

worker.start() # thread or process starts running server_loop

return worker

worker example1
net_pyWorker Example:

if __name__ == '__main__':

if len(sys.argv) != 3 or sys.argv[2] not in WORKER_CLASSES:

print >>sys.stderr, 'usage: interface thread|process'


Worker = WORKER_CLASSES[sys.argv.pop()] # setup() wants len(argv)==2

# Every worker will accept() forever on the same listening socket.

listen_sock = lancelot.setup()

workers = []

for i in range(WORKER_MAX):

workers.append(start_worker(Worker, listen_sock))

worker example2
net_pyWorker Example:

# Check every two seconds for dead workers, and replace them.

while True:


for worker in workers:

if not worker.is_alive():

print, "died; starting replacement worker"


workers.append(start_worker(Worker, listen_sock))

  • Now we have multiple instances of the simple server loop from Listing 7-2.
  • The OS is allowing different threads of execution (threads or processes) to listen on the same socket.
  • Our thanks to POSIX.
  • Let's run this server against our benchmark software.
server simple py1

import lancelot

def handle_client(client_sock):


while True:

question = lancelot.recv_until(client_sock, '?')

answer = lancelot.qadict[question]


except EOFError:


def server_loop(listen_sock):

while True:

client_sock, sockname = listen_sock.accept()


if __name__ == '__main__':

listen_sock = lancelot.setup()


ps ef
net_pyps -ef:
  • 10 data connections and 1 “watcher”

[pletcha@joyous ~]$ ps -ef | grep 'python [s]'

pletcha 14587 29926 0 08:31 pts/2 00:00:00 python process

pletcha 14588 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14589 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14590 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14591 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14592 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14593 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14594 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14595 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14596 14587 0 08:31 pts/2 00:00:05 python process

pletcha 14597 14587 0 08:31 pts/2 00:00:05 python process

all have same parent

  • Please note that standard C Python restricts greatly the parallelism to our “threads” since all run inside a Python environment.
  • Other implementations of Python (Jython or IronPython) handle this better (better locking of shared Python data structures).
  • You need to experiment on the right number of server children you should run. Issues include number of server cores, speed of clients, speed of server RAM bus can affect optimum number of servers.



handling multiple related processes
net_pyHandling multiple, related processes:
  • How do we kill things so all processes die?
  • multiprocessing cleans up your workers if you kill the parent with Ctrl-C or exit normally.
  • but if you kill the parent then the child processes will become orphaned and you will have to kill them too, individually.
killing processes
net_pyKilling processes:

[pletcha@joyous ~]$ ps -ef | grep 'python [s]'

pletcha 15107 29926 0 08:54 pts/2 00:00:00 python process

pletcha 15108 15107 0 08:54 pts/2 00:00:05 python process

pletcha 15109 15107 0 08:54 pts/2 00:00:05 python process

pletcha 15110 15107 0 08:54 pts/2 00:00:05 python process

pletcha 15111 15107 0 08:54 pts/2 00:00:05 python process

[pletcha@joyous ~]$ kill 15107

[pletcha@joyous ~]$ ps -ef | grep 'python [s]'

pletcha 15108 1 0 08:54 pts/2 00:00:05 python process

pletcha 15109 1 0 08:54 pts/2 00:00:05 python process

pletcha 15110 1 0 08:54 pts/2 00:00:05 python process

pletcha 15111 1 0 08:54 pts/2 00:00:05 python process

[pletcha@joyous ~]$ ps -ef | grep 'python [s]'|awk '{print $2}'





[pletcha@joyous ~]$ kill $(ps -ef | grep 'python [s]'|awk '{print $2}')

threading and mulit process frameworks
net_pyThreading and Mulit-process Frameworks:
  • Let somebody else do the work.
  • Worker pools (either threads or processes)
  • Module multiprocessing has a Pool object for distributing work across multiple child processes with all work distributed from a master thread after accept() returns vs single listener socket.
  • Module SocketServer in standard Python library follows latter approach.

ssock.accept() and

process data


ssock.accept() and

process data


ssock.accept() and

process data




NOTE: When a process fork()s it spawns a child copy of itself with copies of

all its current data variables and values; ie, an exact copy of all the same

[code, globals, heap, stack] memory configuration. Hence, if you start a server

process, then create a socket and listen() on it, then fork() repeatedly and each

time, after forking, call accept() in the child/duplicated process, then you succeed

in creating many copies of the same socket identified by the same file descriptor;

all of them accepting on the same server-side socket.












s = socket.socket(...)



for i in range(10):

if fork():

# still in this process



# fork() returns child process

# id in parent process and 0 in

# child process



spawn child upon

successful accept()


child data


single listener














s = socket.socket(...)



while True:

conn = s.accept()

if fork():

# still in this process



# fork() returns child process

# id in parent process and 0 in

# child process


listing 7 10
net_pyListing 7-10

#!/usr/bin/env python

# Foundations of Python Network Programming - Chapter 7 -

# Answering Lancelot requests with a SocketServer.

from SocketServer import ThreadingMixIn, TCPServer, BaseRequestHandler

import lancelot, server_simple, socket

class MyHandler(BaseRequestHandler):

def handle(self):


class MyServer(ThreadingMixIn, TCPServer):

allow_reuse_address = 1

# address_family = socket.AF_INET6 # if you need IPv6

server = MyServer(('', lancelot.PORT), MyHandler)


  • Issues include the slowness of the child in starting up since data can't be processed until after process is created, etc.
  • What if a burst of connections arrives and the server spawns a large number of children. Host could run out of resources.
  • multiprocessing, on the other hand, caps the number of servers and a burst of incoming will just have to get in line.
  • Figure 7-5 shows the benchmark on Listing 7-10. It is less successful than multiprocessing (650 tests/second compared to 750 tests/second when working for 15 clients)
process and thread coordination
net_pyProcess and Thread Coordination:
  • Assumption, so far, is that threads or processes serving one data connection need not share data with threads or processes serving a different data connection.
  • So for example, if you are connecting to the same database from the different worker threads, you are using a thread-safe database API and problems won't arise if the different workers are sharing the same resource (writing to the same log file, for example).
  • If this is not so, then you need to be skilled in concurrent programming.
  • Our author's advise:
in memory sharing

ssock.accept() and

process data


In-memory Sharing:



NOTE: Sharing data through synchronized queues (threads) or

via IPC queues (processes)

database sharing

ssock.accept() and

process data


Database Sharing:



NOTE: Sharing data through 3rd-party data store such as database or

memcached (see next chapter).

process and thread coordination1
net_pyProcess and Thread Coordination:
  • 1: make sure you understand the difference between threads and processes. Threads continue to share after creation but processes only share initially and then can diverge. Threads are “convenient” while processes are “safe”.
  • 2: Use high-level data structures provided by python. For example, there exist thread-safe queue data structures and special queues designed to move data across processes.
  • 3: Limit shared data.
  • 4: Check out the Standard Library and Packages index for something created by others.
  • 5: Follow the path taken by web programmers where shared data is kept by the database.
  • Oldtimer. Saves on daemons. Just one daemon listening to several well-known ports simultaneously.
  • Written with select() or poll().
  • Programs to call upon accept() found in /etc/inetd.conf.
  • Problem: How to give each new process the correct file descriptor of the successfully connected client socket?
  • Answer: Duplicate your newly connected socket as stdin and stdout. Then pass these to the new processes ready to handle the connection using redirection. Finally, write your connection service routine for stdin and stdout only. It won't even know it is connected to a remote client.
event driven servers7
net_pyEvent-driven Servers:
  • See page 289 for this feature