Home arrow Python arrow Page 3 - Basic Threading in Python

Pooling Threads - Python

If you want your application to perform several tasks at once, you can use threads. Python can handle threads, but many developers find thread programming to be very tricky. Among other points, Peyton McCullough covers how to spawn and kill threads in this popular language.

  1. Basic Threading in Python
  2. Using the Threading Module
  3. Pooling Threads
  4. More Thread Tricks
By: Peyton McCullough
Rating: starstarstarstarstar / 157
April 04, 2005

print this article



It's important to remember that threads don't start up instantly. Creating too many of them can slow down your application. It takes time to spawn and later kill threads. Threads can also eat up valuable system resources in large applications. This problem is easily solved by creating a set number of threads (a thread pool) and assigning them new tasks, basically recycling them. Connections would be accepted and then pushed to a thread as soon as it finished with the previous client.

If you still don't understand, compare it to a doctor's office. Let's say that there are five doctors. These are our threads. Patients (clients)  walk into the office, and if the doctors are busy, they are seated in the waiting room.

Obviously, we'll need something that can transfer client data to our threads without running into problems (it will need to be “thread safe”). Python's Queue module does this for us. Client information is stored in a Queue object, where threads can pull them out when needed.

Let's recreate our server to take advantage of a pool of threads:

import pickle
import Queue
import socket
import threading

# We'll pickle a list of numbers, yet again:
someList = [ 1, 2, 7, 9, 0 ]
pickledList = pickle.dumps ( someList )

# A revised version of our thread class:
class ClientThread ( threading.Thread ):

   # Note that we do not override Thread's __init__ method.
   # The Queue module makes this not necessary.

   def run ( self ):

      # Have our thread serve "forever":
      while True:

         # Get a client out of the queue
         client = clientPool.get()

         # Check if we actually have an actual client in the client variable:
         if client != None:

            print 'Received connection:', client [ 1 ] [ 0 ]
            client [ 0 ].send ( pickledList )
            for x in xrange ( 10 ):
               print client [ 0 ].recv ( 1024 )
            client [ 0 ].close()
            print 'Closed connection:', client [ 1 ] [ 0 ]

# Create our Queue:
clientPool = Queue.Queue ( 0 )

# Start two threads:
for x in xrange ( 2 ):

# Set up the server:
server = socket.socket ( socket.AF_INET, socket.SOCK_STREAM )
server.bind ( ( '', 2727 ) )
server.listen ( 5 )

# Have the server serve "forever":
while True:
   clientPool.put ( server.accept() )

As you can see, it's a little more complex than our previous server, but it's not amazingly complicated. The client we created in the previous section can be used to test this server, just like the previous server.

>>> More Python Articles          >>> More By Peyton McCullough

blog comments powered by Disqus
escort Bursa Bursa escort Antalya eskort


- Python Big Data Company Gets DARPA Funding
- Python 32 Now Available
- Final Alpha for Python 3.2 is Released
- Python 3.1: String Formatting
- Python 3.1: Strings and Quotes
- Python 3.1: Programming Basics and Strings
- Tuples and Other Python Object Types
- The Dictionary Python Object Type
- String and List Python Object Types
- Introducing Python Object Types
- Mobile Programming using PyS60: Advanced UI ...
- Nested Functions in Python
- Python Parameters, Functions and Arguments
- Python Statements and Functions
- Statements and Iterators in Python

Developer Shed Affiliates


Dev Shed Tutorial Topics: