How many threads should I use?ΒΆ

The consequences of getting the wrong number of threads are:

Too few: your users will have to wait for a thread to be free. If it takes too long, their connection attempt will timeout.

Too many: you’ll waste resources (database connections, memory, CPU, etc.).

Think of concurrent users and the average time each page takes to load. Be extra careful with the most common / used pages, the most expensive operations, peak traffic and how many concurrent users you’ll be serving.

Example: 5 requests per second, an average of 50 ms per hit -> 1 thread would handle that. 5 requests per second, an average of 5s per hit -> 5*5 = 25s of work requested / second, so you have to calculate how long you want your user to wait. In 5 seconds, for the last example, you’d have 125s of work requested. If you want to make nobody wait more than 5s then you’d have to be able to deliver that amount of work. One thread per request, 25 threads (and a cache in front of TG, optimising the code to see if it can be sped up, etc.)

You also need to be aware that a single cherryPy instance can only use one processor for all of its threads even if you have multiple processors in a machine. You can get around this by starting multiple instances on different ports and put a load balancing proxy in front (lighttpd will do nicely). Also get lighttpd or whatever your front end is to handle the static content directly. This also helps with resilience and scalability.

If you have two processors (dual core is now common) you might want to start two instances with 10 threads rather than one instance with 20 threads for example. You’ll get much better throughput that way.

Also be aware that SQLAlchemy creates a connection pool rather then blindly create a connection per thread. So even if you over estimate the number of threads, using SQLAlchemy will not waste your database resources. This only applies if you use SQLAlchemy of course!