Start multiple python processes using supervisor

In this post I am going to show how to start up multiple instances of a python app using supervisor on Ubuntu.  This will allow us to run more than one process, on a single server and make better use of the resources per server. If you have nginx running on the same server you can then use all of these python processes to serve incoming requests.

Background

I had an application which ran a single python process behind nginx to serve incoming requests. I was using the smallest possible EC2 server to handle this, but even then I noticed that there was still plenty of capacity available on the server. To make better use of the resources, I decided to run more than one python process on separate ports.

To accomplish this I used supervisor to control the process creation and management.

Procedure

First make sure that you have the supervisor package installed on Ubuntu. As the root user, one can install supervisor using the following command

apt-get update
apt-get install supervisor

Once the installation is done make sure that the service is running.

systemctl restart supervisor.service

The next step is to prepare the config file read by supervisor. I did this by editing a file under /etc/supervisor/conf.d/app.conf. you can use an appropriate name for your app.

Paste the following configuration and save the file.

[program:myapp]
numprocs=5
numprocs_start=01
command=/usr/bin/python3 ./app_server.py 89%(process_num)02d
process_name=%(program_name)s_%(process_num)02d
priority=999
autostart=true
autorestart=unexpected
startsecs=5
startretries=3
exitcodes=0,2
stopsignal=TERM
stopwaitsecs=10
directory=/myapp/prd/app
user=appuser
redirect_stderr=false
stdout_logfile=/var/log/supervisor/app-%(process_num)s.out
stdout_logfile_maxbytes=10MB
stdout_logfile_backups=10
stderr_logfile=/var/log/supervisor/app-%(process_num)s.err
stderr_logfile_maxbytes=10MB
stderr_logfile_backups=10

Configuration Review

Let us review the configuration just copied to app.conf

[program:myapp]
numprocs=5
numprocs_start=01
command=/usr/bin/python3 ./app_server.py 89%(process_num)02d
process_name=%(program_name)s_%(process_num)02d

I configured the program section and used the name myapp to identify it.

The numprocs=5 indicates I want to run five instances of my program and the starting integer for my processes is 01. My run command effectively becomes

/usr/bin/python3 ./app_server.py 8901
/usr/bin/python3 ./app_server.py 8902
/usr/bin/python3 ./app_server.py 8903
/usr/bin/python3 ./app_server.py 8904
/usr/bin/python3 ./app_server.py 8905

Usually the process_name need not be set, however in my case I am running more than one instance therefore I modify the process_name.

See more about supervisor configuration here

autostart=true
autorestart=unexpected
exitcodes=0,2

Setting autostart to true ensures that the program will start automatically when supervisord is started. autorestart=unexpected makes supervisord restart the program when the program exits with an exit code that is not associated with the configuration.

directory=/myapp/prd/app
user=appuser

The directory specified is where supervisord does a chdir before starting the process as the user appuser.

stdout_logfile=/var/log/supervisor/app-%(process_num)s.out
stdout_logfile_maxbytes=10MB
stdout_logfile_backups=10
stderr_logfile=/var/log/supervisor/app-%(process_num)s.err

The above defines a stdout and stderr location for the program. Each process has its own location and the file is identified by the process number.

After saving the config file, I can inform supervisord about our new program by making it re-read the configuration.

supervisorctl reread

Supervisor service itself can be stopped and started to completely stop and start the program. You can do this by running

systemctl stop supervisor
systemctl stop supervisor

or

systemctl restart supervisor

After the supervisor has been restarted you can run a ps to verify your program

 ps -ef | grep app_server
 appuser   2183  2174  0 17:36 ?        00:00:19 /usr/bin/python3 ./app_server.py 8904
 appuser   2184  2174  0 17:36 ?        00:00:18 /usr/bin/python3 ./app_server.py 8905
 appuser   2185  2174  0 17:36 ?        00:00:19 /usr/bin/python3 ./app_server.py 8902
 appuser   2186  2174  0 17:36 ?        00:00:19 /usr/bin/python3 ./app_server.py 8903
 appuser   2187  2174  0 17:36 ?        00:00:19 /usr/bin/python3 ./app_server.py 8901

This confirms that I have five instances of the python program running on a single server. Now you can modify your ngxin configuration and proxy requests to all five of these instances.

Further Reading

• Supervisor introduction.
• Read more about supervisor here.

Featured Image Photo Credit:

Shahadat Shemul