= Resolve - Entry #3 by chrism on Jan 19, 2008 1:39 pm
Status: Accepted => Resolved
Somehow during reload a channel related to an http server stays open beyond the lifetime of the server itself. I fixed this by ensuring that all channels related to servers in the asyncore socket_map are closed when the servers are closed, e.g.:
Index: options.py
===================================================================
--- options.py (revision 678)
+++ options.py (working copy)
@@ -902,6 +902,22 @@
def close_httpservers(self):
for config, server in self.httpservers:
server.close()
+ map = self.get_socket_map()
+ # server._map is a reference to the asyncore socket_map
+ for dispatcher in map.values():
+ # For unknown reasons, sometimes an http_channel
+ # dispatcher in the socket map related to servers
+ # remains open *during a reload*. If one of these
+ # exists at this point, we need to close it by hand
+ # (thus removing it from the asyncore.socket_map). If
+ # we don't do this, 'cleanup_fds' will cause its file
+ # descriptor to be closed, but it will still remain in
+ # the socket_map, and eventually its file descriptor
+ # will be passed to # select(), which will bomb. See
+ # also http://www.plope.com/software/collector/253
+ dispatcher_server = getattr(dispatcher, 'server', None)
+ if dispatcher_server is server:
+ dispatcher.close()
def close_logger(self):
self.logger.close()
________________________________________
= Comment - Entry #2 by chrism on Jan 19, 2008 12:29 pm
Traceback (most recent call last):
File "bin/supervisord", line 7, in ?
sys.exit(
File "/Users/chrism/projects/supervisor/supervisor2/src/supervisor/supervisord.py", line 319, in main
go(options)
File "/Users/chrism/projects/supervisor/supervisor2/src/supervisor/supervisord.py", line 329, in go
d.main()
File "/Users/chrism/projects/supervisor/supervisor2/src/supervisor/supervisord.py", line 92, in main
self.run()
File "/Users/chrism/projects/supervisor/supervisor2/src/supervisor/supervisord.py", line 110, in run
self.runforever()
File "/Users/chrism/projects/supervisor/supervisor2/src/supervisor/supervisord.py", line 198, in runforever
r, w, x = self.options.select(r, w, x, timeout)
File "/Users/chrism/projects/supervisor/supervisor2/src/supervisor/options.py", line 973, in select
return select.select(r, w, x, timeout)
select.error: (9, 'Bad file descriptor')
________________________________________
= Request - Entry #1 by chrism on Dec 22, 2007 10:49 am
Status: Pending => Accepted
Supporters added: chrism
William Dode reports:
Hi,
When i type reload in supervisorctl it generaly (not every time) doesn't
restart supervisord :
supervisor> status
crieur RUNNING pid 4623, uptime 0:14:27
webgen RUNNING pid 4681, uptime 0:07:37
supervisor> reload
Really restart the remote supervisord process y/N? y
Restarted supervisord
supervisor> status
error: socket.error, (2, 'No such file or directory'): file: <string> line: 1
supervisord as died (in ps)
in the supervisord.log the last lines :
2007-12-21 18:28:36,545 INFO waiting for crieur, webgen to die
2007-12-21 18:28:36,552 INFO stopped: webgen (terminated by SIGTERM)
2007-12-21 18:28:36,554 INFO stopped: crieur (terminated by SIGTERM)
2007-12-21 18:28:36,585 INFO RPC interface 'supervisor' initialized
2007-12-21 18:28:36,586 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2007-12-21 18:28:36,587 INFO daemonizing the supervisord process
|
|