This concerns a potential solution to my earlier question in which I was getting commands out of sync; you can't run this command now
after wrapping mysql cursors in __enter__
and __exit__
.
I have read elsewhere that Django does not spawn new threads to handle multiple requests, but rather multiple Django processes would normally be spawned in order to achieve parallelism. Despite this advice, I began printing Thread.ident
and Thread.native_id
and noticed that they were changing.
Based on this, I speculatively wrote something like:
def db_connect (): t = threading.current_thread() if not hasattr (t, 'db_connection'): print (f"New MyDBConnectionClass for {t.ident}~{t.native_id}") t.db_connection = MyDBConnectionClass () return t.db_connection.get_cursor () # This object has a __exit__ which closes the cursor
along with
class MyDBConnectionClass (): def __del__(self): t = threading.current_thread() print (f"MyDBConnectionClass deleted for {t.ident}~{t.native_id}")
the view handlers' usage of the cursors is unchanged:
with db_connect() as db: results = db.all (query, args)
So far this seems to have fixed the commands out of sync
error (and the exit code 245 crash mentioned in the original question).
MyDBConnectionClass.__delete__
is not being called, but various instances are created for a few different ident
values.
My current hypothesis is that there is some sort of thread pool going on, and my application was crashing unpredictably because sometimes (often) a view would be handled in the thread which created the initial connection, but sometimes not. This experiment seems to show that giving each thread a distinct connection works, which makes sense, but I am unsatisfied because:
- the Thread objects are apparently not being deleted (or else something else is preventing
MyDBConnection.__del__
from being called) - this doesn't agree with the documentation and examples I have read elsewhere
- it's messy, and Django has a fairly clean design as far as I have seen -- I think I must be missing something
So, what is the correct way to handle mysql connection and cursor objects so that I can do
with my_connection_wrapper.get_cursor() as my_cursor_wrapper: my_cursor_wrapper.foo ()
freely in Django views without leaking resources and without causing thread affinity instability issues (assuming that really is the problem)?