Quantcast
Viewing latest article 11
Browse Latest Browse All 55

How should mysql.connector be shared correctly with threads in Django applications?

This concerns a potential solution to my earlier question in which I was getting commands out of sync; you can't run this command now after wrapping mysql cursors in __enter__ and __exit__.

I have read elsewhere that Django does not spawn new threads to handle multiple requests, but rather multiple Django processes would normally be spawned in order to achieve parallelism. Despite this advice, I began printing Thread.ident and Thread.native_id and noticed that they were changing.

Based on this, I speculatively wrote something like:

def db_connect ():  t = threading.current_thread()  if not hasattr (t, 'db_connection'):    print (f"New MyDBConnectionClass for {t.ident}~{t.native_id}")    t.db_connection = MyDBConnectionClass ()  return t.db_connection.get_cursor () # This object has a __exit__ which closes the cursor

along with

class MyDBConnectionClass ():  def __del__(self):    t = threading.current_thread()    print (f"MyDBConnectionClass deleted for {t.ident}~{t.native_id}")

the view handlers' usage of the cursors is unchanged:

with db_connect() as db:  results = db.all (query, args)

So far this seems to have fixed the commands out of sync error (and the exit code 245 crash mentioned in the original question).

MyDBConnectionClass.__delete__ is not being called, but various instances are created for a few different ident values.

My current hypothesis is that there is some sort of thread pool going on, and my application was crashing unpredictably because sometimes (often) a view would be handled in the thread which created the initial connection, but sometimes not. This experiment seems to show that giving each thread a distinct connection works, which makes sense, but I am unsatisfied because:

  • the Thread objects are apparently not being deleted (or else something else is preventing MyDBConnection.__del__ from being called)
  • this doesn't agree with the documentation and examples I have read elsewhere
  • it's messy, and Django has a fairly clean design as far as I have seen -- I think I must be missing something

So, what is the correct way to handle mysql connection and cursor objects so that I can do

with my_connection_wrapper.get_cursor() as my_cursor_wrapper:  my_cursor_wrapper.foo ()

freely in Django views without leaking resources and without causing thread affinity instability issues (assuming that really is the problem)?


Viewing latest article 11
Browse Latest Browse All 55

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>