Takılan Thread’lerin pystuck ile Debug Edilmesi

pystuck

Geçenlerde RSS feed bildirimlerinin en hızlı şekilde cebime gelmesini sağlamak için oturup kodunu yazmaya başladım ancak async fonksiyonun içerisindeki for döngüsünün son evresinde takıldı ve öylece kaldı. Debug ediyorum fakat iki fonksiyon arasında durmaksızın dönüyor program içerde bir yerde, muhtemelen fonksiyonun içerisindeki bir işlem engelliyor kodu. Programın nerede takıldığını öğrenebilmek için bir biraz arama yaptım ve pystuck isimli open-source bir projeyle karşılaştım.

Pystuck, Remote Python Call (RPyC) ile thread dump’ını alabilmenizi sağlıyor, yani arkaplanda bir debugging server çalıştırarak yapıyor bunu.

  1. Çalışan tüm thread’lerin dump’ını yazdırabilirsiniz (uzun süre çalışan / çalışmayı bloke eden parçacığın tespitinde)
  2. Çalışan kodu kesintiye uğratmadan modül ve değişkenleri inceleyebilirsiniz (deadlock tespitinde)

Şu şekilde kurduktan sonra:

pip install pystuck

Kodda dump’ını almak istediğiniz kısımdan hemen önceki satıra aşağıdaki satırı yerleştirin:

import pystuck; pystuck.run_server()

Program çalışırken ilgili satıra geldiğinde bir ipython shell açılacak, ondan sonra stack-trace’i çekmeye başlayabilirsiniz.

Örneğin, aşağıdaki kod son loop’ta takılıyor:

async def check_feeds() -> None:
        log.info(f"Starting to parse rss feeds.")
        import pystuck; pystuck.run_server()
        for idx, feed_url in enumerate(upwork_categories):
            log.info(f"Successfully parsed one of the categories.")
            # next steps..

Buradaki kod döngünün son adımında takılıyor. Bir terminal açıp pystuck yazıp debugging server’a bağlanıyoruz. Artından %show threads komutuyla dump’ı çekebiliriz:

(venv) C:\Users\ABACI\Desktop\RssFeed>pystuck
Welcome to the pystuck interactive shell.
Use the 'modules' dictionary to access remote modules (like 'os', or '__main__')
Use the `%show threads` magic to display all thread stack traces.

In [1]: %show threads
<_MainThread(MainThread, started 9804)>
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 2141, in <module>
    main()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 2132, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1441, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1448, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/ABACI/Desktop/Upwork Proposal/RssFeed/Main.py", line 200, in <module>
    asyncio.ensure_future(check_feeds())
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\windows_events.py", line 316, in run_forever
    super().run_forever()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 570, in run_forever
    self._run_once()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 1859, in _run_once
    handle._run()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "C:/Users/ABACI/Desktop/Upwork Proposal/RssFeed/Main.py", line 177, in check_feeds
    for idx, feed_url in enumerate(upwork_categories):
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1112, in do_wait_suspend
    self._do_wait_suspend(thread, frame, event, arg, suspend_type, from_this_thread)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1127, in _do_wait_suspend
    time.sleep(0.01)

<WriterThread(pydevd.Writer, started daemon 14824)>
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 216, in run
    self._on_run()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 365, in _on_run
    cmd = self.cmdQueue.get(1, 0.1)
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\queue.py", line 179, in get
    self.not_empty.wait(remaining)
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 306, in wait
    gotit = waiter.acquire(True, timeout)

<ReaderThread(pydevd.Reader, started daemon 20328)>
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 216, in run
    self._on_run()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 290, in _on_run
    r = self.sock.recv(1024)

<PyDBCommandThread(pydevd.CommandThread, started daemon 11656)>
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 216, in run
    self._on_run()
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 144, in _on_run
    self._py_db_command_thread_event.wait(0.3)
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 558, in wait
    signaled = self._cond.wait(timeout)
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 306, in wait
    gotit = waiter.acquire(True, timeout)

<Thread(Thread-6, started daemon 12156)>
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_monkey.py", line 773, in __call__
    ret = self.original_func(*self.args, **self.kwargs)
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\ABACI\PycharmProjects\RssFeed\venv\lib\site-packages\rpyc\utils\server.py", line 266, in start
    self.accept()
  File "C:\Users\ABACI\PycharmProjects\RssFeed\venv\lib\site-packages\rpyc\utils\server.py", line 142, in accept
    sock, addrinfo = self.listener.accept()
  File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\socket.py", line 292, in accept
    fd, addr = self._accept()

<Thread(Thread-7, started daemon 3532)>
  File "C:\Users\ABACI\PycharmProjects\RssFeed\venv\lib\site-packages\pystuck\thread_probe.py", line 15, in thread_frame_generator
    yield (thread_, frame)

Burada sorunun kaynağını bulup ortadan kaldırma işlemine geçebilirsiniz.

Leave a Reply

Your email address will not be published. Required fields are marked *