Geçenlerde RSS feed bildirimlerinin en hızlı şekilde cebime gelmesini sağlamak için oturup kodunu yazmaya başladım ancak async fonksiyonun içerisindeki for döngüsünün son evresinde takıldı ve öylece kaldı. Debug ediyorum fakat iki fonksiyon arasında durmaksızın dönüyor program içerde bir yerde, muhtemelen fonksiyonun içerisindeki bir işlem engelliyor kodu. Programın nerede takıldığını öğrenebilmek için bir biraz arama yaptım ve pystuck isimli open-source bir projeyle karşılaştım.
Pystuck, Remote Python Call (RPyC) ile thread dump’ını alabilmenizi sağlıyor, yani arkaplanda bir debugging server çalıştırarak yapıyor bunu.
- Çalışan tüm thread’lerin dump’ını yazdırabilirsiniz (uzun süre çalışan / çalışmayı bloke eden parçacığın tespitinde)
- Çalışan kodu kesintiye uğratmadan modül ve değişkenleri inceleyebilirsiniz (deadlock tespitinde)
Şu şekilde kurduktan sonra:
pip install pystuck
Kodda dump’ını almak istediğiniz kısımdan hemen önceki satıra aşağıdaki satırı yerleştirin:
import pystuck; pystuck.run_server()
Program çalışırken ilgili satıra geldiğinde bir ipython shell açılacak, ondan sonra stack-trace’i çekmeye başlayabilirsiniz.
Örneğin, aşağıdaki kod son loop’ta takılıyor:
async def check_feeds() -> None:
log.info(f"Starting to parse rss feeds.")
import pystuck; pystuck.run_server()
for idx, feed_url in enumerate(upwork_categories):
log.info(f"Successfully parsed one of the categories.")
# next steps..
Buradaki kod döngünün son adımında takılıyor. Bir terminal açıp pystuck yazıp debugging server’a bağlanıyoruz. Artından %show threads komutuyla dump’ı çekebiliriz:
(venv) C:\Users\ABACI\Desktop\RssFeed>pystuck
Welcome to the pystuck interactive shell.
Use the 'modules' dictionary to access remote modules (like 'os', or '__main__')
Use the `%show threads` magic to display all thread stack traces.
In [1]: %show threads
<_MainThread(MainThread, started 9804)>
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 2141, in <module>
main()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 2132, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1441, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1448, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/ABACI/Desktop/Upwork Proposal/RssFeed/Main.py", line 200, in <module>
asyncio.ensure_future(check_feeds())
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 603, in run_until_complete
self.run_forever()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\windows_events.py", line 316, in run_forever
super().run_forever()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 570, in run_forever
self._run_once()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 1859, in _run_once
handle._run()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\asyncio\events.py", line 81, in _run
self._context.run(self._callback, *self._args)
File "C:/Users/ABACI/Desktop/Upwork Proposal/RssFeed/Main.py", line 177, in check_feeds
for idx, feed_url in enumerate(upwork_categories):
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1112, in do_wait_suspend
self._do_wait_suspend(thread, frame, event, arg, suspend_type, from_this_thread)
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 1127, in _do_wait_suspend
time.sleep(0.01)
<WriterThread(pydevd.Writer, started daemon 14824)>
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 216, in run
self._on_run()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 365, in _on_run
cmd = self.cmdQueue.get(1, 0.1)
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\queue.py", line 179, in get
self.not_empty.wait(remaining)
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
<ReaderThread(pydevd.Reader, started daemon 20328)>
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 216, in run
self._on_run()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 290, in _on_run
r = self.sock.recv(1024)
<PyDBCommandThread(pydevd.CommandThread, started daemon 11656)>
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 216, in run
self._on_run()
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevd.py", line 144, in _on_run
self._py_db_command_thread_event.wait(0.3)
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
<Thread(Thread-6, started daemon 12156)>
File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_monkey.py", line 773, in __call__
ret = self.original_func(*self.args, **self.kwargs)
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 890, in _bootstrap
self._bootstrap_inner()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\ABACI\PycharmProjects\RssFeed\venv\lib\site-packages\rpyc\utils\server.py", line 266, in start
self.accept()
File "C:\Users\ABACI\PycharmProjects\RssFeed\venv\lib\site-packages\rpyc\utils\server.py", line 142, in accept
sock, addrinfo = self.listener.accept()
File "C:\Users\ABACI\AppData\Local\Programs\Python\Python38\lib\socket.py", line 292, in accept
fd, addr = self._accept()
<Thread(Thread-7, started daemon 3532)>
File "C:\Users\ABACI\PycharmProjects\RssFeed\venv\lib\site-packages\pystuck\thread_probe.py", line 15, in thread_frame_generator
yield (thread_, frame)
Burada sorunun kaynağını bulup ortadan kaldırma işlemine geçebilirsiniz.