unittest -> pytest

[bongbui321]

resolve commaai/openpilot#32941

[bongbui321]

The three remaining points that you pointed out can be explained by this:

in the test_comms.py, we only have one instance of libpanda, and all the tests in that file share that one instance. since we are using multithreading, we have 6 tests in total for usbprotocol and only 4 runners, there is a high chance that while running one test, it got switched out to another one while the state of the switched out test remains (same libpanda). This explains why we need pytest-order (to ensure completion of one test before executing the next one).

One failure instance of running without pytest-order is that while executing reset_rx test case it got switched out and test receive_usb is run instead, this will create a failed test, since now the receive usb test now reading the rx_q data from the reset_rx. The order of the test doesn't matter, that is to ensure that the tests are done sequentially rather getting switched out and operate on the current states of another test

The same apply for this:

Why do we need to set a different safety mode here?

Since when we used to use unitttest -> send_usb is run first before the reset_tx so the safety tx hook is already defined from the send_usb test so you don't need to redefine it in reset_tx anymore (you can confirm this by disable running send_usb test) and run the reset _tx only and that test would fail. pytest doesn't follow that test execution order anymore and it might run the reset_tx first, so without the safety hook definition, then it wouldn't get any tx messages

Happy to clarify more if that's not clear

[adeebshihadeh]

We either need to:

1. reset all state such that the order doesn't matter
2. fix this so it can run in parallel with their own instances

Ideally, we can do #2.

[adeebshihadeh]

Closing due to inactivity.