Test fail when run under something - ChannelsLiveServerTestCase

[enzofrnt]

Hi,
I know the issue title might sound a bit strange, but I’m not sure how to better describe what’s going on.

I'm currently working on test cases to improve django-eventstream.

Here are the related tests:
tests/test_stream.py

When I run them individually using:

pytest tests/test_stream.py

everything works fine. But when I run them with:

pytest

or through:

tox

they start failing. It seems like when the tests are executed along with others, or maybe under different process contexts, something changes — perhaps the test priority or isolation — and some of the parsing logic in the code fails to retrieve all the events that were sent.

Is there a way to make sure my tests remain stable and work even when executed with others or through tox?

Thanks in advance!

[carltongibson]

Hi @enzofrnt. There's not much info here...

"Start failing" — How? What does that mean? (Relevant parts of a stack trace maybe?)

Some folks have reported issues with the way pytest handles the event loop... (but no-one pinned that down for me yet, so 🤷).

Pytest-asyncio has a warning about unittest subclasses not being supported, so maybe you're hitting that?

https://pypi.org/project/pytest-asyncio/

Difficult to say too much with more details to go on.

[enzofrnt]

Hi @carltongibson I will try to give you the most trace possible later.
Do you think that the use of pytest-asyncio can help me with my tests ?

[enzofrnt]

Re @carltongibson here some trace of my test :

tox config :

[tox]
envlist = py{310,3.11,3.12,3.13}
isolated_build = true
minversion = 3.0.0

[testenv]
allowlist_externals = pytest
commands =
    pytest ./tests/test_stream.py::RedisListenerTest::test_stream_from_inside
setenv =
    DJANGO_SETTINGS_MODULE = tests.settings
    PYTHONPATH = {toxinidir}

With this test fail probably because not run as main process in the tests.
And then here is the strange behavior :

Événements parsés: [Event(channel='test', type='stream-open', data={}, id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_0'}
Événements parsés: [Event(channel='test', type='test', data='test_0', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_1'}
Événements parsés: [Event(channel='test', type='test', data='test_1', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_2'}
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_3'}
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_4'}
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_5'}
Événements parsés: [Event(channel='test', type='test', data='test_5', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_6'}
Événements parsés: [Event(channel='test', type='test', data='test_6', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_7'}
Événements parsés: [Event(channel='test', type='test', data='test_7', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_8'}
Événements parsés: [Event(channel='test', type='test', data='test_8', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_9'}
Événements parsés: [Event(channel='test', type='test', data='test_9', id=None)]

There is event send and event parsed and here, we can see that some event disappear.

Something that not happening when test run as the main one (note thta it also work when doing pytest ./tests/test_stream.py and no more when doing just pytest ./tests):

(django-eventstream) Book • ➜  ~/Git/django-eventstream git:(default-cross-process-send_event) ✗  pytest ./tests/test_stream.py::RedisListenerTest::test_stream_from_inside
============================================================= test session starts ==============================================================
platform darwin -- Python 3.13.2, pytest-8.3.5, pluggy-1.5.0
rootdir: /Users/enzofournet/Git/django-eventstream
configfile: pytest.ini
plugins: asyncio-0.26.0
asyncio: mode=Mode.STRICT, asyncio_default_fixture_loop_scope=session, asyncio_default_test_loop_scope=function
collected 1 item                                                                                                                               

tests/test_stream.py use default
use default
Début du test
URL de test: http://localhost:50265/events/
RedisListener init
Réponse reçue: 200
Thread d'envoi interne démarré
Thread d'écoute démarré
Départ simultané !
Début de la réception des événements
Début de la lecture des événements
Événements parsés: []
Événements parsés: [Event(channel='test', type='stream-open', data={}, id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_0'}
Événements parsés: [Event(channel='test', type='test', data='test_0', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_1'}
Événements parsés: [Event(channel='test', type='test', data='test_1', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_2'}
Événements parsés: [Event(channel='test', type='test', data='test_2', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_3'}
Événements parsés: [Event(channel='test', type='test', data='test_3', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_4'}
Événements parsés: [Event(channel='test', type='test', data='test_4', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_5'}
Événements parsés: [Event(channel='test', type='test', data='test_5', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_6'}
Événements parsés: [Event(channel='test', type='test', data='test_6', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_7'}
Événements parsés: [Event(channel='test', type='test', data='test_7', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_8'}
Événements parsés: [Event(channel='test', type='test', data='test_8', id=None)]
Envoi de l'événement interne: {'channel': 'test', 'event_type': 'test', 'data': 'test_9'}
Événements parsés: [Event(channel='test', type='test', data='test_9', id=None)]
Tous les événements reçus: 11
send_thread.join()
listen_thread.join()
Événements reçus (11):
- Type: stream-open, Channel: test, Data: {}, Timestamp: 1744460777.592619
- Type: test, Channel: test, Data: test_0, Timestamp: 1744460779.59808
- Type: test, Channel: test, Data: test_1, Timestamp: 1744460779.7091541
- Type: test, Channel: test, Data: test_2, Timestamp: 1744460779.822887
- Type: test, Channel: test, Data: test_3, Timestamp: 1744460779.932273
- Type: test, Channel: test, Data: test_4, Timestamp: 1744460780.0417972
- Type: test, Channel: test, Data: test_5, Timestamp: 1744460780.156214
- Type: test, Channel: test, Data: test_6, Timestamp: 1744460780.266751
- Type: test, Channel: test, Data: test_7, Timestamp: 1744460780.378268
- Type: test, Channel: test, Data: test_8, Timestamp: 1744460780.488517
- Type: test, Channel: test, Data: test_9, Timestamp: 1744460780.603433
.

============================================================== 1 passed in 3.69s ===============================================================
(django-eventstream) Book • ➜  ~/Git/django-eventstream git:(default-cross-process-send_event) ✗ 

Then in the test here :
tests/test_stream.py

this async loop that in a thread seams to have an issue when run not as the main process :

https://github.com/enzofrnt/django-eventstream/blob/d5c3ae8e497ac33dbac2bc0a1ce24e95421e0c00/tests/test_stream.py#L154-L199