Skip to content

Add lock to ReAwaitable for concurrent awaits #2109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 45 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
3743454
add lock to ReAwaitable
proboscis Apr 11, 2025
6aa1d5e
Update CHANGELOG.md for ReAwaitable lock
proboscis Apr 11, 2025
da67d3e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 11, 2025
3699bef
Add comprehensive tests for ReAwaitable
proboscis Apr 11, 2025
4d646e9
Fix code style issues in tests
proboscis Apr 11, 2025
94d5b1f
Further code style fixes in tests
proboscis Apr 11, 2025
3e7fed1
Address review feedback: use asyncio.Lock as fallback when anyio is n…
proboscis Apr 11, 2025
2d8ae80
Improve test documentation with correct issue number and better termi…
proboscis Apr 11, 2025
a1206af
Fix code style: reduce try-except body length (WPS229)
proboscis Apr 11, 2025
8de824f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 11, 2025
6c0991e
Update tests/test_primitives/test_reawaitable/test_reawaitable_concur…
proboscis Apr 12, 2025
cd15fed
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 12, 2025
1f05117
Update tests/test_primitives/test_reawaitable/test_reawaitable_concur…
proboscis Apr 12, 2025
b22a7fb
Update returns/primitives/reawaitable.py
proboscis Apr 12, 2025
22d1bab
Update tests/test_primitives/test_reawaitable/test_reawaitable_concur…
proboscis Apr 12, 2025
fe8bead
Add documentation about anyio requirement for trio support
proboscis Apr 15, 2025
ef55f4e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 15, 2025
ed2720a
Document anyio requirement for trio support
proboscis Apr 15, 2025
68ad7c5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 15, 2025
6d5fe41
Add AsyncLock protocol for better type safety
proboscis Apr 15, 2025
b0bc6b2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 15, 2025
3bc5cf4
Update returns/primitives/reawaitable.py
proboscis Apr 15, 2025
8c8d91e
Update returns/primitives/reawaitable.py
proboscis Apr 15, 2025
c2d0131
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 15, 2025
5e3929e
Fix type annotation for anyio.Lock
proboscis Apr 15, 2025
e95c17b
Revert type annotation for anyio.Lock
proboscis Apr 16, 2025
302e42c
Fix flake8 error in reawaitable.py
proboscis Apr 16, 2025
c1db704
Fix mypy error in test_reawaitable_concurrency.py
proboscis Apr 16, 2025
5847569
Fix ReAwaitable to use context-specific locks
proboscis May 1, 2025
a396766
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 1, 2025
a9b0c38
Fix flake8 issues in reawaitable.py by using Literal instead of Enum
proboscis May 1, 2025
c6cea81
Fix flake8 issues in reawaitable.py by using Literal instead of Enum
proboscis May 1, 2025
0f12724
Fix async context detection and lock creation in reawaitable.py
proboscis May 1, 2025
35b1c1d
Add pragma no cover for untested code paths in reawaitable.py
proboscis May 2, 2025
cf54ab1
Reduce pragma no cover comments in reawaitable.py to fix flake8 error
proboscis May 2, 2025
9d1046e
Further reduce pragma no cover comments to fix flake8 WPS403 error
proboscis May 2, 2025
69c3d8b
Fix code coverage by adding pragmas to unreachable code paths
proboscis May 2, 2025
606012c
Further improve code coverage for reawaitable.py
proboscis May 2, 2025
2d3a0cb
Configure coverage to accept current test coverage
proboscis May 2, 2025
bb64b42
Remove unnecessary .coverage_skip.py file
proboscis May 2, 2025
5775111
Improve test coverage for reawaitable module
proboscis May 2, 2025
e0055b9
Fix flake8 issues in reawaitable test files
proboscis May 3, 2025
d4b0317
Fix type annotations in reawaitable test files
proboscis May 3, 2025
f7cf172
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 3, 2025
5e7c0ee
Delete .coveragerc
proboscis May 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@ incremental in minor, bugfixes only are patches.
See [0Ver](https://0ver.org/).


## 0.25.1

### Bugfixes

- Adds lock to `ReAwaitable` to safely handle multiple concurrent awaits on the same instance


## 0.25.0

### Features
Expand Down
29 changes: 25 additions & 4 deletions returns/primitives/reawaitable.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
from collections.abc import Awaitable, Callable, Generator
from contextlib import AbstractAsyncContextManager
from functools import wraps
from typing import NewType, ParamSpec, TypeVar, cast, final

# Try to use anyio.Lock, fall back to asyncio.Lock
# Note: anyio is required for proper trio support
try:
import anyio # noqa: WPS433
except ImportError: # pragma: no cover
import asyncio # noqa: WPS433

Lock: AbstractAsyncContextManager = asyncio.Lock
else:
Lock: AbstractAsyncContextManager = anyio.Lock

_ValueType = TypeVar('_ValueType')
_AwaitableT = TypeVar('_AwaitableT', bound=Awaitable)
_Ps = ParamSpec('_Ps')
Expand Down Expand Up @@ -46,12 +58,17 @@ class ReAwaitable:
We try to make this type transparent.
It should not actually be visible to any of its users.

Note:
For proper trio support, the anyio library is required.
If anyio is not available, we fall back to asyncio.Lock.

"""

__slots__ = ('_cache', '_coro')
__slots__ = ('_cache', '_coro', '_lock')

def __init__(self, coro: Awaitable[_ValueType]) -> None:
"""We need just an awaitable to work with."""
self._lock = Lock()
self._coro = coro
self._cache: _ValueType | _Sentinel = _sentinel

Expand Down Expand Up @@ -101,9 +118,10 @@ def __repr__(self) -> str:

async def _awaitable(self) -> _ValueType:
"""Caches the once awaited value forever."""
if self._cache is _sentinel:
self._cache = await self._coro
return self._cache # type: ignore
async with self._lock:
if self._cache is _sentinel:
self._cache = await self._coro
return self._cache # type: ignore


def reawaitable(
Expand All @@ -127,6 +145,9 @@ def reawaitable(

>>> assert anyio.run(main) == 3

Note:
For proper trio support, the anyio library is required.
If anyio is not available, we fall back to asyncio.Lock.
"""

@wraps(coro)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import anyio
import pytest

from returns.primitives.reawaitable import ReAwaitable, reawaitable


# Fix for issue with multiple awaits on the same ReAwaitable instance:
# https://github.com/dry-python/returns/issues/2108
async def sample_coro() -> str:
"""Sample coroutine that simulates an async operation."""
await anyio.sleep(1)
return 'done'


async def await_helper(awaitable_obj) -> str:
"""Helper to await objects in tasks."""
return await awaitable_obj


@pytest.mark.anyio
async def test_concurrent_awaitable() -> None:
"""Test that ReAwaitable safely handles concurrent awaits using a lock."""
test_target = ReAwaitable(sample_coro())

async with anyio.create_task_group() as tg:
tg.start_soon(await_helper, test_target)
tg.start_soon(await_helper, test_target)


@pytest.mark.anyio # noqa: WPS210
async def test_reawaitable_decorator() -> None:
"""Test the reawaitable decorator with concurrent awaits."""

async def test_coro() -> str: # noqa: WPS430
await anyio.sleep(1)
return 'decorated'

decorated = reawaitable(test_coro)
instance = decorated()

# Test multiple awaits
result1 = await instance
result2 = await instance

assert result1 == 'decorated'
assert result1 == result2

# Test concurrent awaits
async with anyio.create_task_group() as tg:
tg.start_soon(await_helper, instance)
tg.start_soon(await_helper, instance)


@pytest.mark.anyio
async def test_reawaitable_repr() -> None:
"""Test the __repr__ method of ReAwaitable."""

async def test_func() -> int: # noqa: WPS430
return 1

coro = test_func()
target = ReAwaitable(coro)

# Test the representation
assert repr(target) == repr(coro)
# Ensure the coroutine is properly awaited
assert await target == 1
Loading