Skip to content

docs: fix output rail doc #1159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/getting-started/5-output-rails/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,12 +183,15 @@ You can enable streaming to provide asynchronous responses and reduce the time t
flows:
- self check output
streaming:
enabled: True
chunk_size: 200
context_size: 50

streaming: True
```

The `enabled: True` field is required to enable streaming output rails while the `streaming: True` field is needed to enable streaming generation.

1. Call the `stream_async` method and handle the chunked response:

```python
Expand Down
9 changes: 8 additions & 1 deletion docs/user-guides/configuration-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ nemoguardrails find-providers [--list]
```

The command supports two modes:

- Interactive mode (default): Guides you through selecting a provider type (text completion or chat completion) and then shows available providers for that type
- List mode (`--list`): Simply lists all available providers without interactive selection

Expand Down Expand Up @@ -679,13 +680,14 @@ You can enable streaming to begin receiving responses from the output rail soone

You must set the top-level `streaming: True` field in your `config.yml` file.

For each output rail, add the `streaming` field and configuration parameters.
For the output rails, add the `streaming` field and configuration parameters.

```yaml
rails:
output:
- rail name
streaming:
enabled: True
chunk_size: 200
context_size: 50
stream_first: True
Expand Down Expand Up @@ -735,6 +737,11 @@ The following table describes the subfields for the `streaming` field:
Specifying approximately 25% of `chunk_size` provides a good compromise.
- `50`

* - streaming.enabled
- When set to `True`, the toolkit executes output rails in streaming mode.

- `False`

* - streaming.stream_first
- When set to `False`, the toolkit applies the output rails to the chunks before streaming them to the client.
If you set this field to `False`, you can avoid streaming chunks of blocked content.
Expand Down