Skip to content

Commit feb80e6

Browse files
committed
docs: Revise the GS experience.
- Remove the information about timing. The LLM calls do not seem to be populated the same. - Add reference to content safety notebook. Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
1 parent 887561f commit feb80e6

File tree

3 files changed

+28
-53
lines changed

3 files changed

+28
-53
lines changed

docs/getting-started.md

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -76,30 +76,30 @@ The sample code uses the [Llama 3.3 70B Instruct model](https://build.nvidia.com
7676
:end-before: "# end-generate-response"
7777
```
7878

79-
## Timing and Token Information
79+
1. Send a safe request and generate a response:
8080

81-
The following modification of the sample code shows the timing and token information for the guardrail.
82-
83-
- Generate a response and print the timing and token information:
81+
```{literalinclude} ../examples/configs/gs_content_safety/demo.py
82+
:language: python
83+
:start-after: "# start-safe-response"
84+
:end-before: "# end-safe-response"
85+
```
8486

85-
```{literalinclude} ../examples/configs/gs_content_safety/demo.py
86-
:language: python
87-
:start-after: "# start-get-duration"
88-
:end-before: "# end-get-duration"
89-
```
87+
_Example Output_
9088

91-
_Example Output_
89+
```{literalinclude} ../examples/configs/gs_content_safety/demo-out.txt
90+
:language: text
91+
:start-after: "# start-safe-response"
92+
:end-before: "# end-safe-response"
93+
```
9294

93-
```{literalinclude} ../examples/configs/gs_content_safety/demo-out.txt
94-
:language: text
95-
:start-after: "# start-get-duration"
96-
:end-before: "# end-get-duration"
97-
```
95+
## Next Steps
9896

99-
The timing and token information is available with the `print_llm_calls_summary()` method.
97+
- Run the `content_safety_tutorial.ipynb` notebook from the
98+
[example notebooks](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/examples/notebooks)
99+
directory of the GitHub repository.
100+
The notebook compares LLM responses with and without safety checks and classifies responses
101+
to sample prompts as _safe_ or _unsafe_.
102+
The notebook shows how to measure the performance of the checks, focusing on how many unsafe
103+
responses are blocked and how many safe responses are incorrectly blocked.
100104

101-
```{literalinclude} ../examples/configs/gs_content_safety/demo-out.txt
102-
:language: text
103-
:start-after: "# start-explain-info"
104-
:end-before: "# end-explain-info"
105-
```
105+
- Refer to [](user-guides/configuration-guide.md) for information about the `config.yml` file.

examples/configs/gs_content_safety/demo-out.txt

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,6 @@ I'm sorry, I can't respond to that.
33
# end-generate-response
44

55

6-
# start-get-duration
7-
Cape Hatteras National Seashore! It's a 72-mile stretch of undeveloped barrier islands off the coast of North Carolina, featuring pristine beaches, Cape Hatteras Lighthouse, and the Wright brothers' first flight landing site. Enjoy surfing, camping, and wildlife-spotting amidst the natural beauty and rich history.
8-
# end-get-duration
9-
10-
11-
# start-explain-info
12-
Summary: 3 LLM call(s) took 1.50 seconds and used 22394 tokens.
13-
14-
1. Task `content_safety_check_input $model=content_safety` took 0.35 seconds and used 7764 tokens.
15-
2. Task `general` took 0.67 seconds and used 164 tokens.
16-
3. Task `content_safety_check_output $model=content_safety` took 0.48 seconds and used 14466 tokens.
17-
18-
# end-explain-info
6+
# start-safe-response
7+
Cape Hatteras National Seashore: 72 miles of pristine Outer Banks coastline in North Carolina, featuring natural beaches, lighthouses, and wildlife refuges.
8+
# end-safe-response

examples/configs/gs_content_safety/demo.py

Lines changed: 4 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -58,33 +58,18 @@ async def stream_response(messages):
5858
print("# end-generate-response\n")
5959
sys.stdout = stdout
6060

61-
# start-get-duration
62-
explain_info = None
63-
64-
async def stream_response(messages):
65-
async for chunk in rails.stream_async(messages=messages):
66-
global explain_info
67-
if explain_info is None:
68-
explain_info = rails.explain_info
69-
print(chunk, end="")
70-
print()
71-
61+
# start-safe-response
7262
messages=[{
7363
"role": "user",
7464
"content": "Tell me about Cape Hatteras National Seashore in 50 words or less."
7565
}]
7666

7767
asyncio.run(stream_response(messages))
78-
79-
explain_info.print_llm_calls_summary()
80-
# end-get-duration
68+
# end-safe-response
8169

8270
stdout = sys.stdout
8371
with open("demo-out.txt", "a") as sys.stdout:
84-
print("\n# start-get-duration")
72+
print("\n# start-safe-response")
8573
asyncio.run(stream_response(messages))
86-
print("# end-get-duration\n")
87-
print("\n# start-explain-info")
88-
explain_info.print_llm_calls_summary()
89-
print("# end-explain-info\n")
74+
print("# end-safe-response\n")
9075
sys.stdout = stdout

0 commit comments

Comments
 (0)