-
Notifications
You must be signed in to change notification settings - Fork 371
This AI sample lacks risk & safety evaluation implementation #156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Step 1) Provision an AI project. See infra in https://github.com/Azure-Samples/azureai-basic-python/tree/main/infra Step 2) Generate simulated QA pairs from the app using the adversial simulator. See callback code in https://github.com/Azure-Samples/azure-search-openai-demo/pull/2233/files#diff-8da4e313bfcffa446d60e3717caf532fa6344fdf6c210d869717088828197476 Step 3) Run the safety evaluators against the simulated answers: |
You can use this endpoint: |
cc @alfredodeza |
All done! |
As per email sent on 13th Dec with same subject, please double-check that this AI sample implements evaluations. In particular, what we are looking for is:
The text was updated successfully, but these errors were encountered: