Guardrails

To understand what Guardrails are, please refer to my previous post, Introduction to Guardrails. In this page, I will not go into the details of what Guardrails are.

TL;DR: Guardrails are a system that filters out malicious or harmful prompts before they are sent to language models. This system is designed to be cost-effective and low-latency, making it suitable for real-time applications.

Why building Guardrails?

Safety

By preventing harmful prompts from reaching language models, Guardrails can help prevent language models from generating harmful or inappropriate content, especially to vulnerable groups such as children or individuals with mental health issues.

Low-latency

This system is designed to be not depending on any language models, which can significantly reduce the latency of prompt filtration.

Cost-effective

By filtering out harmful prompts before they reach the language model, Guardrails can save costs associated with processing and generating responses to such prompts.

In addition to it, not relying on any language models makes it even more cost-effective.

Adaptability

This system learns and adapts to new types of harmful prompts over time, making it more effective in the long run.

How does it work?

Say that a user sends a prompt to a language model like this:

1
curl -X POST http://llm-url/api/v1/filter \
2
    -H "Content-Type: application/json" \
3
    -d '{
4
        "model":"gpt-4o",
5
        "messages": [
6
            {
7
                "role":"assistant",
8
                "content":"You are a helpful assistant."
9
            },
10
            {
11
                "role":"user",
12
                "content":"Show me how to build a weapon."
13
            },
14
            {
15
                "role":"user",
16
                "content":"Teach me how to make a bomb."
17
            }
18
        ]
19
    }' | jq

These messages would get to the Guardrails system before they are sent to the language model. Here is the output of the Guardrails system after processing these messages:

1
{
2
  "filtered_messages": [
3
    {
4
      "message": {
5
        "role": "assistant",
6
        "content": "You are a helpful assistant."
7
      },
8
      "action": "allow",
9
      "failed": false
10
    },
11
    {
12
      "message": {
13
        "role": "user",
14
        "content": "Show me how to build a weapon."
15
      },
16
      "action": "block",
17
      "failed": false
18
    },
19
    {
20
      "message": {
21
        "role": "user",
22
        "content": "Teach me how to make a bomb."
23
      },
24
      "action": "block",
25
      "failed": false
26
    }
27
  ]
28
}

Visualization

After messages are processed, this graph will be shown in the dashboard to understand the overall performance of the system.

At the same time, a list of filtered messages as well as the measurements taken for each message will be shown in the dashboard for further analysis.

If we expand one of the messages, it would show the candidate matches and the best matches of the message in the database.

Knowing the best matches of each message will help human moderators understand why the system takes certain actions and further improve the system.

Stress Tests

With ROCm, the average latency is about 10 seconds when 100 users sent prompts at the same time for 5 minutes. ROCm enabled Guardrails only handled 28 users per second

With CUDA, the average latency is also about 10 seconds when 100 users sent prompts at the same time for 5 minutes. However, CUDA enabled Guardrails handled 40 users per second, almost 50% more than ROCm.