Dec 20, 2023 # Announcing Guardrails AI 0.3.0 Company Snowglobe 🎉Exciting News! The team has been hard at work and is excited to announce that the latest release of guardrails, v0.3.0, is now live! ## TL;DR 🎉 **New Features** 🎉 - Streaming! - Anthropic and Hugging Face models support - Input Validation - New validators! - OnTopic validator - Competitor check validator - ToxicLanguage validator 🔧 **Improvements** 🔧 - [History and logs usability](https://docs.guardrailsai.com/0-3-migration/#history-logs-improvements) - Improved [isHighQualityTranslation](https://docs.guardrailsai.com/examples/translation_with_quality_check/) validator - [LlamaIndex example of using GuardrailsOutputParser](https://docs.guardrailsai.com/examples/llamaindex-output-parsing) 👏🏽 **New Contributors** 👏🏽 We'd also like to thank our two new contributors with this release! - [@emekaokoli19](https://github.com/emekaokoli19) made their first contribution in [#486](https://github.com/guardrails-ai/guardrails/pull/486)! - [@tthoraldson](https://github.com/tthoraldson) made their first contribution in [#411](https://github.com/guardrails-ai/guardrails/pull/411)! ## Recap — what is Guardrails AI? Guardrails AI allows you to define and enforce assurance for AI applications from structuring output to quality controls. Guardrails AI does this by creating a _firewall-like_ bounding box around the LLM application (a Guard) that contains a set of validators. A Guard can include validators from our library or a custom validator that could enforce what your application is intended to do. ## Streaming As developers begin making usability improvements to LLM applications, streaming becomes an important tool in the toolbox. Guardrails now supports streaming for both unstructured text and structured JSON! When streaming is enabled, the received chunks are concatenated to form few valid fragments that are validated one by one. As soon as each fragment is validated, it's streamed to the user. In order to form fragments, for JSON outputs, we check each chunk whether it's a valid JSON and if it's not, we either wait for more chunks or parse it accordingly. Once, we have a somewhat valid fragment, we perform sub-schema validation between the fragment and the expected output schema. ```python # Wrap the OpenAI API call with the `guard` object fragment_generator = guard( openai.Completion.create, prompt_params={"doctors_notes":doctors_notes}, engine="text-davinci-003", max_tokens=1024, temperature=0, stream=True, ) for op in fragment_generator: clear_output(wait=True) print(op) time.sleep(0.5) ``` See the [Streaming](https://docs.guardrailsai.com/examples/streaming/) documentation for more details. ## Input Validation Input Validation is one of the most requested features to date! Guardrails now supports validating inputs (prompts, instructions, msg_history) with string validators. ```python from guardrails.validators import TwoWords from pydantic import BaseModel class Pet(BaseModel): name: str age: int guard = Guard.from_pydantic(Pet) guard.with_prompt_validation([TwoWords(on_fail="exception")]) outcome = guard( openai.ChatCompletion.create, prompt="This is not two words", ) outcome.error ``` See the [InputValidation](https://docs.guardrailsai.com/examples/input_validation/) documentation for more details. ## Anthropic and Hugging Face models support We've heard your requests for better support of other models and are happy to share that we now support Anthropic and Hugging Face models! ```python import torch from guardrails import Guard from guardrails.validators import ValidLength, ToxicLanguage from transformers import AutoModelForCausalLM, AutoTokenizer prompt = "Hello, I'm a language model," guard = Guard.from_string( validators=[ToxicLanguage(on_fail="fix")], prompt=prompt ) torch_device = "cuda" if torch.cuda.is_available() else "cpu" tokenizer = AutoTokenizer.from_pretrained("gpt2") model = AutoModelForCausalLM.from_pretrained( "gpt2", pad_token_id=tokenizer.eos_token_id ).to(torch_device) model_inputs = tokenizer(prompt, return_tensors="pt").to(torch_device) response = guard( llm_api=model.generate, max_new_tokens=40, tokenizer=tokenizer, **model_inputs, ) ``` See the docs on [LLM API wrappers](https://docs.guardrailsai.com/llm_api_wrappers/#hugging-face) for more details. ## New Validators ### OnTopic Validator We've released the [OnTopic validator](https://docs.guardrailsai.com/api_reference/validators/#guardrails.validators.OnTopic) your LLM application on topic - one of the most requested validators to date! The OnTopic validator accepts at least one valid topic and an optional list of invalid topics. The default behavior first runs a Zero-Shot model, and then falls back to ask OpenAI's gpt-3.5-turbo if the Zero-Shot model is not confident in the topic classification (score < 0.5). In our experiments this LLM fallback increases accuracy by 15% but also increases latency (more than doubles the latency in the worst case). Both the Zero-Shot classification and the GPT classification may be toggled. ```python # Create the Guard with the OnTopic Validator guard = gd.Guard.from_string( validators=[ OnTopic( valid_topics=valid_topics, invalid_topics=invalid_topics, device=device, disable_classifier=False, disable_llm=True, on_fail="exception", ) ] ) # Test with a given text output = guard.parse(llm_output=text) print(output.error) ``` See the documentation on the [OnTopic](https://docs.guardrailsai.com/examples/response_is_on_topic/) validator for more details. ### CompetitorCheck validator This validator checks LLM output to flag sentences naming one of your competitors and removes those sentences from the final output. When setting on-fail to 'fix' this validator will remove the flagged sentences from the output. You need to provide an extensive list of your competitors' names including all common variations (e.g. JP Morgan, JP Morgan Chase, etc.) the compilation of this list will have an impact on the ultimate outcome of the validation. ```python # Create the Guard with the CompetitorCheck Validator guard = gd.Guard.from_string( validators=[CompetitorCheck(competitors=competitors_list, on_fail="fix")], description="testmeout", ) # Test with a given text output = guard.parse(llm_output=text, metadata={}) print(output) ``` See the documentation on [CompetitorCheck](https://docs.guardrailsai.com/examples/competitors_check/) validator for more details! ### ToxicLanguage validator This validator checks whether an LLM-generated response contains toxic language. It uses the pre-trained multi-label model from HuggingFace - unitary/unbiased-toxic-roberta to check whether the generated text is toxic. It supports both full-text-level and sentence-level validation. ```python # Test with validation method 'full' full_guard = gd.Guard.from_string( validators=[ToxicLanguage(validation_method="full", on_fail="fix")], description="testmeout", ) # Parse the raw response raw_response = "Stop being such a dumb piece of shit. Why can't you comprehend this?" output = full_guard.parse(llm_output=raw_response) print(output) ``` See the documentation on [ToxicLanguage](https://docs.guardrailsai.com/examples/toxic_language/) validator for more details! ## Validation Outcome Previous when calling `__call__` or `parse` on a Guard, the Guard would return a tuple of the raw LLM output and the validated output or just the validated output respectively. Now, in order to communicate more information, we respond with a ValidationOutcome class that contains the above information and more. ```python guard = Guard(...) response = guard(...) # or response = guard.parse(...) validated_output = response.validated_output ``` See [ValidationOutcome](https://docs.guardrailsai.com/api_reference/validation_outcome/#ValidationOutcome) in the API Reference for more information on these additional properties. ## History & Logs Improvements If you're familiar with Guardrails, then you might have used the `Guard.state` property to inspect how the Guard process behaved over time. In order to make the Guard process more transparent, as part of v0.3.0 we redesigned how you access this information. Now, on a Guard, you can access logs related to any `__call__` or `parse` call within the current session via Guard.history. ```python most_recent_call = guard.history.last print("status: ", most_recent_call.status) print("tokens used: ", most_recent_call.tokens_consumed) ``` See the [Logs documentation](https://docs.guardrailsai.com/concepts/logs) or [0.3.x migration guide](https://docs.guardrailsai.com/0-3-migration) for more details. ## Breaking changes - Refactoring response object from Guardrails. We now use [ValidationOutcome](https://docs.guardrailsai.com/api_reference/validation_outcome/#ValidationOutcome). - Refactoring access to logs, see the new [structure](https://docs.guardrailsai.com/concepts/logs/) here. ## Other changes - Shiny new docs! Restructuring documentation with more Pydantic examples, etc. ## Migration guide For more details on how to migrate to 0.3.0 please see our [migration guide](https://docs.guardrailsai.com/0-3-migration/). ## Take it for a spin! You can install the latest version of Guardrails with: `pip install guardrails-ai` There are a number of ways to engage with us: - Join the discord: [https://discord.gg/kVZEnR4WQK](https://discord.gg/kVZEnR4WQK) - Star us on Github: [https://github.com/guardrails-ai/guardrails](https://github.com/guardrails-ai/guardrails) - Follow us on Twitter: [https://twitter.com/guardrails_ai](https://twitter.com/guardrails_ai) - Follow us on LinkedIn: [https://www.linkedin.com/company/guardrailsai/](https://www.linkedin.com/company/guardrailsai/) We're always looking for contributions from the open source community. Check out [guardrails/issues](https://github.com/guardrails-ai/guardrails/issues) for a list of good starter issues.