Skip to content

Creating Rules for Success

Overview

In the Chatbot - Rulesets course you learned all about creating Griptape Rules and Rulesets and how to use them to give an Agent directions. In that case, it was mostly about providing personality and making sure the output was in JSON format. In this course, we'll use rules to help direct the Agent to use ShotGridTool and Vector Database when appropriate.

If you haven't viewed that course before, I highly recommend doing so - especially the section on creating Personality as it provides a wonderful overview of creating and using Rulesets, what they're used for, and why Griptape has them.

Importing

In order to use Rules and Rulesets, we need to import them into our app first. In the imports section of app.py, add the following import statement:

app.py
# ...

from griptape.rules import Rule, Ruleset

# ...

Create the Ruleset

Our first step will be to create the Ruleset for the agent. This ruleset will contain all the rules we will give it.

In app.py find the area of the code where you instantiate the agent, and insert the ruleset before it.

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent"
)

# Instantiate the agent
# ...

Rules

Act as...

We want the agent to have the correct context for its work, so the first ruleset will tell it to act as a studio coordinator who is an expert in Autodesk ShotGrid

Create a parameter for rules, and then add this first rule to the list of rules:

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        Rule("Act as a studio coordinator who is an expert with Autodesk ShotGrid"),
    ],
)

# ...

Main objective

Sometimes it's helpful to remind the LLM what its main objective is. In this case, we want to remind it that we want it to update ShotGrid using the ShotGridTool.

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        Rule("Act as a studio coordinator who is an expert with Autodesk ShotGrid"),
        Rule(
            "Your main objective is to find and update information in ShotGrid using the ShotGridTool"
        ),
    ],
)

# ...

Vector Database

Now let's remind it that it should use the VectorStoreTool when it needs information about how to use the API. This will ensure the most up-to-date and accurate use of the API.

Note: This rule is multiple lines long, so we're going to use a Python function dedent that will allow us to keep the code looking nice.

First, import dedent in your imports section of the code.

app.py
# ...

from textwrap import dedent

# ...

Then, add the new rule:

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        # ...
        Rule(
            dedent(
                """
            For specific information about how to use ShotGrid API activities, the 
            VectorStoreTool should be used. Take the necessary time to consult the 
            VectorStoreTool to ensure the most accurate and context-aware decisions 
            when interacting with the ShotGridTool and API.
            """
            )
        ),
    ],
)

# ...

Batch mode

ShotGrid provides access to a batch method that allows you to perform multiple create, update, and delete operations in a single request. Each operation in the batch is specified as a dictionary, and the operations are performed in the order they are provided.

We would like to ensure the Agent uses batch mode when appropriate.

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        # ...
        Rule(
            dedent(
                """
            Always use the ShotGrid batch() method when performing multiple create, update, or delete operations in ShotGrid.
            This should be done in a single request to improve efficiency and speed. 
            Batch method uses "request_type", "entity_type", and "entity_id".
            Failure to use the batch() method in these instances will be considered a violation of the rules."""
            )
        ),
    ],
)

Tip

While working on the rule for batch mode, I found that the LLM was sometimes not using it and would jump to just using create or update methods instead. I asked the agent for a good rule to enforce using the batch method, and it came up with this very strict rule.

And .. it worked! Sometimes just asking the LLM for the best rule provides excellent results.

Every task

Lastly, one final rule is to ensure the Agent uses the ShotGrid and doesn't just pretend to use it.

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        # ...
        Rule(
            dedent(
                """
            When asked to find, list, create, update, delete, retrieve details of entities, 
            update task status, update task data, list task assignments, retrieve the history of changes to 
            entities, manage versions of assets, manage project timelines, track project 
            progress, manage notes and reviews, manage user roles and permissions, or 
            integrate with other tools and workflows, the ShotGrid Tool API is always used."""
            )
        )
    ]
)

# ...

Full list of rules

Here are all the rules.

app.py
# ...

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        Rule("Act as a studio coordinator who is an expert with Autodesk ShotGrid"),
        Rule(
            "Your main objective is to find and update information in ShotGrid using the ShotGridTool"
        ),
        Rule(
            dedent(
                """
            For specific information about how to use ShotGrid API activities, the 
            VectorStoreTool should be used. Take the necessary time to consult the 
            VectorStoreTool to ensure the most accurate and context-aware decisions 
            when interacting with the ShotGridTool and API.
            """
            )
        ),
        Rule(
            dedent(
                """
            Always use the ShotGrid batch() method when performing multiple create, update, or delete operations in ShotGrid.
            This should be done in a single request to improve efficiency and speed. 
            Batch method uses "request_type", "entity_type", and "entity_id".
            Failure to use the batch() method in these instances will be considered a violation of the rules."""
            )
        ),
        Rule(
            dedent(
                """
            When asked to find, list, create, update, delete, retrieve details of entities, 
            update task status, update task data, list task assignments, retrieve the history of changes to 
            entities, manage versions of assets, manage project timelines, track project 
            progress, manage notes and reviews, manage user roles and permissions, or 
            integrate with other tools and workflows, the ShotGrid Tool API is always used."""
            )
        ),
    ],
)

# ...

Giving the rules to the Agent

The rules won't make any difference if you don't give them to the Agent. Let's do that now.

app.py
# ...

# Instantiate the agent
agent = Agent(
    tools=[
        DateTimeTool(off_prompt=False),
        shotgrid_tool,
        vector_store_tool
        # ReverseStringTool(off_prompt=False),
    ],
    rulesets=[shotgrid_agent_ruleset],
    stream=True
)

# ...

Try it out

Go ahead and interact with the agent. Work with different projects, create and update assets, and try and create tasks. As you are working with it, figure out where the agent isn't performing as expected, and update rules and rulesets to do exactly what you want it to do. Refine and iterate until you're happy with the results.

Clean up

Before moving on to the next section, let's remove some unused code from our app. We are no longer using the ReverseStringTool, so you can happily remove it from the imports section, and also remove it from your Agent.

Code Review

Nice work in this section - we've added rules to ensure the agent behaves as expected, using the Tools we've given it. Let's take a look at the current state of the app.

app.py
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
from dotenv import load_dotenv
from textwrap import dedent
import os

from griptape.artifacts import ListArtifact
from griptape.structures import Agent
from griptape.utils import Chat
from griptape.tools import DateTimeTool, VectorStoreTool
from griptape.drivers import (
    LocalVectorStoreDriver,
    OpenAiChatPromptDriver,
    OpenAiEmbeddingDriver,
)
from griptape.engines.rag import RagEngine
from griptape.engines.rag.modules import PromptResponseRagModule
from griptape.engines.rag.stages import ResponseRagStage
from griptape.loaders import WebLoader
from griptape.rules import Rule, Ruleset
from griptape.chunkers import TextChunker

from shotgrid_tool import ShotGridTool

load_dotenv()

# Create the vector database
vector_store_driver = LocalVectorStoreDriver(embedding_driver=OpenAiEmbeddingDriver())

# Create the query engine
rag_engine = RagEngine(
    response_stage=ResponseRagStage(
        response_modules=[PromptResponseRagModule(prompt_driver=OpenAiChatPromptDriver(model="gpt-4o-mini"))]
    ),
)

# API Documentation
shotgrid_api_urls = [
    "https://developers.shotgridsoftware.com/python-api/reference.html",
    "https://developers.shotgridsoftware.com/python-api/cookbook/usage_tips.html",
    "https://developers.shotgridsoftware.com/python-api/cookbook/attachments.html",
    "https://developers.shotgridsoftware.com/python-api/cookbook/tasks/updating_tasks.html",
    "https://developers.shotgridsoftware.com/python-api/cookbook/tasks/task_dependencies.html",
    "https://developers.shotgridsoftware.com/python-api/cookbook/tasks/split_tasks.html",
]

# Load the API documentation
artifacts = WebLoader().load_collection(shotgrid_api_urls)

# Convert to List Artifact
artifacts = ListArtifact(list(artifacts.values()))

# Chunk the documentation
artifacts = TextChunker().chunk(artifacts)

# Upsert documentation  into the vector database
namespace = "shotgrid_api"
vector_store_driver.upsert_text_artifacts({namespace: artifacts})

# Instantiate the Vector Store Tool
vector_store_tool = VectorStoreTool(
    description="Contains information about ShotGrid api. Use it to help with ShotGrid client requests.",
    vector_store_driver=vector_store_driver,
    query_params={"namespace": namespace},
    off_prompt=False,
)

SHOTGRID_URL = os.environ["SHOTGRID_URL"]
SHOTGRID_API_KEY = os.environ["SHOTGRID_API_KEY"]
SHOTGRID_SCRIPT = "Griptape API"
SHOTGRID_USER = os.environ["SHOTGRID_USER"]
SHOTGRID_PASSWORD = os.environ["SHOTGRID_PASSWORD"]


# Instantiate the tool
shotgrid_tool = ShotGridTool(
    base_url=SHOTGRID_URL,
    api_key=SHOTGRID_API_KEY,
    script_name=SHOTGRID_SCRIPT,
    user_login=SHOTGRID_USER,
    user_password=SHOTGRID_PASSWORD,
    login_method="user",
    off_prompt=False,
)

# Create the ruleset
shotgrid_agent_ruleset = Ruleset(
    name="ShotGrid Agent",
    rules=[
        Rule("Act as a studio coordinator who is an expert with Autodesk ShotGrid"),
        Rule("Your main objective is to find and update information in ShotGrid using the ShotGridTool"),
        Rule(
            dedent(
                """
            For specific information about how to use ShotGrid API activities, the 
            VectorStoreTool should be used. Take the necessary time to consult the 
            VectorStoreTool to ensure the most accurate and context-aware decisions 
            when interacting with the ShotGridTool and API.
            """
            )
        ),
        Rule(
            dedent(
                """
            Always use the ShotGrid batch() method when performing multiple create, update, or delete operations in ShotGrid.
            This should be done in a single request to improve efficiency and speed. Batch method uses "request_type", "entity_type", and "entity_id".
            Failure to use the batch() method in these instances will be considered a violation of the rules."""
            )
        ),
        Rule(
            dedent(
                """
            When asked to find, list, create, update, delete, retrieve details of entities, 
            update task status, update task data, list task assignments, retrieve the history of changes to 
            entities, manage versions of assets, manage project timelines, track project 
            progress, manage notes and reviews, manage user roles and permissions, or 
            integrate with other tools and workflows, the ShotGrid Tool API is always used."""
            )
        ),
    ],
)

# Instantiate the agent
agent = Agent(
    tools=[DateTimeTool(off_prompt=False), shotgrid_tool, vector_store_tool],
    rulesets=[shotgrid_agent_ruleset],
    stream=True,
)

# Start chatting
Chat(agent).start()

Finished

Success

Congratulations! You have created a Griptape ShotGrid Tool!

Well done, you've successfully created a Griptape Tool that allows you to connect to and work with external applications.

You have learned:

  • How Griptape Tools work.
  • How to build your own Tools.
  • How to handle user authentication with external tools.
  • How to vectorize and provide extra documentation to asset Griptape Agents.
  • How to craft Rules and Rulesets to sculpt the Agent's performance.

We hope you enjoyed this course, please head over to Discord and share your results!