AI Static Alert

AI Static Alerts allow you to receive notifications when certain AI usage metrics or behaviors cross predefined thresholds. Follow the steps below to create an alert using a static threshold value.

1. Navigate to Posture Management in the FireTail platform.
2. Click the Alerting tab.
3. Click Create Alert and select AI Static as the alert type.
4. In the Name section fill out the following:
   
   • Alert Name - Enter a name for the alert.
   • ‍Enabled - Toggle on/off to activate or deactivate the alert.
5. ‍AI Filters - Apply filters to define what should be monitored by the alert. 
   
   • Click Add Filter.
   • Select the required element from the Type dropdown. Options include:
     
     • Model ID
     • Model Name
     • LLM Name
     • Model Provider
     • Max Tokens
     • Log Type
     • Input Tokens
     • Output Tokens
     • Totel Tokens
     • Stop Reason
   • Choose the appropriate Operator.
   • Enter a Value.
   • Click Submit.
6. Set trigger conditions for the alert:
   
   • ‍Whenever the number of requests is - Set the alert trigger based on whether the number of requests exceeds, falls below, or matches a specific threshold.This defines when the alert is triggered. Choose the appropriate value :
     
     • ‍Greater - Select this to get an alert when the request is greater than the threshold value.
     • ‍Greater/Equal - Select this to get an alert when the request is greater than or equal to the threshold value.
     • ‍Equal - Select this to get an alert when the request is equal to the threshold value.
     • ‍Lower/Equal - Select this to get an alert when the request is lower than or equal to the threshold value.
     • ‍Lower - Select this to get an alert when the request is lower than the threshold value.
   • ‍Than - Set the threshold value. 
   • ‍Within the last - Choose a time window for monitoring (e.g., last 6 hours). FireTail checks data at one-third intervals. For example, if you select 6 hours, checks are made every 2 hours, examining the previous 6 hours. This is displayed under the Runs this check every field.
7. In the Metrics section, you can define a specific metric to monitor for unusual activity. 
   • In the Metric name dropdown select the type of metric you want to track (log count, latency, input tokens, output tokens total tokens)
   • In the Metric stat dropdown, select a statistical operation (sum, average, min, max, and so on ).
8. In the Control Settings section you can adjust parameters to manage alert frequency and timing, this helps prevent over-notification. Adjust the following:
   • ‍After every trigger don't run this check for - After an alert is triggered, enter a 'cooldown' period during which subsequent alerts are suppressed.
   • ‍Delay evaluating the first check by - Delay the first evaluation after an alert is created, by the select timeframe, serving as a grace period.
9. ‍Notification Integration - Select the method in which you will receive your alert notification. Select a previously created integration from the dropdown, or click Create to create a new integration. 
   • ‍Note: when you create a notification integration, you can define the text and information that will be displayed when an alert is sent.
   • Learn how to Customize notifications.
10. Review the alert diagram in the Preview section.
11. Click Submit.

The alert is now created and listed under the Alerts tab. You will receive alerts via the selected notification integration when triggered.