TL;DR

Realtime evaluations are now available, allowing users to monitor and assess the performance of their agents as they run.

New

• Signals & Billing: The “agent steps” metric has been renamed to “Signals steps”, and the Hobby plan now supports up to 15k Signals steps. A paywall has been added for Signals.
• LLM Providers: Support for OpenAI (large language model - AI text generation) has been added, alongside improvements to mock LLM providers.
• Trace View: Users can now filter traces by custom SQL columns, enhancing debugging capabilities.

Fixes Worth Knowing

• Slack notifications now truncate long messages instead of dropping them entirely.
• The landing page “Contact Us” link now directs to Cal.com (scheduling tool).
• UI issues in the debugger and settings have been resolved.
• Usage email notifications have been improved for clarity.

Before You Upgrade

If you are using custom LLMs, ensure your llm_base_url environment variable is properly configured, as empty values will now be ignored.

TL;DR

The SQL editor now supports new tables and fixes, and PDF rendering has improved accessibility and error handling.

New

• SQL editor now includes new tables.
• PDF rendering displays sheet titles for screen readers.
• More detailed error messages are now available at the span level (individual data points).

Fixes Worth Knowing

• Labels filter in the UI now uses named tags instead of key-value pairs.
• Fixed an issue where the receiver could panic (crash) unexpectedly.
• PDF rendering now includes accessibility improvements.