Braintrust
Connect
Connect Braintrust from the terminal. Pass --help for the flags Braintrust accepts.
praxen integration connect --type braintrustA service token from Settings → Service tokens in Braintrust, created in the organization you want to connect, in a permission group that can read and update all projects.
Connect your Braintrust organization with a service token.
- In Braintrust, open Settings → Service tokens and create a token in the organization you want to connect. Add it to a permission group that can read and update all projects, such as Owners or Engineers.
- Paste the token into Praxen Inc. Praxen Inc. checks that the token belongs to one organization and can read and update all of its projects. Personal API keys are refused, because they follow a person into every organization they join.
Read Braintrust logs, traces, and experiments through Praxen Inc., query them with SQL, and request API changes. Changes require safety review and approval.
Praxen Inc. watches each project for changes in errors, latency, tokens, and cost.
Your Braintrust time window alerts reach Praxen Inc.: Praxen Inc. adds its webhook to each one, so a firing alert opens an issue and a recovery marks it recovered. Log alerts send to a single action, so Praxen Inc. does not attach to them.
Praxen Inc. uses these Braintrust APIs:
| Braintrust API | What it unlocks |
|---|---|
POST /api/apikey/login, POST /api/self/get_object_info | Finds the token's organization and checks that it can read and update all projects, when you connect and every 6 hours. |
GET /v1/project | Lists your projects. |
POST /btql | Runs read-only SQL over logs, experiments, and datasets: trace search, trace trees, and the error, latency, token, and cost series per project. |
GET /v1/experiment, GET /v1/experiment/{id}/summarize | Lists experiments and compares their scores with a baseline. |
GET /v1/project_automation, PATCH /v1/project_automation/{id} | Finds your time window alerts and adds Praxen Inc.'s webhook to each one, when you connect and every hour, and removes it when you disconnect. |
braintrustApi | Calls any Braintrust API; calls that change state require safety review and approval. API keys, secrets, inserts, the AI proxy, and function invoke are not available. |
Disconnecting removes Praxen Inc.'s webhook from your alerts and leaves the service token in Braintrust. Remove the token under Settings → Service tokens in Braintrust. An alert whose only action is Praxen Inc.'s keeps it, because Braintrust requires one action, and Praxen Inc. ignores its deliveries.
Disconnect
Disconnecting stops every sync and check and deletes the credentials Praxen Inc. held. Praxen Inc. removes what Braintrust's API lets it remove. The following stays until you remove it yourself:
- Braintrust service token
praxen integration disconnect <id>Tools
7Operations agents can perform in threads.