Smithery is now a part of Arcade.dev! Read more in our announcement here
Instruments Python and TypeScript code with MLflow Tracing for observability...
Based on the user's project, load the appropriate guide:
references/python.mdreferences/typescript.mdIf unclear, check for package.json (TypeScript) or requirements.txt/pyproject.toml (Python) in the project.
When the target is Databricks, read references/databricks.md before editing code and configure a UnityCatalog trace location. Calling only mlflow.set_tracking_uri("databricks") and mlflow.set_experiment(...) without a trace location uses legacy workspace experiment storage; that does not satisfy a request to send traces to Databricks.
Inspect existing project or environment configuration for candidate destinations, the optional table prefix, and SQL warehouse. Before binding an experiment or provisioning UC resources, follow the schema-selection workflow in references/databricks.md: ask the user to choose an existing schema or create a new one unless they have already explicitly chosen the destination. Never select an arbitrary accessible schema. Ask for any missing required values before implementing tracing. Do not silently fall back to legacy workspace trace storage. Use legacy storage only when the user explicitly requests it.
Verify auth and the target workspace before the first run. An expired token, or a default profile pointed at the wrong workspace, drops traces silently at export with no error raised.
databricks current-user me --profile <name> # fails if auth is expired, without printing a token
python -c "import mlflow; print(mlflow.get_tracking_uri())" # confirm databricks or databricks://<name>
If auth is expired, run databricks auth login --profile <name>. Never print or persist the output of databricks auth token in an agent transcript.
Trace these operations (high debugging/observability value):
| Operation Type | Examples | Why Trace |
|---|---|---|
| Root operations | Main entry points, top-level pipelines, workflow steps | End-to-end latency, input/output logging |
| LLM calls | Chat completions, embeddings | Token usage, latency, prompt/response inspection |
| Retrieval | Vector DB queries, document fetches, search | Relevance debugging, retrieval quality |
| Tool/function calls | API calls, database queries, web search | External dependency monitoring, error tracking |
| Agent decisions | Routing, planning, tool selection | Understand agent reasoning and choices |
| External services | HTTP APIs, file I/O, message queues | Dependency failures, timeout tracking |
Skip tracing these (too granular, adds noise):
Rule of thumb: Trace operations that are important for debugging and identifying issues in your application.
After instrumenting the code, always verify that tracing is working.
Planning to evaluate your agent? Tracing must be working before you run
agent-evaluation. Complete verification below first.
mlflow.search_traces() or MlflowClient().search_traces() to check that traces appear in the experiment. If the trace is not found, try mlflow.flush_trace_async_logging() to flush the background queue.import mlflow
mlflow.flush_trace_async_logging()
traces = mlflow.search_traces(locations=["<experiment_id>"])
print(f"Found {len(traces)} trace(s)")
assert len(traces) > 0, "No traces were logged — check tracking URI and experiment settings"
trace = traces.iloc[0]
spans = mlflow.get_trace(trace.trace_id).data.spans
print(f"Trace has {len(spans)} span(s)")
for span in spans:
print(f" - {span.name} ({span.span_type})")
references/databricks.md; an experiment link alone does not open the trace.Check these in order:
search_traces() right after the run can return zero before the background queue flushes (up to a few seconds later). Call mlflow.flush_trace_async_logging() before searching, as shown above.mlflow.set_tracking_uri(...) called before the agent run? Without this, traces go to a local ./mlruns directory instead of the configured server.mlflow.autolog() or framework-specific mlflow.<framework>.autolog() raise any warnings during setup? Check stderr for patching failures.search_traces() matches the experiment active when the code ran (mlflow.get_experiment_by_name(...) to confirm).For automated validation, use agent-evaluation/scripts/validate_tracing_runtime.py.
Log user feedback on traces for evaluation, debugging, and fine-tuning. Essential for identifying quality issues in production.
See references/feedback-collection.md for:
mlflow.log_feedback()See references/production.md for:
mlflow-tracing)See references/advanced-patterns.md for:
See references/distributed-tracing.md for:
See references/databricks.md for the required Databricks default: storing traces in Unity Catalog Delta tables by binding an experiment to a UnityCatalog trace location (catalog, schema, table prefix).
Tracing is now in place. When you move on to debug or improve the agent's behavior, read the spans first. Do not fall back to reading source code and output files alone. The trace shows what each step actually received, produced, and decided, which is the evidence source that pins down where behavior went wrong.
Load the fix-agent-issue skill for this. It grounds the diagnosis in the trace, what the agent did, what it should have done, and why, before any code change, and codifies the fix as a regression test so it sticks. Reach for it as soon as you start asking why the agent produced a given output, not only when someone explicitly reports a bug.
