An engineer uses code-based logging to deploy an AI agent with `mlflow.langchain.log_model(lc_model="/path/to/agent.py", ...)`. When the deployed serving endpoint receives an inference request, which sequence of events occurs?

Question

Accepted Answer

B. The serving environment restores the Python environment from logged dependencies, runs agent.py to instantiate the agent, then calls `invoke()` for each request.. In MLflow's code-based logging (Models from Code), when the serving environment starts, it restores the Python environment from logged pip requirements, runs the agent's source file (agent.py) to re-instantiate the agent, and keeps it in memory. For each inference request, the agent's `invoke()` method is called on the in-memory agent — not reloaded per request.

An engineer uses code-based logging to deploy an AI agent with `mlflow.langchain.log_model(lc_model="/path/to/agent.py", ...)`. When the deployed serving endpoint receives an inference request, which sequence of events occurs?

Related Questions

Discussion

An engineer uses code-based logging to deploy an AI agent with mlflow.langchain.log_model(lc_model="/path/to/agent.py", ...). When the deployed serving endpoint receives an inference request, which sequence of events occurs?

Related Questions

Discussion

An engineer uses code-based logging to deploy an AI agent with `mlflow.langchain.log_model(lc_model="/path/to/agent.py", ...)`. When the deployed serving endpoint receives an inference request, which sequence of events occurs?