LiteLLM

LiteLLMBase

LiteLLMBase(model_name: str, system_instruction: str | None = None, tools: list[dict[str, Any]] | None = None, **kwargs)

Bases: CodeActModel

Base class for all code action models in freeact.

It uses LiteLLM for model access and cost tracking. It tracks conversation state in the history attribute. Subclasses must implement the extract_code method for extracting code from a LiteLLMResponse.

Parameters:

Name	Type	Description	Default
model_name	str	The LiteLLM-specific name of the model.	required
system_instruction	str | None	A system instruction that guides the model to generate code actions.	None
tools	list[dict[str, Any]] | None	A list of tool definitions. Some implementation classes use tools for passing code actions as argument while others include code actions directly in their response text.	None
**kwargs		Default completion kwargs used for request and feedback calls. These are merged with request and feedback specific completion kwargs where the latter have higher priority in case of conflicting keys.	{}

Attributes:

Name	Type	Description
history	list[dict[str, Any]]	List of conversation messages. User messages are either actual user queries sent via the request method or code execution results sent via the feedback method.

Source code in freeact/model/litellm/model.py

def __init__(
    self,
    model_name: str,
    system_instruction: str | None = None,
    tools: list[dict[str, Any]] | None = None,
    **kwargs,
):
    self.model_name = model_name
    self.completion_kwargs = kwargs

    self.tools = tools or []
    self.tool_names = [tool_name(tool) for tool in self.tools]

    self.history: list[dict[str, Any]] = []

    if system_instruction:
        self.history.append({"role": "system", "content": system_instruction})

extract_code abstractmethod

extract_code(response: LiteLLMResponse) -> str | None

Extracts Python code from the response.

Source code in freeact/model/litellm/model.py

@abstractmethod
def extract_code(self, response: LiteLLMResponse) -> str | None:
    """Extracts Python code from the response."""
    pass

feedback

feedback(feedback: str, is_error: bool, tool_use_id: str | None, tool_use_name: str | None, **kwargs) -> LiteLLMTurn

Constructs a new message with role tool if tool_use_id is defined, or with role user otherwise. Message content is feedback. It returns a LiteLLMTurn. After the turn is consumed, the input message and assistant message are added to the model's conversation history.

Parameters:

Name	Type	Description	Default
feedback	str	The feedback text from code execution.	required
is_error	bool	Whether the feedback represents an error condition.	required
tool_use_id	str | None	Identifier for the specific tool use instance.	required
tool_use_name	str | None	Name of the tool that was used.	required
**kwargs		Completion kwargs supported by LiteLLM.	{}

Returns:

Name	Type	Description
LiteLLMTurn	LiteLLMTurn	Represents a feedback interaction with the model where feedback is submitted by a freeact agent.

Source code in freeact/model/litellm/model.py

def feedback(
    self,
    feedback: str,
    is_error: bool,
    tool_use_id: str | None,
    tool_use_name: str | None,
    **kwargs,
) -> LiteLLMTurn:
    """Constructs a new message with role `tool` if `tool_use_id` is defined,
    or with role `user` otherwise. Message content is `feedback`. It returns
    a [LiteLLMTurn][freeact.model.litellm.model.LiteLLMTurn]. After the turn
    is consumed, the input message and assistant message are added to the model's
    conversation `history`.

    Args:
        feedback (str): The feedback text from code execution.
        is_error (bool): Whether the feedback represents an error condition.
        tool_use_id (str | None): Identifier for the specific tool use instance.
        tool_use_name (str | None): Name of the tool that was used.
        **kwargs: Completion kwargs supported by LiteLLM.

    Returns:
        LiteLLMTurn: Represents a feedback interaction with the model where `feedback`
            is submitted by a `freeact` [agent][freeact.agent.CodeActAgent].
    """
    if tool_use_id is not None:
        feedback_message = {
            "role": "tool",
            "tool_call_id": tool_use_id,
            "content": feedback,
        }
    else:
        feedback_message = {
            "role": "user",
            "content": feedback,
        }

    return LiteLLMTurn(self._stream(feedback_message, **kwargs))

request

request(user_query: str, **kwargs) -> LiteLLMTurn

Constructs a new message with role user and content user_query and returns a LiteLLMTurn. After the turn is consumed, the user message and assistant message are added to the model's conversation history.

Parameters:

Name	Type	Description	Default
user_query	str	The user's input query or request.	required
**kwargs		Completion kwargs supported by LiteLLM.	{}

Returns:

Name	Type	Description
LiteLLMTurn	LiteLLMTurn	Represents a single user interaction.

Source code in freeact/model/litellm/model.py

def request(
    self,
    user_query: str,
    **kwargs,
) -> LiteLLMTurn:
    """Constructs a new message with role `user` and content `user_query`
    and returns a [LiteLLMTurn][freeact.model.litellm.model.LiteLLMTurn].
    After the turn is consumed, the user message and assistant message are
    added to the model's conversation `history`.

    Args:
        user_query: The user's input query or request.
        **kwargs: Completion kwargs supported by LiteLLM.

    Returns:
        LiteLLMTurn: Represents a single user interaction.
    """
    user_message = {"role": "user", "content": user_query}
    return LiteLLMTurn(self._stream(user_message, **kwargs))

LiteLLM

LiteLLM(model_name: str, execution_output_template: str, execution_error_template: str, system_instruction: str | None = None, tools: list[dict[str, Any]] | None = None, **kwargs)

Bases: LiteLLMBase

A default implementation of LiteLLMBase that

• formats code execution feedback based on provided output and error templates.
• implements extract_code by extracting the first Python code block from the response text.

Parameters:

Name	Type	Description	Default
model_name	str	The LiteLLM-specific name of the model.	required
execution_output_template	str	A template for formatting successful code execution output. Must define a{execution_feedback} placeholder.	required
execution_error_template	str	A template for formatting code execution errors. Must define a {execution_feedback} placeholder.	required
system_instruction	str | None	A system instruction that guides the model to generate code actions.	None
tools	list[dict[str, Any]] | None	A list of tool definitions. Some implementation classes use tools for passing code actions as argument while others include code actions directly in their response text.	None
**kwargs		Default completion kwargs used for request and feedback calls. These are merged with request and feedback specific completion kwargs where the latter have higher priority in case of conflicting keys.	{}

Source code in freeact/model/litellm/model.py

def __init__(
    self,
    model_name: str,
    execution_output_template: str,
    execution_error_template: str,
    system_instruction: str | None = None,
    tools: list[dict[str, Any]] | None = None,
    **kwargs,
):
    super().__init__(
        model_name=model_name,
        system_instruction=system_instruction,
        tools=tools,
        **kwargs,
    )
    self.execution_output_template = execution_output_template
    self.execution_error_template = execution_error_template

extract_code

extract_code(response: LiteLLMResponse) -> str | None

Extracts the first Python code block from response.text.

Override this method to customize extraction logic.

Source code in freeact/model/litellm/model.py

def extract_code(self, response: LiteLLMResponse) -> str | None:
    """Extracts the first Python code block from `response.text`.

    **Override this method to customize extraction logic.**
    """
    return code_block(response.text, 0)

LiteLLMTurn

LiteLLMTurn(iter: AsyncIterator[str | LiteLLMResponse])

Bases: CodeActModelTurn

Source code in freeact/model/litellm/model.py

def __init__(self, iter: AsyncIterator[str | LiteLLMResponse]):
    self._iter = iter
    self._response: LiteLLMResponse | None = None

LiteLLMResponse dataclass

LiteLLMResponse(text: str, is_error: bool, token_usage: Dict[str, int] = dict(), tool_use: ToolUse | None = None, code: str | None = None)

Bases: CodeActModelResponse

code_block

code_block(text: str, index: int, **kwargs) -> str | None

Finds the index-th block matching pattern in text.

Source code in freeact/model/litellm/utils.py

def code_block(text: str, index: int, **kwargs) -> str | None:
    """Finds the `index`-th block matching `pattern` in `text`."""
    blocks = code_blocks(text, **kwargs)
    return blocks[index] if blocks else None

code_blocks

code_blocks(text: str, pattern: str = '```python\\n(.*?)```') -> list[str]

Finds all blocks matching pattern in text.

Source code in freeact/model/litellm/utils.py

def code_blocks(text: str, pattern: str = r"```python\n(.*?)```") -> list[str]:
    """Finds all blocks matching `pattern` in `text`."""
    blocks = re.findall(pattern, text, re.DOTALL)
    return [block.strip() for block in blocks]

sanitize_tool_name

sanitize_tool_name(tool_name: str) -> str

Sanitizes a tool name by replacing non-alphanumeric characters with underscores.

Source code in freeact/model/litellm/utils.py

def sanitize_tool_name(tool_name: str) -> str:
    """Sanitizes a tool name by replacing non-alphanumeric characters with underscores."""
    return re.sub(r"[^a-zA-Z0-9_-]", "_", tool_name)

tool_name

tool_name(tool: dict[str, Any]) -> str

Extracts the tool name from a tool definition.

Source code in freeact/model/litellm/utils.py

def tool_name(tool: dict[str, Any]) -> str:
    """Extracts the tool name from a [tool definition](https://platform.openai.com/docs/guides/function-calling#defining-functions)."""
    return tool["function"]["name"]