Skip to content

LLM services

Required Hotfix for Version 3.7.0

It has been determined that hotfix hotFix_ANS_370_004 is required for the LLM service to work properly on Module Suite version 3.7.0. Please ensure this hotfix is applied to your system before using the LLM service. Failure to apply this hotfix may result in unexpected behavior or errors when using LLM-related features.

To obtain and apply this hotfix, please contact AnswerModules support or refer to the official documentation for hotfix installation procedures.

Breaking Changes in 4.0

Module Suite 4.0 removes the Assistant API, legacy file helpers, and several deprecated builder methods. If you are upgrading from 3.x, review the 4.0 Migration Guide at the bottom of this page before deploying.

Integrate Large Language Models in your workflow

Introduction

Large Language Models (LLMs) are revolutionizing the way organizations process and leverage information. These sophisticated AI models, trained on vast amounts of textual data, can understand, generate, and manipulate human-like text with remarkable accuracy. As businesses increasingly deal with enormous volumes of unstructured data, integrating LLMs into existing workflows has become a game-changer for enhancing productivity, improving decision-making processes, and unlocking new insights.

Module Suite plays a crucial role in seamlessly incorporating LLMs into your enterprise content management ecosystem. By bridging the gap between your organization's content repositories and cutting-edge AI capabilities, Module Suite empowers you to:

  1. Automate content classification: Leverage LLMs to automatically categorize and classify documents, making information retrieval faster and more accurate.

  2. Enhance search functionality: Utilize natural language processing to improve search results, allowing users to find relevant information using conversational queries.

  3. Generate intelligent summaries: Create concise summaries of lengthy documents, enabling quick understanding of key points without manual review.

  4. Streamline content creation: Assist users in drafting documents, emails, and reports by providing AI-powered suggestions and completions.

  5. Facilitate knowledge discovery: Uncover hidden patterns and relationships within your content, leading to valuable insights for decision-makers.

  6. Improve data extraction: Extract relevant information from unstructured documents, making it easier to populate structured databases or forms.

By integrating LLMs through Module Suite, organizations can harness the power of AI to transform their content management processes, leading to increased efficiency, reduced manual effort, and improved overall productivity.

LLM Integration Considerations

While LLMs offer significant benefits, it's important to consider factors such as data privacy, model selection, and fine-tuning requirements when integrating them into your workflow. Module Suite provides the necessary tools and interfaces to address these considerations effectively.

Architecture and Networking

Module Suite acts as a central hub for communication between Extended ECM (xECM), various LLM API services, and local resources.

Here's an overview of how the networking and communication work:

flowchart TD
    subgraph ECM["Extended ECM (xECM)"]
        MS[Module Suite]
    end

    API1[OpenAI API]
    API2[Azure AI API]
    API3[Ollama API]

    MS <--> |Internal APIs| ECM
    MS <--> |HTTP/REST| API1
    MS <--> |HTTP/REST| API2
    MS <--> |HTTP/REST| API3

    style ECM fill:#f9f,stroke:#333,stroke-width:2px
    style MS fill:#bbf,stroke:#333,stroke-width:2px
    style API1 fill:#bfb,stroke:#333,stroke-width:2px
    style API2 fill:#bfb,stroke:#333,stroke-width:2px
    style API3 fill:#bfb,stroke:#333,stroke-width:2px

Integration with xECM

Module Suite runs directly on xECM, providing seamless access to all xECM APIs. This tight integration allows for efficient data exchange and leveraging of xECM's content management capabilities.

LLM API Communication

Module Suite implements independent communication channels to various LLM API providers, which can be: - Public internet services (e.g., OpenAI) - VPN-accessible services (e.g., Azure AI) - On-premises solutions (e.g., LLAMA3 using Ollama to expose an API)

OpenAI-Compatible API Privileged Support

Module Suite features a rich API specifically designed for OpenAI-compatible API service providers, most commonly used with OpenAI and Azure. This allows for flexible integration with different LLM services while maintaining a consistent interface.

Local Embedding Indexes

To enhance performance and maintain data control, Module Suite allows administrators to configure and create local embedding indexes. These indexes are typically used for implementing Retrieval-Augmented Generation (RAG) systems. Key points include:

  • Based on an adapted version of the Lucene open-source indexing engine
  • Requires sending text chunks to the LLM API service provider for embedding computation
  • Does not store entire documents outside your organization
  • Provides full control over chunking policies and methodologies

Permission Considerations

When implementing local embedding indexes, it's crucial to ensure that permissions are properly considered. This helps maintain data security and access control in line with your organization's policies. We will explore this in detail in the following sections.

Typical Communication Sequence

Below is a diagram illustrating a typical communication sequence when using Module Suite with xECM and an LLM API service for implementing a RAG:

sequenceDiagram
    participant User
    participant xECM
    participant Module Suite UI Widget
    participant Module Suite (Script Engine)
    participant LocalIndex
    participant LLMAPI

    User->>xECM: Request content
    xECM->>Module Suite UI Widget: Pass request
    Module Suite UI Widget->>Module Suite (Script Engine): Pass request and history
    Module Suite (Script Engine)->>LocalIndex: Query local index
    LocalIndex-->>Module Suite (Script Engine): Return relevant chunks coordinates
    Module Suite (Script Engine)->>xECM: Retrive relevant chunks (context)
    xECM-->>Module Suite (Script Engine): Returns relevant chunks
    Module Suite (Script Engine)->>LLMAPI: Send prompt with context
    LLMAPI-->>Module Suite (Script Engine): Return LLM response
    Module Suite (Script Engine)->>Module Suite UI Widget: Process and format response
    Module Suite UI Widget->>xECM: Render result
    xECM->>User: Display result

Service Provider Support in Module Suite

Module Suite offers extensive support for various LLM API providers, with a focus on OpenAI-compatible APIs and limited support for other providers. Below is a detailed breakdown of the supported features for each provider type.

OpenAI API Providers (OpenAI and Microsoft Azure AI)

Module Suite provides comprehensive support for OpenAI-compatible APIs, including those from OpenAI itself and Microsoft Azure AI. The following features are supported:

  1. Chat Completion
  2. Text Completion
  3. Function Invocation
  4. Vision (image analysis and processing)
  5. Text-to-Speech
  6. Speech-to-Text
  7. Embeddings
  8. Fine-tuning
  9. Moderation
  10. Responses API
  11. Vector Stores
  12. File Management
  13. Batch Processing
  14. Evaluations (Evals)

This wide range of supported features allows for versatile integration of AI capabilities into your Extended ECM workflows, enabling tasks from simple text generation to complex multimodal interactions.

Ollama API Support

For Ollama-based API providers, Module Suite currently offers limited but essential support:

  1. Embeddings
  2. Chat Completion

While more restricted than the OpenAI API support, these features still allow for crucial functionalities such as semantic search and conversational AI interactions using on-premises or self-hosted models.

graph TD
    A[Module Suite API Support] --> B[OpenAI APIs]
    A --> C[Ollama APIs]

    B --> D[Chat Completion]
    B --> E[Text Completion]
    B --> F[Function Invocation]
    B --> G[Vision]
    B --> H[Text-to-Speech]
    B --> I[Speech-to-Text]
    B --> J[Responses API]
    B --> K[Embeddings]
    B --> L[Fine-tuning]
    B --> M[Moderation]
    B --> P[Vector Stores]
    B --> Q[File Management]
    B --> R[Batch Processing]
    B --> S[Evaluations]

    C --> N[Embeddings]
    C --> O[Chat Completion]

    style B fill:#f9f,stroke:#333,stroke-width:2px
    style C fill:#bbf,stroke:#333,stroke-width:2px
    style N stroke:#333,stroke-width:2px
    style O stroke:#333,stroke-width:2px

API Support Evolution

The landscape of LLM APIs is rapidly evolving. Module Suite's API support is regularly updated to include new providers and features. Always refer to the latest documentation for the most up-to-date information on supported APIs and features.

Choosing the Right API Provider

When selecting an API provider for your Module Suite implementation, consider the following factors:

  • Feature requirements: Assess which AI capabilities are crucial for your use case.
  • Data privacy and compliance: Determine if you need to keep data on-premises or if cloud-based solutions are acceptable.
  • Performance needs: Evaluate the response times and throughput required for your applications.
  • Cost considerations: Compare pricing models of different providers, especially for high-volume usage.
  • Integration complexity: Consider the ease of integration with your existing infrastructure.

Components of the LLM Service Integration

Module Suite provides a comprehensive set of components to enable seamless integration with LLM services. These components work together to offer a robust and flexible AI-enhanced experience within the Extended ECM environment. Let's explore each of these components:

Content Script

Module Suite features a set of dedicated extension package to support LLM API integration.

OpenAI Extension Package Service

The OpenAI service is a dedicated Content Script extension package service specifically designed for OpenAI-compatible API integrations. Key features include:

  • Multi-profile support for flexible configuration
  • Comprehensive implementation of OpenAI's API features
  • Optimized for use with OpenAI and Azure AI services

LLM Extension Package Service

The LLM service is a more general-purpose Content Script extension package service for integrating various LLM providers. Its features include:

  • Multi-profile configuration for supporting different LLM services
  • Currently focused on Ollama API support
  • Extensible architecture for future LLM provider integrations

Widgets

Smart Pages Widget (named CARL)

This Smart Pages widget brings AI-powered capabilities directly into the Extended ECM user interface. This widget:

  • Provides an interactive AI assistant interface within Smart Pages
  • Leverages the power of LLM models for various tasks
  • Enhances user productivity by offering AI-assisted functionalities

Beautiful WebForm Widget (named CARL)

This Beautiful WebForm Widget extends AI capabilities to WebForms, allowing for:

  • AI-enhanced form interactions
  • Intelligent form filling assistance
  • Dynamic content generation based on form context

Services

Content Script Service (named carl)

This Content Script Service acts as the backend engine for LLM-related functionalities, providing:

  • Integration between widgets and LLM services
  • Business logic for processing AI requests and responses
  • Customizable workflows for AI-assisted operations

Code Snippets

Content Script Snippets

Module Suite includes several Content Script snippets that:

  • Facilitate quick implementation of common LLM-related tasks
  • Provide reusable code for developers to extend AI functionalities
  • Demonstrate best practices for integrating AI capabilities into Content Scripts

CARL (Content Server Artificial intelligence Resource and Liaison)

CARL is a feature of Module Suite, introduced with version 3.5, that implements a Content Script co-pilot based on the integration with GPT family models.

CARL Integration in Content Script Editor

When enabled in the Base Configuration, CARL integrates directly into the Content Script editor, offering:

  • AI-assisted code completion and suggestions
  • Context-aware help and documentation
  • Intelligent debugging assistance

CARL Beta Status

CARL is currently in beta. As a beta feature, it may undergo changes and improvements in future releases. Users are encouraged to provide feedback to help shape its development.

graph TD
    A[Module Suite] --> B[OpenAI Service]
    A --> C[LLM Service]
    A --> D[LLM Integration Features]
    D --> E[Smart Pages Widget CARL]
    D --> F[Beautiful WebForm Widget CARL]
    D --> G[Content Script Service carl]
    A --> H[Content Script Snippets]
    A --> I[CARL Co-pilot Feature]
    I --> J[Content Script Editor Integration]

    B --> K[OpenAI/Azure AI]
    C --> L[Ollama]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style D fill:#bbf,stroke:#333,stroke-width:2px
    style I fill:#fbb,stroke:#333,stroke-width:2px
    style B stroke:#333,stroke-width:2px
    style C stroke:#333,stroke-width:2px

Integration Use Cases

Module Suite offers various capabilities for integrating AI-powered functionalities into your Extended ECM environment. Let's explore common use cases and how to implement them.

Chat Completion

Chat completion allows you to create interactive, context-aware conversations with an AI assistant. This functionality is valuable for implementing:

  • Intelligent chatbots for user support
  • Virtual assistants for guided ECM operations
  • Interactive help systems within your ECM applications
  • Natural language interfaces for complex queries or tasks

Example: Basic Chat Interaction

Here's a simple example of how to implement a chat completion interaction:

def systemPreamble = """You are C.A.R.L. (Content Server Artificial intelligence Resource and Liaison), an LLM designed to help users working with OpenText Extended ECM"""
def defaultTemperature = 0.7
def model = "gpt-4"
def maxTokens = 2000

try {
    // Our services are implemented using fluent APIs and builders to simplify their usage
    // Use the auto-completion feature of the editor (CTRL+Space) to explore available configuration options
    def reqBuilder = openai.newChatCompletionRequestBuilder()
        .model(model)
        .temperature(defaultTemperature)
        .maxTokens(maxTokens)
        .n(1) // Request only one completion (most common use case)

    def req = reqBuilder.build()

    // Instruct the agent about its purpose and constraints using a system message
    req.addChatMessage("system", systemPreamble)

    // Add user request or message
    req.addChatMessage("user", "Does xECM support the concept of metadata?")

    // Submit the request and synchronously wait for the response
    def result = openai.createChatCompletion(req)

    // Extract and output the content of the first (and only) choice
    out << result.choices[0].message.content
} catch (Exception e) {
    out << "An error occurred: " + e.getMessage()
}

This code configures the request with specific parameters such as the model to use (GPT-4), temperature setting for response randomness, and maximum token limit. The system message defines C.A.R.L.'s role, followed by a user question about xECM's metadata support. The openai.newChatCompletionRequestBuilder() method initializes the request, which is then configured and built. The addChatMessage() method is used to add both system and user messages to the conversation. Finally, the createChatCompletion() method sends the request to the OpenAI API and retrieves the response.

This example showcases the ease of use of the OpenAI service in Module Suite, allowing developers to quickly implement AI-powered chat functionalities within their Extended ECM environment. The service handles the complexities of API interaction, allowing developers to focus on crafting effective prompts and integrating the responses into their applications.

Use Autocompletion

Remember to use the auto-completion feature of the editor (CTRL+Space) to explore available configuration options when working with the request builder.

def systemPreamble = """You are C.A.R.L. (Content Server Artificial intelligence Resource and Liaison), an LLM designed to help users working with OpenText Extended ECM"""
def defaultTemperature = 0.7
def model = "llama3"
def maxTokens = 2000
def msgID = "MyMessage"

try {
    // Use the llm service with a LangChain-based builder for Ollama
    def reqBuilder = llm.newLangChainChatCompletionRequestBuilder("ollama")
        .model(model)
        .temperature(0.0)
        .logRequestsAndResponses(true)

    // Set a custom timeout for non-OpenAI services
    reqBuilder.builder.timeout(java.time.Duration.ofSeconds(180))

    def req = reqBuilder.build()

    // Add system message to define C.A.R.L.'s role
    req.addChatMessage("system", systemPreamble)

    // Add user query
    req.addChatMessage("user", "Does xECM support the concept of metadata?")

    // Submit the request and wait for the response
    def result = llm.createChatCompletion(req)

    // Output the AI-generated response
    out << result.choices[0].message.content
} catch (Exception e) {
    out << "An error occurred: " + e.getMessage()
}

This example illustrates the flexibility of Module Suite's AI integration.

Key points to note:

  • The llm service is used instead of a specific provider service, allowing for more generic implementations.
  • The newLangChainChatCompletionRequestBuilder method is used with "ollama" as the provider.
  • LangChain is utilized for integration with non-OpenAI services, offering additional configuration options like custom timeouts.
  • The overall structure of setting up the request, adding messages, and retrieving the response remains similar to the previous example.

By using the generic llm service, you can easily switch between different AI providers or models while maintaining a consistent implementation structure. This flexibility allows you to choose the most suitable AI backend for your specific use case or requirements.

Different models different results

Experiment with different models and providers to find the best balance of performance, cost, and capabilities for your ECM AI integration needs.

Chat Completion (continued)

Example: Streaming Chat Completion

Streaming chat completion allows for real-time delivery of AI-generated responses, enhancing the responsiveness of your AI-powered applications. This is particularly useful for:

  • Creating more interactive and dynamic user experiences
  • Implementing typing-like effects in chatbots
  • Handling long-form content generation without waiting for the entire response

Here's an example of how to implement streaming chat completion:

def systemPreamble = """You are C.A.R.L. (Content Server Artificial intelligence Resource and Liaison), an LLM designed to help users working with OpenText Extended ECM"""
def defaultTemperature = 0.7
def model = "gpt-4"
def maxTokens = 2000
def msgID = "MyMessage"

try {
    def reqBuilder = openai.newChatCompletionRequestBuilder()
        .model(model)
        .temperature(defaultTemperature)
        .n(1)

    def req = reqBuilder.build()
    req.addChatMessage("system", systemPreamble)
    req.addChatMessage("user", "Does xECM support the concept of metadata?")

    def iterator = 0
    result = openai.streamChatCompletion(req, { block ->
        // This closure is invoked with streaming chunks as they become available
        def map = [
            content: block.choices?[0]?.message?.content,
            role:    block.choices?[0]?.message?.role,
            finishReason: (block.choices?[0]?.message?.content != null) ? block.choices?[0]?.finishReason : 'stop'
        ]
        log.debug("GOT {}", map) // Log for debugging
        cache.put(msgID + "_" + iterator++, 500, map)
    })

    out << result.choices
} catch (Exception e) {
    out << "An error occurred: " + e.getMessage()
}

Key points about this streaming implementation:

The streamChatCompletion method is used instead of createChatCompletion. A closure is provided as the second argument to streamChatCompletion. This closure is called for each chunk of the response as it becomes available. The closure implements a producer-consumer pattern:

It acts as the producer, storing each chunk in the memcache service. The UI can act as the consumer, retrieving chunks from the memcache to display in real-time.

Each chunk is stored in the memcache with a unique key combining the msgID and an incrementing iterator. The finishReason is tracked to determine when the response is complete.

This approach allows for more responsive AI interactions, as the UI can start displaying the response before it's fully generated. It's particularly useful for longer responses or when you want to create a more dynamic, "typing" effect in your AI interface.

Use debouncing

When implementing the UI for streaming responses, consider using techniques like debouncing to balance between real-time updates and performance.

Additional considerations: Producer-Consumer Pattern

The streaming chat completion implementation uses a producer-consumer pattern to handle real-time data flow. Here's a sequence diagram illustrating this process:

Default Consumer Service

A default implementation of the consumer service, named "carl", is provided as a Content Script Service.

You can find it at: Content Script Volume:CSTools:CARL:CSServices

This service helps manage the consumption of streamed chat completion responses, making it easier to implement the consumer side of the producer-consumer pattern in your applications.

Sequence Diagram

Here's a sequence diagram illustrating the producer-consumer process, including the default "carl" service:

sequenceDiagram
    participant U as UI
    participant S as Script
    participant A as AI Service
    participant C as Memcache

    U->>S: Initiate chat (msgID)
    S->>A: streamChatCompletion request
    activate A
    loop For each chunk
        A-->>S: Stream chunk
        S->>C: Store chunk (msgID_i)
        S->>S: Log chunk
    end
    deactivate A
    S-->>U: Completion notification

    loop Until all chunks received
        U->>C: Request chunk (msgID_i)
        C-->>U: Return chunk
        U->>U: Update display
    end
This diagram illustrates the following sequence:

  1. The UI initiates the chat, providing a unique msgID.
  2. The script sends a streaming chat completion request to the AI service.
  3. As the AI service generates the response:
  4. It streams chunks of the response back to the script.
  5. The script stores each chunk in the memcache with a unique key (msgID_i, where i is an incrementing counter).
  6. The script also logs each chunk for debugging purposes.
  7. Once all chunks are received, the script notifies the UI that the completion is finished.
  8. The UI then repeatedly requests chunks from the memcache using the msgID and incrementing counter.
  9. As the UI receives each chunk, it updates the display, creating a real-time streaming effect.

This pattern allows for efficient handling of large responses and provides a smooth, responsive user experience. The memcache serves as a buffer between the AI service's output rate and the UI's consumption rate, ensuring that no data is lost and that the UI can process the response at its own pace.

Adapt it as needed

The actual implementation may vary depending on your specific UI needs.This diagram represents a general approach that can be adapted to various technical environments.

Benefits of Using the Default "carl" Service

When implementing streaming chat completion in your applications, consider using the provided "carl" service to streamline your development process and ensure robust handling of streamed responses.

Using the provided "carl" service offers several advantages:

  1. Simplified Implementation: You don't need to write your own consumer logic, reducing development time and potential errors.
  2. Consistency: The service ensures a consistent approach to consuming streamed responses across your applications.
  3. Optimization: The service may include optimizations for efficient retrieval and assembly of chunked responses.
  4. Maintenance: As part of Module Suite, the service will be maintained and updated, ensuring compatibility with future versions.

To use the "carl" service in your applications, you can call it from your UI code after initiating a streaming chat completion. The service will handle the retrieval and assembly of the chunked response, allowing you to focus on displaying the results to the user.

Function Calling

Function calling allows the AI to interact directly with Content Server, performing actions or retrieving information as needed. This powerful feature enables the AI to manipulate content and execute operations within the Extended ECM environment.

Example: Creating Folders Using AI

In this example, we'll demonstrate how to use function calling to create folders in Content Server based on natural language input.

def systemPreamble = """You are C.A.R.L. (Content Server Artificial intelligence Resource and Liaison), an LLM designed to help users working with OpenText Extended ECM"""
def defaultTemperature = 0.7
def model = "gpt-4"
def maxTokens = 2000
def msgID = "MyMessage"

try {
    def reqBuilder = openai.newChatCompletionRequestBuilder()
        .model(model)
        .temperature(defaultTemperature)
        .n(1)

    def req = reqBuilder.build()
    req.addChatMessage("system", systemPreamble)
    req.addChatMessage("user", "Create a folder for each month of the year in ${self.parent.ID}")

    // Define the function for creating a folder
    def func = openai.newChatFunctionBuilder()
        .name("createFolder")
        .description("Create a folder in the given space (identified by its ID)")
        .build()

    func.addStringParameter("folderName", "The name of the folder", true)
    func.addNumberParameter("parentID", "Parent Space Identifier", true)

    // Implement the function executor
    func.executor = { jsonArguments ->
        try {    
            def slurper = new JsonSlurper()
            def args = slurper.parseText(jsonArguments)
            def newNode = docman.createFolder(docman.getNode(args.parentID as Long), args.folderName)
            return "Created <a data-ampw='am-action' data-action='am_goTo' data-params='${newNode.ID}' data-toggle='click' href='#'>${newNode.name}</a>"
        } catch(e) {
            log.error("Unable to handle the request", e)
            return "Something went wrong"
        }
    }

    req.setFunctions([func])

    result = openai.createChatCompletion(req)
    out << result.choices[0].message
} catch (Exception e) {
    log.error("An error occurred ", e)
    out << "An error occurred: " + e.getMessage()
}
def systemPreamble = """You are C.A.R.L. (Content Server Artificial intelligence Resource and Liaison), an LLM designed to help users working with OpenText Extended ECM"""
def defaultTemperature = 0.7
def model = "gpt-4"
def maxTokens = 2000
def msgID = "MyMessage"

try {
    def reqBuilder = openai.newChatCompletionRequestBuilder()
        .model(model)
        .temperature(defaultTemperature)
        .n(1)

    def req = reqBuilder.build()
    req.addChatMessage("system", systemPreamble)
    req.addChatMessage("user", "Create a folder for each month of the year in ${self.parent.ID}")

    // Define the function for creating a folder
    def func = openai.newChatFunctionBuilder()
        .name("createFolder")
        .description("Create a folder in the given space (identified by its ID)")
        .build()

    func.addStringParameter("folderName", "The name of the folder", true)
    func.addNumberParameter("parentID", "Parent Space Identifier", true)

    // Implement the function executor
    func.executor = { jsonArguments ->
        try {    
            def slurper = new JsonSlurper()
            def args = slurper.parseText(jsonArguments)
            def newNode = docman.createFolder(docman.getNode(args.parentID as Long), args.folderName)
            return "Created <a data-ampw='am-action' data-action='am_goTo' data-params='${newNode.ID}' data-toggle='click' href='#'>${newNode.name}</a>"
        } catch(e) {
            log.error("Unable to handle the request", e)
            return "Something went wrong"
        }
    }

    req.setFunctions([func])

    result = openai.streamChatCompletion(req, { block->  //This is an asyncrnous method, meaning that the script execution
                                                        // won't wait for the completion to be terminated. The method accept 
                                                        // as its second parameter a closure that will be invoked with the streaming
                                                        // chunks as they become available

        def map = [
            content:block.choices?[0]?.message?.content,
            role:   block.choices?[0]?.message?.role,
            finishReason:(block.choices?[0]?.message?.content != null)?block.choices?[0]?.finishReason:'stop'
        ]
        log.debug("GOT {}", map) // Let's also print it into the log file for debugging
        cache.put(msgID+"_"+iterator++, 500, map) 

    })
    out << result*.content
} catch (Exception e) {
    log.error("An error occurred ", e)
    out << "An error occurred: " + e.getMessage()
}

example 2

This example demonstrates several key concepts:

  • Function Definition: We define a createFolder function using the newChatFunctionBuilder(). This function takes two parameters: folderName and parentID.

  • Function Implementation: The executor closure contains the actual implementation of the function. It uses the Content Server API (docman) to create a new folder.

  • AI Integration: The function is added to the chat completion request, allowing the AI to call it when necessary.

  • Natural Language Processing: The user's request to "Create a folder for each month of the year" is interpreted by the AI, which then calls the createFolder function multiple times to fulfill the request.

  • Error Handling: The implementation includes error handling to manage potential issues during folder creation.

  • Interactive Response: The function returns an HTML link that allows users to navigate directly to the newly created folder.

Benefits of Function Calling

Function calling in Module Suite offers several advantages:

  1. Direct Interaction: The AI can perform actions directly in Content Server, bridging the gap between natural language requests and system operations.

  2. Flexibility: You can define custom functions to extend the AI's capabilities, tailoring it to your specific ECM needs.

  3. Safety: By defining specific functions, you control what actions the AI can perform, ensuring security and preventing unintended operations.

  4. Complex Operations: You can implement complex workflows by combining multiple function calls based on user requests.

Error handling

When implementing functions, ensure proper error handling and logging to maintain system stability and aid in troubleshooting.

Functions everywhere

Consider implementing additional functions for common ECM tasks, such as searching for documents, updating metadata, or initiating workflows. This can greatly enhance the AI's utility within your Extended ECM environment.

Document Assembly

Document assembly is a powerful use case that combines AI-generated content with document creation and manipulation within the Extended ECM system. This approach allows for the automatic generation of documents based on user requests, leveraging AI to create content and Module Suite's capabilities to assemble and store the document.

Example: Create a presentation letter in Word

def systemPreamble = "You are D.O.C.S. (Document Organizing and Creation System) an AI agent tasked to support users in creating documents.The content of the document MUST always be passed to the 'createDocument' function as valid X-HTML code (example: <b>DEMO</b>). Always wrap the content in a single 'div' element. Avoid the usage of HTML tag such as (BR). The font family for every text must be: font-family: Poppins, sans-serif tables must have a borders: 1px solid #fff;"

data = [:]
data.company = [name:"CreativeAnswer SA", address:"Via Penate 4, 6850 Mendrisio Switzerland", description:"""
At CreativeAnswer, we are a united marketing agency & software house blending creative brilliance with tech mastery.

We have developed a unique human-AI approach to help you create standout, cost-effective AI-driven marketing solutions with a competitive edge.

This isn't your typical AI tool. We don't settle for generic AI apps, nor do we just churn out text or images. Our creative and tech teams work in perfect sync, shaping ideas with proprietary AI, adapting and customizing them to your needs, and delivering the personal experience your customers want… effortlessly.

We bring you the most valuable approach to GenAI marketing, tailored exclusively for you. Just give it a try.

CreativeAnswer is a proud part of:
- AnswerModules Group, the award-winning Swiss tech company delivering personalized ECM software to 120+ enterprises, including 10 Global Fortune's 500
- Microsoft for Startups Founders Hub
"""]

data.user    = [name:"Patrick Vitali", role:"CTO"]

def defaultTemperature = 0.7
def model = "gpt-4o"
def maxTokens = 2000
def msgID = "MyMessage"
try{

    def reqBuilder  = openai.newChatCompletionRequestBuilder()
    .model(model)
    .temperature(defaultTemperature)
    .n(1) 

    def req = reqBuilder.build()

    func =  llm.newOpenAIChatFunctionBuilder().name("createDocument")
            .description("Given a title, creates a new document, matching user's requests with content generated by the AI agent").build()

    func.parameters = []
    func.addStringParameter("title",   "The document title", true)
    func.addStringParameter("content", "The document's content. The content must be a valid  X-HTML code (example: <b>DEMO</b>)", true)

    func.executor = { jsonArguments ->
        try{
            def slurper = new JsonSlurper()
            def args = slurper.parseText(jsonArguments)

            if(!args.title.endsWith('.docx')){
                args.title = "${args.title}.docx"
            }

            docTemplate = docman.getNodeByPath("CreativeAnswer:Marketing:Corporate Identity:Template.docx")
            newNode = docman.getNodeByName(docTemplate.parent, args.title)

            //Load contents from a Docx file for processing
            def doc = docx.loadWordDoc(docTemplate)

            //Creates a temporary resouce
            def res = docman.getTempResource("out", "docx") 

            //Updates the custom-xml databinding based on the OpenDoPE standard. Since: 2.3.0
            //Notice the combined used of multiple Content Script services: docx, docman, html, cache
            def xml = ""
            if(newNode){
                //Update xml with custom values
                xml = """<doc>
                            <content>&lt;div&gt;${html.escapeXML(html.htmlToXhtml(args.content))}&lt;/div&gt;</content>
                            <docID>${newNode.ID}</docID>
                        </doc>"""
                doc.updateOpenDoPEBindings(xml, true, true)
                doc.save(newNode)
            }else{
                newNode = doc.save(docTemplate.parent, args.title)
                xml = """<doc>
                            <content>&lt;div&gt;${html.escapeXML(html.htmlToXhtml(args.content))}&lt;/div&gt;</content>
                            <docID>${newNode.ID}</docID>
                        </doc>"""
                doc.updateOpenDoPEBindings(xml, true, true)
                doc.save(newNode)
            }
            ulrEditTemplate = "${url}?func=Edit.Edit&nodeid=${newNode.ID}&uiType=1&viewType=1&nexturl=${params.nextUrl}" 

            return "The requested document has been created with title: ${args.title}. <br>You can finalize the draft using the link that follows:<a target='_blank' href='${ulrEditTemplate}'>Edit</a>"

        }catch(e){
            log.error("Error ",e)
            return "Something went wrong "+e.message
        }
    }

    req.setFunctions([func])

    req.addChatMessage("system", //one out: system, assistant, user
                    systemPreamble) //We instruct the agent about its purpose and constraints using system messages at the beginning
    //of the chat
    req.addChatMessage("system", "When creating documents you should consider the following context, expressed in the form of a JSON structure: ${JsonOutput.toJson(data)}")

    //The user request   
    req.addChatMessage("user", "Create a presentation letter to be sent to John Doe, ACME's CMO") //Add user request or message

    result = openai.streamChatCompletion(req, { block->
        def map = [
            content:block.choices?[0]?.message?.content,
            role:   block.choices?[0]?.message?.role,
            finishReason:(block.choices?[0]?.message?.content != null)?block.choices?[0]?.finishReason:'stop'
        ]
        log.debug("GOT {}", map) // Let's also print it into the log file for debugging
        cache.put(msgID+"_"+iterator++, 500, map) 

    })
    out << result*.content

} catch (Exception e) {
    out << "An error occurred: " + e.getMessage()
}

example 2

This example demonstrates how to create a presentation letter using AI-generated content and Module Suite's document manipulation features.

Key components of this implementation include:

  1. AI Content Generation: The AI is instructed to create document content based on user requests and provided context.

  2. Document Creation Function: A specialized function is made available to the AI for creating documents within the ECM system.

  3. Content Formatting: The AI is guided to provide content in a specific format (X-HTML) to ensure compatibility with the document creation process.

  4. Context Provision: Relevant data, such as company and user information, is provided to the AI to inform the content generation process.

  5. Document Template Usage: The implementation utilizes a predefined document template as a base for new documents.

  6. Dynamic Document Update: The system checks for existing documents with the given title, updating if found or creating new ones if not.

  7. OpenDoPE Standard: The implementation leverages the OpenDoPE standard for XML data binding, allowing for dynamic content insertion into documents.

  8. Streaming Response: The AI's response is streamed, enabling real-time updates and potentially faster response times for large documents.

Benefits of AI-Assisted Document Assembly

  1. Efficiency: Automates the process of creating standard documents, saving time and reducing manual effort.

  2. Consistency: Ensures that created documents follow a consistent structure and style.

  3. Contextual Relevance: By providing context to the AI, the generated content can be highly relevant and personalized.

  4. Flexibility: Can be adapted for various document types and use cases within the ECM system.

  5. Integration: Seamlessly combines AI capabilities with existing ECM features and document templates.

Additional considerations: Implementation details

When implementing AI-assisted document assembly:

  • Ensure that the AI is properly constrained to generate content in the required format and style.
  • Regularly validate the generated content to maintain quality and consistency.
  • Consider implementing error handling and logging to manage potential issues during document creation.
  • Design the user interface to provide clear feedback on the document creation process and results.

Note

The specific implementation details may vary depending on your ECM environment and chosen AI service. Consult your Module Suite documentation for precise integration steps.

Tip

Consider expanding this approach to other document types, such as reports, contracts, or proposals. You can create specialized templates and AI instructions for each document type to further streamline your document creation processes.

Embedding Index Generation

Embedding index generation is a crucial step in creating AI-powered search and retrieval systems within your ECM environment. This use case demonstrates how Module Suite can generate embedding indexes for documents, enabling advanced semantic search capabilities.

Example: Indexing a single document

The provided implementation showcases the generation of an embedding index for documents in the ECM system. A key feature of this approach is its flexibility in handling different document types and chunking methods.

    try {
        // Retrieve the document node from the Enterprise Workspace
        def document = docman.getNodeByPath(docman.getEnterpriseWS(), "Documents:MS_3_7_0_Manual.pdf")
        def docID = document.ID as String
        def lastVersion = document.lastVersion
        def list = null
        def sources = [:]

        // Prepare document metadata for indexing
        // We store metadata instead of raw text for efficiency:
        // 1. Document ID
        // 2. Document type (pdf or raw)
        // 3. Chunk dimension (e.g., page number or word count)
        // 4. Chunk identifier
        if (lastVersion.mimeType == "application/pdf") {
            // For PDF documents: Extract text page by page
            def content = pdf.getTextForPages(document)
            sources = content.collectEntries { entry -> 
                [("${docID}_pdf_1_${entry.key}"): entry.value] 
            }
        } else {
            // For non-PDF documents: Split content into word-based chunks
            def content = docman.getContentAsRawText(document)
            def chunkSize = 100 // Number of words per chunk (adjustable)
            def index = 0
            sources = content.split("\\s").toList().collate(chunkSize) *.join(" ").collectEntries {
                val -> [("${docID}_raw_${chunkSize}_${index++}"): val]
            }
        }

        // Process sources in batches of 10
        list = sources.entrySet().toList().findAll { it.value }.collate(10)
        def chunk = 0

        list.each { subEnties ->
            log.debug("Processing Chunk ${chunk} {}", subEnties *.value)
            try {
                // Generate embeddings for the current batch
                def req = llm.newOpenAIEmbeddingRequestBuilder()
                            .model("text-embedding-ada-002")
                            .input(subEnties *.value)
                            .build()
                def embeddings = llm.createEmbeddings(req).data

                // Build vectorial index using embeddings and metadata
                llm.newCSVectorialIndexManager().buildIndex(docID, embeddings, subEnties *.key)
            } catch (e) {
                // Log errors for individual chunks without stopping the entire process
                log.error("Unable to process chunk ${chunk}", e)
            }
            log.debug("End processing Chunk ${chunk++}")
        }
    } catch (e) {
        // Log any overall process errors
        log.error("An error occurred during index generation.", e)
    }
try {
    // Retrieve the document node using the provided path
    def document = docman.getNode(2031317 )
    def docID = document.ID as String
    def lastVersion = document.lastVersion
    def list = null
    def sources = [:]

    if (lastVersion.mimeType == "application/pdf") {
        // If the document is a PDF, extract text for each page and create embeddings
        def content = pdf.getTextForPages(document)
        sources = content.collectEntries { entry -> [("${docID}_pdf_1_${entry.key}"): entry.value] }
    } else {
        // If not a PDF, split the content into chunks and generate embeddings
        def content = docman.getContentAsRawText(document)
        def chunkSize = 100 // The number of words each chunk should be made of
        def index = 0
        sources = content.split("\\s").toList().collate(chunkSize) *.join(" ").collectEntries {
            val -> [("${docID}_raw_${chunkSize}_${index++}"): val]
        }

    }

    list = sources.entrySet().toList().findAll { it.value }.collate(10)
    def chunk = 0
    list.each { subEnties ->
        log.debug("Processing Chunk ${chunk} {}", subEnties *.value)
        try {
            // Create an embedding request for the chunk

            model = llm.newCSEmbeddingModelBuilder("ollama")
            .modelName("llama3")
            .logRequests(true)
            .logResponses(true)
            .build();

            def embeddings = []
            subEnties.each{
                embeddings << llm.newCSEmbedding( model.embed(it.value) )
            }

            // Build a vectorial index for the document using the embeddings
            llm.newCSVectorialIndexManager().buildIndex(docID, embeddings, subEnties*.key)
        } catch (e) {
            log.error("Unable to process chunk ${chunk}", e)
        }
        log.debug("End processing Chunk ${chunk++}")
    }
} catch (e) {
    log.error("An error occurred. ", e)
}

Key Features

  1. Document Type Flexibility: The system can handle various document types, with specific handling for PDF files and a general approach for other text-based documents.

  2. Adaptive Chunking: The chunking strategy adapts based on the document type:

  3. For PDFs: Text is extracted page by page.
  4. For other documents: Content is split into chunks based on a specified word count.

  5. Metadata-Rich Indexing: Instead of storing raw text, the index stores metadata that can be used to retrieve the original text:

  6. Document ID
  7. Document type (PDF or raw)
  8. Chunk dimensions (e.g., page number for PDFs, word count for other documents)
  9. Chunk identifier

  10. Configurable Chunk Size: For non-PDF documents, the chunk size (number of words per chunk) can be easily adjusted.

  11. Batch Processing: Embeddings are generated and indexed in batches to manage resource usage efficiently.

Additional Considerations: Implementation Highlights

This implementation demonstrates several important concepts:

  1. Document Retrieval: The system retrieves documents from the ECM using the document management API.
  2. Type-Specific Processing: Different processing logic is applied based on the document's MIME type.
  3. Flexible Text Extraction:

    • For PDFs, text is extracted page by page using a PDF processing service.
    • For other documents, raw text is extracted and split into word-based chunks.
  4. Metadata Generation: Each chunk is associated with a unique identifier that includes:

    • Document ID
    • Document type (PDF or raw)
    • Chunk size or page number
    • Chunk index
  5. Embedding Generation: The system uses an LLM service to generate embeddings for each chunk of text.

  6. Index Building: A vectorial index is built using the generated embeddings and associated metadata.

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is a powerful technique that combines the strengths of large language models with the ability to access and utilize specific, up-to-date information from your Extended ECM system. This approach significantly enhances the accuracy and relevance of AI-generated responses by grounding them in your organization's actual content.

Example: Using a all the documents in a folder as a Knowledge Base

In this use case, we demonstrate how to implement a RAG system within the Module Suite. The system performs the following key steps:

  1. Embeds the user's question using a specified embedding model.
  2. Searches a pre-built set of vector indexes of your ECM content for relevant information.
  3. Retrieves the actual text content of the most relevant chunks.
  4. Incorporates this retrieved context into the prompt for the large language model.
  5. Generates a response using the LLM, now informed by the relevant context.

An index for each document

This example assumes that an embedding index has been created for each document

// Configuration parameters for the RAG system
def numberOfSnippetToConsider = 1      // Number of relevant snippets to use for context
def distanceSnippetTrashold = 0.8      // Cosine similarity threshold for relevant snippets
def embeddingModel = "text-embedding-ada-002"  // Model used for generating embeddings
def defaultTemperature = 0.8           // Temperature for LLM response generation
def model = "gpt-4"                    // LLM model to use for generating responses
def maxTokens = 2000                   // Maximum number of tokens in the LLM response
def msgID = "msgID"                    // Unique identifier for the message (used in streaming scenarios)

// Function to retrieve relevant context based on the user's question
def getContext = { String prompt ->
    // Get all document IDs in the KB folder (assuming documents are of subtype 144)
    def indexes = docman.getNodeByPath("KB").childrenFast.findAll{it.subtype == 144}.collect{it.ID as String}

    // Generate embedding for the user's prompt
    def req = openai.newEmbeddingRequestBuilder().model(embeddingModel).input([prompt]).build()
    def promptEmbedding = openai.createEmbeddings(req).data

    // Search the vector indexes for similar content
    def context = llm.newCSVectorialIndexManager().getIndexesSearcher(*indexes)
                    .query(promptEmbedding[0], 100, 100)
                    .findAll{it.score >= distanceSnippetTrashold}

    def textContext = ""
    if(context) {
        // Parse the index key to extract document information
        // Format: DOCID_identifier_chunkDimension_chunkNumber
        def (docID, extMode, dimension, chunkNum) = context[0].text.split("_")

        // Retrieve the actual text content based on the extraction mode
        switch(extMode) {
            case "raw":
                // For raw text, split into words and retrieve the specific chunk
                textContext = docman.getContentAsRawText(docman.getNodeFast(docID as Long))
                                .split("\\s")
                                .toList()
                                .collate(dimension as int)
                                *.join(" ")[chunkNum as int]
                break
            case "pdf":
                // For PDFs, retrieve the specific page(s)
                textContext = pdf.getTextForPages(docman.getNodeFast(docID as Long), 
                                                chunkNum as int, 
                                                (chunkNum as int) + (dimension as int))[(chunkNum as int)]
                break
        }
    }
    return textContext
}

// Main execution
def question = "Which is the biggest update on Module Suite 3.2.0 ?"

// Set up the chat completion request
def builder = openai.newChatCompletionRequestBuilder()
def req = builder.model(model)
                .user("u" + users.current.ID)
                .temperature(defaultTemperature)
                .n(1)
                .build()

// Add system message to define CARL's role
req.addChatMessage("system", "You're C.A.R.L. (Content Server Artificial intelligence Resource and Liaison) an LLM designed to help users working with OpenText Extended ECM")

// Retrieve relevant context for the question
def context = getContext(question)

// Add user message with or without context
if(context) {
    req.addChatMessage("user", """Given the following context: ${context}
Answer the user question: ${question}""")
} else {
    req.addChatMessage("user", question)
}

// Generate and output the response
result = openai.createChatCompletion(req)
out << result.choices[0].message.content

Key Components

  1. Embedding Generation: The system uses the OpenAI API to generate embeddings for the user's question.

  2. Vector Index Search: A custom CSVectorialIndexManager is used to search pre-built indexes of your ECM content.

  3. Context Retrieval: Based on the search results, the system retrieves the actual text content from your ECM documents, handling different document types (e.g., raw text, PDF) appropriately.

  4. LLM Integration: The retrieved context is incorporated into the prompt sent to the LLM (in this case, GPT-4), allowing it to generate more accurate and relevant responses.

Benefits

  1. Improved Accuracy: By grounding the LLM's responses in your actual ECM content, the system provides more accurate and relevant answers.

  2. Up-to-date Information: The system can access the latest information in your ECM, ensuring responses reflect the most current data.

  3. Customization: The RAG approach allows the AI to leverage your organization's specific knowledge and terminology.

  4. Reduced Hallucination: By providing relevant context, the likelihood of the LLM generating incorrect or fabricated information is significantly reduced.

Additional Considerations: Implementation Highlights

  • Index Management: Ensure that your vector indexes are kept up-to-date as your ECM content changes.

  • Performance Optimization: Consider caching frequently accessed content or embeddings to improve response times.

  • Privacy and Security: Be mindful of what content is being sent to external LLM services and ensure compliance with your organization's data policies.

This implementation demonstrates a basic RAG system that can be further customized and optimized based on your specific needs and use cases within the Extended ECM environment.

Close the loop

Consider implementing a feedback mechanism to continuously improve the relevance of retrieved contexts and the quality of generated responses.

Do not index just once

The effectiveness of the RAG system heavily depends on the quality and coverage of your vector indexes. Regular maintenance and updates of these indexes are crucial for optimal performance.

Configuration

Base Configuration parameters description, default and purpose.

LLM Service Configuration Overview

The following table outlines the configuration parameters for the LLM service:

Parameter Default Value Description
amcs.llm.activeProfiles default Comma-separated list of active profiles
amcs.llm.provider.default openai The LLM API Service provider
amcs.llm.auth_id.default empty Used if the provider uses OAuth authentication
amcs.llm.auth_secret.default empty The provider's integration key (secret)
amcs.llm.auth_uri.default empty The URL for generating an authorization URL (for OAuth)
amcs.llm.api_uri.default https://api.openai.com/v1/ The APIs endpoint
amcs.llm.model_id.default empty The default model to be used (deprecated)
amcs.llm.temperature.default 0.0 The default request's temperature (deprecated)
amcs.llm.data_residency.default empty Named data-residency region pinning the API base URL: global, us, eu or ae
amcs.llm.organization.default empty OpenAI organization identifier used to attribute usage and quota
amcs.llm.project.default empty OpenAI project identifier used to attribute usage and quota
amcs.llm.max_retries.default empty Maximum number of automatic retries for failed requests
amcs.llm.net_timeout.default 3000 The default network timeout (in milliseconds)
amcs.llm.net_log_enabled.default false Network Request and Responses Logging
amcs.llm.propagate_errors.default false Raise provider errors to the calling script instead of returning a null result
amcs.llm.index_store empty The storage path for embedding indexes

Data residency

amcs.llm.data_residency.default selects one of OpenAI's named data-residency regions and resolves the API base URL accordingly, so that requests never leave that region. When both this parameter and amcs.llm.api_uri.default are set, the explicit endpoint wins.

Newer model families

Model names are passed through to the provider as plain strings, so the latest families -- including gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna and gpt-6-astra -- can be used as soon as your provider exposes them, with no Module Suite upgrade required. Module Suite recognises the gpt-5.x and gpt-6 families as reasoning-capable, so CARL enables reasoning and the apply_patch tool for them automatically.

Active Profiles

The amcs.llm.activeProfiles parameter supports multiple profile configurations. You can register a new profile by adding the proper "Custom variables" to the Base Configuration. This property allows you to enable existing profiles. Profiles that are not enabled are not considered.

Supported Providers

As of the time of writing, the supported LLM API Service providers (amcs.llm.provider.default) are: - openai - azure - ollama

Deprecated Parameters

The following parameters are marked as deprecated: - amcs.llm.model_id.default: The default model to be used - amcs.llm.temperature.default: The default request's temperature

Consider using alternative methods to specify these values in your requests.

API Key Security

The amcs.llm.auth_secret.default parameter is used to store your LLM provider's API key. Ensure that this value is kept secure and not exposed in any logs or public configurations. It's recommended to use secure methods for storing and managing API keys in your production environment.

Network Logging

The amcs.llm.net_log_enabled.default parameter enables logging of network requests and responses. This should be set to false in production environments to prevent potential exposure of sensitive information.

Error Propagation

By default, when the LLM provider rejects a request (invalid model, expired key, content filter, quota exceeded, ...) llm.createChatCompletion() logs the failure and returns null. This is kept as the default behaviour for backward compatibility, but it prevents the calling script from telling why the request failed.

Setting amcs.llm.propagate_errors.default to true makes the service raise an ExecutionFaultException instead, wrapping the original provider error as its cause:

try {
    def result = llm.createChatCompletion(req)
    out << result.choices[0].message.content
} catch (ExecutionFaultException e) {
    // e.getMessage() reports the provider's own error message
    // e.getCause() gives access to the original provider exception
    out << "The LLM request failed: " + e.getMessage()
}

The behaviour can also be decided per request, without changing the configuration. This is useful when only a specific scenario needs strict error handling (or, conversely, needs to tolerate a failure). Set it on the request builder:

def req = openai.newChatCompletionRequestBuilder()
                .model("gpt-4")
                .propagateErrors(true)
                .addChatMessage("user", question)
                .build()

def result = llm.createChatCompletion(req)

or directly on an already built request:

req.propagateErrors = true   // raise on failure
req.propagateErrors = false  // tolerate a failure (returns null)

Leaving propagateErrors unset (the default) makes the request follow the amcs.llm.propagate_errors configuration parameter of the profile in use.

Embedding Index Storage

The amcs.llm.index_store parameter specifies the storage path for embedding indexes. In most cases, this should be a shared storage location accessible to all cluster nodes (e.g., EFS) to ensure consistency across the system (e.g. \\otadminBe01\EFS\llm_indexes\).

OAuth Authentication

If your LLM provider uses OAuth authentication, you'll need to set the amcs.llm.auth_id.default and amcs.llm.auth_uri.default parameters accordingly. These are used for generating and managing OAuth tokens.

Defining New LLM Service Profiles

To create a new profile for the LLM service, you need to add custom variables to the Base Configuration. The process involves replicating the existing configuration parameters but replacing the "default" suffix with the name of your new profile. This allows you to maintain multiple configurations for different LLM providers or use cases within the same system.

For example, to create an "azure" profile for Azure AI, you would add the following custom variables:

Parameter Value
amcs.llm.api_uri.azure https://amai-east-us.openai.azure.com/
amcs.llm.auth_secret.azure 7a234aaaa666666d5a245245422cvbs23a
amcs.llm.provider.azure azure

Similarly, to create an "ollama_prod" profile for Ollama, you would add:

Parameter Value
amcs.llm.provider.ollama_prod ollama
amcs.llm.api_uri.ollama_prod http://15.201.113.243:11434

By defining these custom variables, you create distinct profiles that can be activated using the amcs.llm.activeProfiles parameter. This flexibility allows you to easily switch between different LLM providers or configurations without modifying the core settings. Remember to include all necessary parameters for each profile to ensure proper functionality.

Profile Naming

Choose clear and descriptive names for your profiles to easily identify their purpose or associated provider. For instance, use names like "azure_prod", "openai_dev", or "ollama_test" to indicate both the provider and the environment.

Secure Configuration

When defining profiles, especially those containing sensitive information like API keys, ensure that your configuration files and storage are properly secured and follow your organization's security best practices.

Generation runs in Module Suite 4.1.0

Module Suite 4.1.0 (Polaris) adds Generation run behavior for CARL, the AI Form Builder, the Component AI Builder, and Application Builder. A Generation run is one execution of a generation request. The sections below describe how that run is configured and what it can read while it writes. An AI agent can also call tools defined in an MCP server definition.

Aether owns every Generation run. One create or refactor request from an editor is one run. The run keeps its starting snapshot, the model chosen when it starts, its attempts, its events, and the actions it paused for. Each attempt starts fresh and ends with one outcome: a candidate, a request for more knowledge, a question for the user, or a failure.

flowchart LR
    CARL["CARL"] --> Run
    AIFB["AI Form Builder"] --> Run
    CAIB["Component AI Builder"] --> Run
    AppB["Application Builder"] --> Run
    Run["Generation run in Aether"] --> Catalog["Model catalog"]
    Run --> Dict["Content Server API dictionary"]
    Run --> Corpus["Knowledge corpus"]
    Run --> Cards["Environment context cards"]
    Run --> Candidate["Candidate: validated, reviewed"]
    Candidate --> Editor["Result returns to the editor"]

An editor starts the run, and Aether resolves the model from its catalog, grounds the run on the host's API dictionary and on the knowledge corpus, and reads any environment context cards the user picked. In the chat editors on Content Server, the run can also explore Content Server with the user's approval. The validated candidate goes back to the editor for review. Cards are read only by the chat editors, not by Application Builder project runs.

Aether configuration

Aether configuration is separate from the Content Script API. The models an editor can offer come from the model catalog: the list of models AnswerModules offers in Aether. Each model in the catalog has a label, which is the name the user sees in the editor's model picker. The editor sends the selected model, and Aether runs it. A model may be offered only for some kinds of run, so the picker of one editor can list fewer models than another.

Content Server can narrow the models its editors show. The editors filter the picker with the Base Configuration parameter amcs.aether.allowedModels, so users see only the models that the list names.

Parameter Where Description
amcs.aether.allowedModels Content Server Base Configuration Models the editors offer in their picker. A display filter for the picker

Not a spending limit

amcs.aether.allowedModels only filters the picker. Tenant quotas apply across all models, so Module Suite 4.1.0 has no enforced per-model spending limit.

Content Server API dictionary

The host supplies the Content Server API dictionary for a Generation run. The dictionary is the Content Script API of that Content Server: every bound service and every documented type reachable from those services, with full signatures. It includes the extension services, such as llm, openai, and docusign, and any customer extension. Module Suite Runtime (MSR) describes only its own API, so a Content Server run is never grounded on the MSR API.

The dictionary describes the API of the installation, not of one node, so every node of a Content Server cluster shares one dictionary. When you install or upgrade an extension, the API changes, and the next run asks the host for the new dictionary.

sequenceDiagram
    participant Editor
    participant Aether
    participant Host as Content Server
    Editor->>Aether: Start Generation run
    alt Dictionary already known for this host
        Aether->>Aether: Run starts with the dictionary, no pause
    else First run on this host
        Aether-->>Editor: Pause once, tool api_dictionary
        Editor->>Host: Ask for the API dictionary
        Host-->>Editor: API dictionary
        Editor->>Aether: Answer the pause with the dictionary
    end
    Aether->>Aether: Model reads the dictionary while it writes

When Aether already has the dictionary for that host, the run starts with it. On the first run on a host, the first attempt pauses once, before the model's first turn, and the editor asks the host for its dictionary. The resumed attempt keeps the dictionary it received.

API lookups, lint suggestions, and the list of services in the model's instructions all read the host's dictionary. The model can read the API of a returned object before it writes a call. Code completion in the editor also uses the dictionary once a run has obtained it for that host. Until then, completion uses only the editor's own members.

When the host cannot supply the dictionary, the run continues, and the API lookups answer host_api_unavailable, which is not retryable. When the user denies the request or the host returns an invalid dictionary, the run does not ask again. After any other failure, the next attempt asks again before the model's first turn. Aether never substitutes another dictionary for the host's.

Size limits on dictionary entries

A member or type whose name exceeds 200 characters, whose signature exceeds 2,000 characters, or whose type id exceeds 500 characters is dropped from the dictionary. Descriptions are flattened and truncated to 2,000 characters before they can reach a prompt.

Selecting a model

An editor can select a model for a Generation run, or leave the selection on Auto. Auto is the default and is always available. On Auto, the run uses the model AnswerModules configures in Aether for that kind of run. When the user selects a model, the editor sends only that model's catalog id, so users do not need to know which provider serves which model.

A selection is accepted for a content script, a form in the AI Form Builder, a component in the Component AI Builder, a smart page, and a workflow. A model in the catalog may be offered for only some of those runs. Manifest planning, manifest generation, and script repair keep the model configured for them.

The choice is kept on the run when it starts. Retries and repairs reuse it. A later Generation run uses the model selected in the editor when that run starts. A run that continues the previous conversation is one of those later runs, so it uses the editor's current choice. When the model changes, the new run starts a new provider conversation.

A model in the catalog may have its own output limit, which also applies to a retry after a truncated response. The selected model uses the reasoning policy configured for the run when it can run that policy. Otherwise it uses the provider default, and the selection is still accepted.

Aether rejects an unknown model (unknown_model) and a model the run cannot use (model_not_available_for_recipe). The editor sends the same request once more on Auto and tells the user, so the prompt is kept.

flowchart TD
    Req["Editor sends the request"] --> Choice{"Model selection"}
    Choice -->|"Auto"| AutoModel["Model configured for this kind of run"]
    Choice -->|"Selected model"| Intake{"Intake check"}
    Intake -->|"Unknown model"| Unknown["400 unknown_model"]
    Intake -->|"Model not offered for this run"| NotAvail["400 model_not_available_for_recipe"]
    Unknown --> Retry["Editor resends once on Auto and tells the user"]
    NotAvail --> Retry
    Retry --> AutoModel
    Intake -->|"Accepted"| Selected["Selected model"]
    AutoModel --> Kept["Choice kept on the run"]
    Selected --> Kept
    Kept --> Reuse["Retries and repairs reuse the same model"]

A rejected selection never loses the prompt: the editor falls back to Auto once and reports the fallback. An accepted selection is kept on the run, so every later attempt of that run uses the same model.

Changing models in a conversation

A follow-up run that continues the previous conversation uses the model selected in the editor at that moment. When that changes the provider or the model, the new run starts a new provider conversation and does not reuse the provider's prompt cache.

Generation trace and token use

The run's lifecycle on success is preparing, generating, validating, awaiting review, approved, publishing, and published. An invalid candidate enters repairing, and a fresh, focused repair attempt starts. A publication failure returns the run to approved. A user approves the result before it is published, and every publication creates a new external version.

Each Generation run keeps a trace of its attempts: the model used, the tool calls, and the outcome of each attempt. The review of the candidate uses that trace.

A generated package includes chat.json, generation.json, and validation.json. A plain script keeps only its Groovy source as node content on Content Server.

Token use is counted for the tenant that started the run. Tenant quotas apply across all models.

Client tool approval

CARL in the Content Script editor, the AI Form Builder, the Component AI Builder, and Application Builder ask the user to approve a client tool before the tool runs. A client tool is a tool that the editor answers in the browser, such as the API dictionary request or a Content Server exploration. The approval card asks "Run this tool?" and offers Deny, Allow, and Always allow.

By default, the API dictionary request (api_dictionary) is approved automatically, so most users never see a card for it. When an approval card for the API dictionary does appear, it appears at most once for each version of the host's API, because later runs already have the dictionary. When the user denies the request, the run is not asked again.

In the three chat editors on Content Server, each Content Server exploration (explore_content_server) asks for approval by default. A run may show several approval cards. Always allow stops the cards for that user in that browser. Reading a file of a card the user picked (read_selected_object) is approved automatically, because the user already chose that object. See Content Server exploration in the editors.

A consent prompt

The approval card is a consent prompt, not a security boundary. Every exploration runs with the user's Content Server session, so Content Server permissions apply, and an exploration never writes.

AI agents

For the full scripting contract of the AMAgentController behaviour, see AI agents.

An AI agent is an Application Builder artifact. The manifest records it as component type ai-agent. The artifact is a Content Script. Application Builder generates it with the content-script recipe, an agent script kind, and an agent contract check. A Content Script stored under agents/ is discovered as an AI agent. By default that script serves the chat backend. The optional chat UI is a shell component that references the agent, on a shell page.

The script declares the agent with the agentRequest hook. The hook runs once per model call and returns the profile, the request, the tools, any MCP profiles, and the tools that have side effects. Every tool has an executor. The runtime owns the conversation input and runs each tool call on the Content Server request thread, where a tool can use the Content Server API.

The request the hook returns holds the model, the instructions, the output schema, and the options. The hook runs on every model call because the provider's previous-response chaining does not carry instructions, tools, or schema forward. Closure tools and MCP tools run the same way.

// Called once per model call. The script builds each value.
agentRequest = { turn ->
    [profile, request, tools, mcpProfiles, sideEffects]
}
sequenceDiagram
    participant UI as Chat UI
    participant Agent as Agent script
    participant Model
    UI->>Agent: chat with the user message
    loop Each model call
        Agent->>Agent: agentRequest hook returns profile, request, tools, MCP profiles, sideEffects
        Agent->>Model: Model call with the conversation input and tools
        Model-->>Agent: Streamed text and tool calls
        Agent->>Agent: Write text deltas to the cache
        opt The model called a tool
            Agent->>Agent: Run the tool on the Content Server request thread
            Note over Agent: tool.started reports a tool listed in sideEffects
        end
    end
    UI->>Agent: chat_poll
    Agent-->>UI: Events such as text.delta, tool.started, output.final

The agent loop runs on the Content Server request thread, because Content Script services work only there. The chat UI does not read the answer from the chat response: it polls chat_poll, so a proxy that cuts the held-open request does not turn a successful run into a failure.

The agent's base endpoint is /otcs/cs/runcs/<agentNodeId>, and each action is <base>/<action>. The actions are chat, chat_poll, chat_stop, chat_attach, chat_detach, chat_new, and chat_session.

chat_poll returns protocol, events, next, and end. The event types are text.delta, reasoning.summary, tool.started, tool.finished, output.final, error, stopped, busy, and expired. Events are never deleted on read, so the same next always returns the same events. A lost response, a second tab, or a reload is safe. output.final carries HTML rendered from the Markdown answer: raw HTML is escaped, images are removed, and links get rel="noopener noreferrer".

No approval event for agent tools

The poll protocol reserves tool.approval_required, but Module Suite 4.1.0 never emits it. Client tool approval applies to the Generation runs of the chat editors and of Application Builder, not to AI agent tools.

A script can also run an agent headless with docman.runContentScript(agentId, [agentInput: ...]). The call runs synchronously and returns the text, the output, the tool calls, and the usage. A headless result also returns files. Pass them back as agentInput.files to work on the same files again.

A chat run uses the signed-in user. A headless run uses the identity of the calling script, so a workflow step may run tools as a privileged agent user. LLM and MCP profiles are configuration objects of kinds llm-profile and mcp-profile, referenced through dependsOn. Secrets never enter the manifest.

A conversation is a session in the Content Server database cache, owned by the user who created it. Every session action checks that owner. Uploads go into an "Agent attachments" folder in the user's Personal Workspace, with one child folder per conversation. They are cleaned up after their session is gone.

Parameter Default Description
amcs.openai.agentMaxConcurrentRuns 4 Maximum concurrent chat runs on one Content Server node. Each user has at most one active run
amcs.openai.agentSessionTtlSeconds 7 days Lifetime of an agent conversation session

Worker threads

Each running agent occupies one Content Server worker thread for the length of the run. Stopping is cooperative: the runtime checks the stop flag inside the stream, before each tool call, and between iterations.

Zero data retention

A profile that does not store responses on the provider side works only for an agent with no tools that makes a single model call. In Module Suite 4.1.0, Content Server is the only target host for an AI agent.

The agent can call tools on files the user has attached. The model receives the extracted text of an embedded file. A tool opens the file when the work needs its structure, such as tracked changes, comments, embedded images, charts, or exact cell values. For a Word document, the tool loads it with docx.loadWordDoc and reads revisions and comment anchors with listRevisions and listCommentAnchors.

Each attached file has a record from its last version: node id, name, MIME type, size, and whether the file was embedded. A file is embedded when embedTypes allows its type (every type by default) and its size is at most maxAttachmentBytes. Every other file is metadata-only. The model still sees it in the file list, and a tool can open it. A file over the size limit remains metadata-only, and the message still runs.

Each turn tells the model the node id of every file on the conversation, including files attached on earlier turns. File names are JSON-quoted. The conversation holds at most 50 files. To attach more, the user starts a new chat. The hook's turn opens one of those files with file and a node id. The call accepts only a file attached to the conversation. A file that was never attached fails with file_not_attached. A file that is gone fails with file_gone. The tool result keeps that code. A tool that looks up any other node by id uses the Content Server API, under Content Server permissions.

// Inside a tool executor: open a file attached to this conversation
def doc = docx.loadWordDoc(turn.file(nodeId))
def revisions = doc.listRevisions()
def anchors = doc.listCommentAnchors()

turn.file is an opt-in allowlist. It protects the tools written to use it, which matters most for a headless run under a privileged user.

Prompt injection

Attached documents and snippets can carry instructions. List every tool that changes content in sideEffects, so the runtime reports it on tool.started. The HTML sanitizer removes images and unsafe links, so a manipulated answer cannot leak data through URLs the browser fetches on its own.

Environment context cards

Environment context cards pass selected Content Server objects into a chat editor. On Content Server, the Content Script editor, the AI Form Builder, and the Component AI Builder take the selection. The user picks a node, a user or a group, or a table by name, with the standard Content Server dialogs. In the Content Script editor, Add the selected workspace also adds the editor's selected workspace as one card. Each pick becomes a card. A request carries at most four cards, and a workspace card is one of them.

Content Server writes the card as a manifest of identity facts and of the files or facts that may be read. Each entry has one purpose line. A form lists its five artifact files. A classic form lists none. A shell component lists its three files. The card records the node those files belong to, and the version at which the manifest was written. Reads use that version.

The Generation run reads one listed file of a card at a time through read_selected_object. The tool is registered on a run that has cards, has no Application Builder project, and whose editor answers exploration. It accepts an id only when that id is on a card, or is one of the scripts a workspace card lists. The editor page answers the read.

The id on a card is the node the files belong to: the Form View of a form, the Smart Page of a shell component, the id of a user or group, or the lower-case name of a table. A workspace card is identified as workspace:<name>, with the workspace name as written, and that id is never read as a node. Duplicate picks are merged on that id, and a card for the object open in the editor is refused.

A workspace card lists every script of the workspace that the user can open, with its node id and nickname, and the entries it skipped and why: an entry the user cannot open, or an entry that is not a script. The card does not show the scripts' paths. The run reads the source of any listed script through read_selected_object, with that script's node id. A workspace with no scripts cannot be added. Each send reads the workspace again as it is then, so scripts filed in it since are listed too, and the send stops when the workspace was deleted.

flowchart TD
    Pick["User picks a node, a member, or a table"] --> Card["Content Server writes the card manifest"]
    Card --> Run["Generation run with at most four cards"]
    Run --> Tool["read_selected_object"]
    Tool --> OnCard{"Id on a card, or a script a workspace card lists?"}
    OnCard -->|"No"| Refused["Read refused"]
    OnCard -->|"Yes"| Kind{"Kind of read"}
    Kind -->|"Whole file"| Whole["Up to 60,000 characters"]
    Kind -->|"Search"| Search["Up to 30 hits, 300 characters each"]
    Kind -->|"Line range"| Range["Up to 200 lines and 20,000 characters"]
    Whole -->|"Longer file"| Cut["First 60,000 characters, length, line count, pointer to search"]
    Whole --> Budget["Attempt budget: 6 bodies, 8 searches"]
    Cut --> Budget
    Search --> Budget
    Range --> Budget

The model never receives a card's content up front, only its manifest. It reads the file it needs, as a whole, as search hits, or as a line range, and each kind of read has its own limit inside the attempt's budget.

A whole file comes back as stored, up to 60,000 characters. Above that limit the result is the first 60,000 characters, the full length, the line count, and a pointer to search. A search is plain text and case-insensitive: at most 30 hits, each at most 300 characters. A line range is at most 200 lines and 20,000 characters. A JSON file is pretty-printed before a search or a range, and the line numbers refer to that text. One attempt may return 6 bodies — a whole file, a line range, or a table — and run 8 searches. A repeated read returns the earlier result. Two ranges of one file are two reads, and both stay in the conversation.

Every read pauses the attempt while the editor page answers. The budgets survive the pause: when the attempt resumes, the tool rebuilds its counts and stored results from the attempt's tool trace. The limits apply per read, not as a total, so a file the user picked is never refused for size alone.

Ask about one file

A request about form labels reads only the form's locales.json, not the whole form bundle. Name the object and the aspect you need, and the run reads only that file.

Redaction

Card text is redacted with the other sources of the run. The editor's answers on a paused attempt, and the input of the paused call, are redacted with the tool results.

A repair keeps the cards. The run reads a card first and explores Content Server for what no card gives. When the user named or clearly referred to an object that no card provides, the run looks for it with an exploration, which asks the user first. The attempt fails with a reason that starts with environment_object_missing only when the exploration does not find the object or the user refuses the exploration. The failure is not retried: the same run would send the same cards. The user adds the object as a card and sends a new message. An object the script can take as a parameter, look up at run time, or create — a destination folder or a template — stays part of the design. It becomes a parameter or a named variable. A choice that only the user can make becomes a question for the user.

Application Builder project runs keep their own Content Server reads. Environment context cards belong to the chat editors.

On Content Server, the Component AI Builder answers these card reads. When the component targets Module Suite Runtime (MSR), the same editor also answers the MSR REST catalog.

Content Server exploration in the editors

On Content Server, a Generation run in the Content Script editor, the AI Form Builder, or the Component AI Builder can explore Content Server with explore_content_server. The run reads the cards the user picked first, and explores for what no card gives, such as the scripts of a folder the user selected. Exploration is read-only: it runs with the user's Content Server session, so it sees only what the user can see, and it never writes.

Command What it reads
list_nodes Nodes of one kind, under an ancestor where the kind needs one
describe_node One node and the aspects it offers
get_node_aspects Aspects of several nodes, each aspect cut at 4,000 characters
list_tables and describe_table Tables, and the columns of one table
read_file One file of a node the user can see, whole, as a search, or as a line range

An aspect of 4,000 characters is too short for a script. A run without an Application Builder project therefore reads a file it found in full with read_file, with the rules of read_selected_object: a whole file up to 60,000 characters, a plain-text search, or a range of at most 200 lines. read_file has its own budget of 6 bodies and 8 searches per attempt.

sequenceDiagram
    participant User
    participant Editor
    participant Aether
    participant CS as Content Server
    User->>Editor: Send a request with the selected cards
    Editor->>Aether: Start Generation run
    loop Each exploration
        Aether-->>Editor: Pause, tool explore_content_server
        Editor->>User: Approval step, Run this tool?
        alt Allow or Always allow
            User->>Editor: Allow
            alt List or describe
                Editor->>CS: List nodes or read aspects with the user's session
                CS-->>Editor: Nodes and aspects, each aspect up to 4,000 characters
            else Read file
                Editor->>CS: read_file for a file the run found, with the user's session
                CS-->>Editor: The file, up to 60,000 characters
            end
            Editor->>Aether: Answer the pause
        else Deny
            User->>Editor: Deny
            Editor->>Aether: Exploration refused
        end
        Aether->>Aether: Rebuild the explore budget from the run's trace
    end

Each exploration asks the user for approval by default, as described under Client tool approval, and a read_file call is an exploration too. When the user denies an exploration, the run gets no Content Server answer for that call. When the run needed an object the user named, it then fails with environment_object_missing.

The exploration budgets of an attempt are limited: distinct list queries, node and aspect pairs, and read_file bodies and searches. Every exploration pauses the attempt while the editor answers. After a pause, Aether rebuilds the explore budget from the run's trace, without the calls the tool refused on its own, so a pause never resets the budget. The same applies to Application Builder runs.

Point at the folder

Add the folder that holds the scripts you mean as a card and ask for the change. The run lists the folder and reads the sources in full after you allow it, so you do not need to add each script as a card.

Knowledge corpus

The knowledge corpus is the curated, reviewed, versioned store of Content Server knowledge that a Generation run consults. The corpus is global for every tenant. In Module Suite 4.1.0 it includes category attribute type codes and a document-categorization agent.

The corpus is organized in items, each with revisions, sections, and blocks. An item is a concept, a reference, a guide, an index, or an example. An example is reviewed prose with a Groovy code block. An example marked as a template has a target and a validation profile, and a run can start from it: when the starting script is empty, Aether seeds the workspace with a structural template chosen for the target and the kind of change. The model adapts that template and keeps its return contract.

The knowledge a run uses is approved evidence: references to corpus items, revisions, and sections that Aether selects and attaches to the run. When an attempt needs more knowledge, it ends with one focused query. Aether retrieves approved evidence for that query and starts a fresh attempt. Reads from the corpus are gated by scope, applicability, publication trust, and withdrawals.

OAuth 2 credentials

The LLM service and the OpenAI service accept OAuth 2 credentials. The LLM service profile parameters are listed under LLM Service Configuration Overview. Azure OpenAI calls now complete. That Azure fix is also in a 4.0 hotfix.

PDF attachments

A PDF attachment can be read during a Generation run without a browser export.

Note

For files attached to an AI agent conversation, embedding and metadata-only handling follow embedTypes and maxAttachmentBytes, as described under AI agents.

Tool calls and reasoning models

Tool calls work when the selected model is a reasoning model. A model in the catalog may come with a fixed reasoning policy. The Generation run uses the configured reasoning policy when the selected model can run it. When the model cannot run that policy, the run uses the provider default.

A reasoning policy is disabled, low, medium, high, or provider-default. disabled always sends an explicit disabled setting. provider-default leaves the reasoning behavior to the provider. Because a selected model falls back to provider-default instead of failing, a selection never fails a run after intake.

Each provider advertises what it supports. A request that needs unsupported reasoning, strictness, tools, prompt caching, or output limits fails before the provider is called. The model picker lists a model with a fixed reasoning policy only when that model can run the policy.

Limits

Limit Value
Environment context cards per request 4, a workspace card included
Scripts listed on a workspace card 100
Whole-file read of a card file or of a file found by exploration 60,000 characters
Search of a card file or of a file found by exploration 30 hits, 300 characters each
Line-range read of a card file or of a file found by exploration 200 lines and 20,000 characters
Card reads per attempt 6 bodies and 8 searches
read_file reads per attempt 6 bodies and 8 searches
Aspect returned by an exploration 4,000 characters
Explorations per attempt Limited, and kept across a pause
Files on an AI agent conversation 50
Concurrent AI agent chat runs per Content Server node 4 by default (amcs.openai.agentMaxConcurrentRuns)
Active AI agent runs per user 1
API dictionary member or type name 200 characters
API dictionary signature 2,000 characters
API dictionary type id 500 characters
API dictionary description Truncated to 2,000 characters

Troubleshooting

Code Meaning What to do
unknown_model Aether does not know the selected model The editor resends once on Auto. Choose another model in the picker, or ask your administrator to correct the model names in amcs.aether.allowedModels
model_not_available_for_recipe The selected model is not offered for this kind of run The editor resends once on Auto. Choose another model, or keep Auto
host_api_unavailable The host could not supply its Content Server API dictionary The run continues, but API lookups cannot answer. Start a new run. If the code persists, contact your administrator
file_not_attached An agent tool asked turn.file for a file never attached to the conversation Attach the file, or look up the node with the Content Server API
file_gone The attached file no longer exists Attach the file again
environment_object_missing An object the user named is not on any card, and the exploration did not find it or the user refused the exploration Add the object as a card and send a new message. The run does not retry

Migrating to Module Suite 4.0

Module Suite 4.0 introduces breaking changes in the LLM extension. The legacy third-party OpenAI client library (io.github.lambdua) has been removed and replaced with the official OpenAI Java SDK (com.openai:openai-java). This section provides a detailed migration guide for scripts that rely on removed or changed APIs.

Breaking Changes

The changes described below are breaking. Existing scripts that use the removed APIs will fail at runtime after upgrading to 4.0. Review and update your scripts before deploying the new version.

Removed: Assistant API (CSOpenAI)

The entire OpenAI Assistants API surface has been removed from the openai service. OpenAI itself has deprecated the Assistants API in favor of the Responses API.

Removed builder methods:

Removed Method Replacement
openai.newAssistantBuilder() Use Responses API: openai.newResponseRequestBuilder()
openai.newAssistantUpdateBuilder() Use Responses API
openai.newRunBuilder() Use Responses API
openai.newThreadRequest() Use Responses API
openai.newMessageCreationRequest() Use Responses API
openai.newAssistantEventHandler() Use Responses API with streaming

Removed CRUD methods:

Removed Method Replacement
openai.createAssistant(...) Use Responses API
openai.listAssistants(...) Use Responses API
openai.getAssistant(...) Use Responses API
openai.deleteAssistant(...) Use Responses API
openai.updateAssistant(...) Use Responses API
openai.createThread(...) Use Responses API
openai.getThread(...) Use Responses API
openai.updateThread(...) Use Responses API
openai.deleteThread(...) Use Responses API
openai.createRun(...) Use Responses API
openai.getRun(...) Use Responses API
openai.listRuns(...) Use Responses API
openai.cancelRun(...) Use Responses API
openai.executeRequiredActions(...) Use Responses API with function calling
openai.listMessages(...) Use Responses API
openai.createMessage(...) Use Responses API

Removed wrapper classes (entire api.openai.assistant package):

CSOpenAIAssistant, CSOpenAIAssistantCreationRequest, CSOpenAIAssistantCreationRequestBuilder, CSOpenAIAssistantUpdateRequest, CSOpenAIAssistantUpdateRequestBuilder, CSOpenAIAssistantEventHandler, CSOpenAIAssistantStreamManager, CSOpenAIThread, CSOpenAIThreadRequest, CSOpenAIRun, CSOpenAIRunRequest, CSOpenAIRunRequestBuilder, CSOpenAIMessage, CSOpenAIMessageCreationRequest, and related types.

Migration Path

The Responses API is the recommended replacement for all Assistant-based workflows. See the OpenAI Responses API documentation for the equivalent patterns. Module Suite 4.0 provides openai.newResponseRequestBuilder() and openai.createResponse() / openai.streamResponse() as replacements.

Removed: Legacy File Helpers (CSOpenAI and CSLLMService)

The old file management methods based on CSOpenAIRemoteFile have been removed from both the openai and llm services. These are replaced by the newer File API introduced in 3.9.0.

Removed Method (CSOpenAI) Replacement (CSOpenAI)
openai.uploadFile(String purpose, CSResource resource, ...) openai.uploadFile(openai.newFileCreateParamsBuilder(purpose, resource).build())
openai.getFile(String fileId, ...) openai.retrieveFile(fileId)
openai.listFiles(...) openai.listFiles(openai.newFileListParamsBuilder().build())
openai.deleteFile(CSOpenAIRemoteFile file, ...) openai.deleteFile(file.id)
openai.getFileContent(CSOpenAIRemoteFile file, ...) openai.retrieveFileContent(file.id)
Removed Method (CSLLMService) Replacement (CSOpenAI)
llm.uploadFile(String purpose, CSResource resource, ...) openai.uploadFile(openai.newFileCreateParamsBuilder(purpose, resource).build())
llm.getFile(String fileId, ...) openai.retrieveFile(fileId)
llm.listFiles(...) openai.listFiles(openai.newFileListParamsBuilder().build())
llm.deleteFile(CSOpenAIRemoteFile file, ...) openai.deleteFile(file.id)
llm.getFileContent(CSOpenAIRemoteFile file, ...) openai.retrieveFileContent(file.id)

Return type changes:

  • The old methods returned CSOpenAIRemoteFile instances. The new methods return CSOpenAIFileObject instances.
  • CSOpenAIFileObject provides the same core properties (id, filename, bytes, purpose, createdAt) with consistent naming.

Example: Migrating File Upload

// Old API: upload and get file info
def remoteFile = openai.uploadFile("assistants", myResource)
out << "File ID: ${remoteFile.id}"

// Old API: retrieve file info
def fileInfo = openai.getFile(remoteFile.id)
out << "Filename: ${fileInfo.filename}"

// Old API: delete file
openai.deleteFile(remoteFile)
// New API: upload file using builder
def params = openai.newFileCreateParamsBuilder("assistants", myResource).build()
def fileObject = openai.uploadFile(params)
out << "File ID: ${fileObject.id}"

// New API: retrieve file info
def fileInfo = openai.retrieveFile(fileObject.id)
out << "Filename: ${fileInfo.filename}"

// New API: delete file by ID
openai.deleteFile(fileObject.id)

Removed: Legacy Builders and Completion APIs (CSLLMService)

Several deprecated builder factory methods and legacy completion APIs have been removed from the llm service:

Removed Method Replacement
llm.newOpenAIChatCompletionRequestBuilder() openai.newChatCompletionRequestBuilder()
llm.newOpenAIFineTuneRequestBuilder() openai.newFineTuneJobCreateParamsBuilder()
llm.newOpenAICompletionRequestBuilder() openai.newCompletionRequestBuilder()
llm.newOpenAIEmbeddingRequestBuilder() openai.newEmbeddingRequestBuilder()
llm.newOpenAIModerationRequestBuilder() openai.newModerationRequestBuilder()
llm.newOpenAISpeechRequestBuilder() openai.newSpeechRequestBuilder()
llm.newOpenAITranscriptionRequestBuilder() openai.newTranscriptionRequestBuilder()
llm.newOpenAITranslationRequestBuilder() openai.newTranslationRequestBuilder()
llm.createOpenAIModerationRequestBuilder(...) openai.createOpenAIModerationRequestBuilder(...)

Removed: Legacy Fine-Tune Builder (CSOpenAI)

Removed Method Replacement
openai.newFineTuneRequestBuilder() openai.newFineTuneJobCreateParamsBuilder(model, fileId)

Removed: Legacy Completion APIs (CSOpenAI)

Removed Method Replacement
openai.createCompletion(CSOpenAICompletionRequest, ...) openai.createCompletion(request) using newCompletionRequestBuilder() (non-deprecated overload)
openai.streamCompletion(CSCompletionRequest, Closure, ...) openai.streamCompletion(request, closure) using newCompletionRequestBuilder()

Changed: newChatMessage Return Type (CSLLMService)

The llm.newChatMessage(role, content, name) method now returns dev.langchain4j.data.message.ChatMessage instead of the removed com.theokanning.openai.completion.chat.ChatMessage.

Scripts that stored the result in a typed variable (e.g., ChatMessage msg = llm.newChatMessage(...)) must update the import. Scripts using def are unaffected.

Changed: CSOpenAIModel Properties

The CSOpenAIModel wrapper (returned by openai.listModels() and openai.getModel()) has changed:

Removed Property Notes
model.root No longer available in the official SDK
model.parent Always returns null
model.permission No longer available in the official SDK
model.getOpenAIModel() Removed to avoid exposing third-party SDK types
New Property Description
model.created Unix timestamp of model creation

Changed: CSOpenAIVectorStore Package

The CSOpenAIVectorStore class has moved from:

  • com.answer.modules.cscript.api.external.llm.api.openai.assistant.CSOpenAIVectorStore

to:

  • com.answer.modules.cscript.api.external.llm.api.openai.vectorstore.CSOpenAIVectorStore

Scripts referencing the old fully-qualified class name must be updated. Scripts using the openai.createVectorStore(...) / openai.getVectorStore(...) methods are not affected since the return type resolves automatically.

Changed: CSOpenAISearchResult

CSOpenAISearchResult is now a plain API object with firstId, lastId, hasMore, and data properties. It no longer delegates to the removed OpenAiResponse type. Existing scripts accessing .data, .firstId, .lastId, or .hasMore are compatible. Scripts that accessed internal OpenAiResponse properties will need updating.

Removed: Fine-Tune and Remote File Wrapper Classes

The following orphaned wrapper classes have been deleted:

  • CSOpenAIFineTuneRequest, CSOpenAIFineTuneRequestBuilder (under api/openai/finetune/)
  • CSOpenAIFineTuneResult, CSOpenAIFineTuneEvent (under api/openai/finetune/)
  • CSOpenAIRemoteFile (under api/openai/)

Use CSOpenAIFileObject and the new fine-tuning APIs (openai.newFineTuneJobCreateParamsBuilder()) instead.