Rich Chat Rendering & UI Components
While agents typically respond with standard conversational text, the Vue.ai platform allows you to render Rich UI Components directly in the chat interface. This includes interactive forms, data charts, scrollable cards, downloadable files, and quick reply buttons.
This process is handled by the Renderer node or by configuring your Action to return specifically formatted JSON. The Renderer converts raw responses into rich interactive UI components — unlike the Formatter, which produces human-readable markdown text.
Two Ways to Render Rich UI
The Vue.ai platform provides two distinct methods for generating rich chat components:
Option 1: The Renderer Node (LLM-Powered)
The Renderer is an optional processing node that automatically maps raw responses into UI components.
- How it works: It uses a dedicated LLM to decide which visual component (card, chart, form) best represents the data.
- Refinement: You can provide optional instructions to the node to refine its behavior (e.g., "Always use bar charts for comparison data").
- Best for: Dynamic data where the exact UI structure isn't known ahead of time.
Option 2: Forced Rendering (Action-Powered)
Instead of relying on the Renderer node's reasoning, you can configure an Action (Workflow or API) to return a specific data structure that forces the UI to render a particular component.
- How it works: The Action output contains explicit
typeandmetadatakeys. - Best for: Deterministic scenarios where you want absolute control over the UI components (e.g., a specific onboarding form).
Latency vs. Determinism
Using the Renderer node involves an additional LLM call, which increases overall response latency. For straightforward or deterministic use cases where the UI structure is always the same, it is better to force rendering directly through the Action output to minimize wait times and ensure consistency.
Forced Rendering Mechanism
To force a particular type of rendering, the response from the upstream action (or the Formatter node) must return a stringified JSON object containing two specific keys:
type: Defines the UI component to render (text,card,files,form, orchart).metadata: Contains the data payload matching the component's schema.
Important Notice on Formatting
If your action natively returns this forced rendering structure, ensure that you disable the Format Output option on the Action, so the Formatter does not overwrite your rendering schema.
Rendering Types & Schemas
1. Text with Quick Replies
Useful for providing standard text followed by clickable suggestion chips.
Schema Overview:
content(string): The text message (supports Markdown).quickReplies(array): List of chips.label(string): Text shown on the chip.action: What happens when clicked.text(string): Message sent to the bot.send(boolean): Optionally send the message immediately.
Expected Metadata Schema
{
"type": "object",
"properties": {
"content": {
"type": "string",
"description": "The message of the response, which can include Markdown."
},
"quickReplies": {
"type": "array",
"description": "An optional list of quick reply chips to display below the message.",
"items": {
"type": "object",
"properties": {
"label": {
"type": "string",
"description": "the text that is displayed to the user in each reply chip"
},
"action": {
"type": "object",
"description": "defines the text that is sent to the bot when the user clicks on the quick reply chip",
"properties": {
"text": {
"type": "string"
},
"send": {
"type": "boolean"
}
},
"required": [
"text"
]
}
},
"required": [
"label",
"action"
]
}
}
},
"required": [
"content"
]
}
Example Payload:
{
"type": "text",
"metadata": {
"content": "Would you like to proceed?",
"quickReplies": [
{
"label": "Yes",
"action": { "text": "Yes, we can proceed", "send": true }
},
{
"label": "No",
"action": { "text": "No, do not proceed", "send": true }
}
]
}
}
2. Cards
Renders a visually appealing, scrollable carousel of items. Ideal for product recommendations or comparing options side-by-side.
Schema Overview:
content(string): Message displayed above the cards.cards(array): The list of items to display.title(string): Card header.description(string): Card body (supports Markdown).imageUrl(string, optional): Image to display inside the card.imageAction(object, optional): Action triggered when clicking the image.buttons(array): Action buttons within the card (typecan be"prefill"or"link").
quickReplies(array, optional): Global quick replies shown below the carousel.
Expected Metadata Schema
{
"type": "object",
"properties": {
"content": {
"type": "string",
"description": "The text that appears before the cards."
},
"cards": {
"type": "array",
"description": "The array of card objects to be displayed.",
"items": {
"type": "object",
"properties": {
"title": {
"type": "string",
"description": "A title for this card"
},
"description": {
"type": "string",
"description": "The main description of the card, which can include Markdown."
},
"imageUrl": {
"type": "string",
"description": "The URL of an image to display within the card."
},
"buttons": {
"type": "array",
"items": {
"type": "object",
"properties": {
"label": {
"type": "string",
"description": "The text that is displayed to the user in each button"
},
"action": {
"type": "object",
"properties": {
"text": {
"type": "string",
"description": "The text that is sent to the bot when the user clicks on the button"
},
"type": {
"type": "string",
"description": "The type of action to perform - use prefill for text to be sent to the bot, link for a link to be opened in a browser",
"enum": [
"prefill",
"link"
]
},
"url": {
"type": "string",
"description": "The URL that is opened in a browser when the user clicks on the button, only provide this if button is of type link."
}
},
"if": {
"properties": {
"type": {
"const": "prefill"
}
}
},
"then": {
"required": [
"text"
]
},
"else": {
"if": {
"properties": {
"type": {
"const": "link"
}
}
},
"then": {
"required": [
"url"
]
}
},
"required": [
"text",
"type"
]
}
},
"required": [
"label",
"action"
]
}
},
"imageAction": {
"type": "object",
"description": "The image action to be executed when the user clicks on the card's image.",
"properties": {
"text": {
"type": "string",
"description": "The text to be sent when the user clicks on the image."
}
}
}
},
"required": [
"title",
"description"
]
}
},
"quickReplies": {
"type": "array",
"description": "An optional list of quick reply chips to display below the message.",
"items": {
"type": "object",
"properties": {
"label": {
"type": "string",
"description": "the text that is displayed to the user in each reply chip"
},
"action": {
"type": "object",
"description": "defines the text that is sent to the bot when the user clicks on the quick reply chip",
"properties": {
"text": {
"type": "string"
},
"send": {
"type": "boolean"
}
},
"required": [
"text"
]
}
},
"required": [
"label",
"action"
]
}
}
},
"required": [
"cards"
]
}
Example Payload:
{
"type": "card",
"metadata": {
"content": "Here are your top credit card recommendations:",
"cards": [
{
"title": "Rewards Platinum",
"description": "Earn 3x points on dining and travel.",
"imageUrl": "https://example.com/card1.png",
"buttons": [
{
"label": "Apply Now",
"action": { "type": "link", "url": "https://example.com/apply" }
}
]
}
]
}
}
3. Charts
Allows you to plot mathematical, scientific, or business data directly in the chat. Supported chart types include line, bar, area, pie, radar, and scatter.
Option A: Explicit Chart Rendering
Use this when you want to define exactly how the chart should look by providing the full schema.
Schema Overview:
content(string): Introductory text.title(string): The overall chart title.charts(array): Data series and configurations.chartId(string): Unique identifier (e.g.,"c1").type(string): e.g.,"bar"or"line".xAxis(string): Label for the horizontal axis.x_axis_identifier(string): The key mapping to the categories.series_identifiers(array): Keys for the data series.stacked(boolean): Usetruefor stacked bar charts.
Expected Metadata Schema
{
"type": "object",
"properties": {
"content": {
"type": "string",
"description": "An introductory message or summary that is displayed over the full set of charts"
},
"title": {
"type": "string",
"description": "An overall title for the set of charts"
},
"charts": {
"type": "array",
"description": "An array of chart objects to be displayed.",
"items": {
"type": "object",
"properties": {
"chartId": {
"type": "string",
"description": "a unique identifier given to the chart object, like c1, c2, and so on"
},
"title": {
"type": "string",
"description": "a title for this chart object"
},
"type": {
"type": "string",
"description": "the type of chart",
"enum": [
"line",
"bar",
"area",
"pie",
"radar",
"scatter"
]
},
"xAxis": {
"type": "string",
"description": "label to be used for the x-axis"
},
"x_axis_identifier": {
"type": "string",
"description": "The x-axis identifier of the data series to be displayed as the x-axis of this chart"
},
"series_identifiers": {
"type": "array",
"description": "The series identifiers of the single/multiple data series to be displayed with this chart",
"items": {
"type": "string",
"description": "Individual series identifier"
}
},
"stacked": {
"type": "boolean",
"description": "Set to True only for stacked bar graphs, skip if otherwise."
}
},
"required": [
"chartId",
"title",
"type",
"xAxis",
"x_axis_identifier",
"series_identifiers"
]
}
},
"quickReplies": {
"type": "array",
"description": "An optional list of quick reply chips to display below the message.",
"items": {
"type": "object",
"properties": {
"label": {
"type": "string",
"description": "the text that is displayed to the user in each reply chip"
},
"action": {
"type": "object",
"description": "defines the text that is sent to the bot when the user clicks on the quick reply chip",
"properties": {
"text": {
"type": "string"
},
"send": {
"type": "boolean"
}
},
"required": [
"text"
]
}
},
"required": [
"label",
"action"
]
}
}
},
"required": [
"content",
"title",
"charts"
]
}
Option B: LLM-Powered Chart Selection (Special Case)
When using the Renderer node to dynamically generate charts, you can pass a natural language query alongside the raw data. This ensures the plotting logic is separate from the data itself.
Example Payload:
{
"query": "plot line graphs: sales over time and profit over time",
"data": {
"sales": [100, 150, 200, 180],
"time": ["Jan", "Feb", "Mar", "Apr"],
"profit": [20, 30, 45, 35]
}
}
query: A natural language string describing how to plot the data. The LLM uses this to determine the chart type and axis mappings.data: An object containing the raw field names and their corresponding data series.
By using this structure, you ensure that the raw data is never processed by the LLM itself, but is correctly mapped and rendered by the platform's visualization engine.
4. Files
Allows you to display one or more files (PDFs, images, CSVs, etc.) directly in the chat with metadata and preview capabilities.
Schema Overview:
content(string): Introductory text displayed above the files.files(array): A list of file objects to display.name(string): The display name of the file.url(string): The direct download/preview URL.mime_type(string): The standard MIME type (e.g.,application/pdf).size(number, optional): The file size in bytes.
quickReplies(array, optional): Interactive suggestion chips.
Expected Metadata Schema
{
"type": "object",
"properties": {
"content": {
"type": "string",
"description": "An introductory message or summary that is displayed over the full set of files."
},
"files": {
"type": "array",
"description": "An array of file objects to be displayed.",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name of the file to be displayed to the user."
},
"url": {
"type": "string",
"description": "The file URL"
},
"mime_type": {
"type": "string",
"description": "The mime type of the file, like text/csv, application/pdf, audio/mpeg, and so on. This is used to determine how to preview the file.",
"enum": [
"text/plain",
"text/markdown",
"application/pdf",
"audio/mpeg",
"video/mp4",
"text/csv",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/zip"
]
},
"size": {
"type": "number",
"description": "The size of the file, which can be displayed to the user in the file preview."
}
},
"required": [
"name",
"url"
]
}
},
"quickReplies": {
"type": "array",
"description": "An optional list of quick reply chips to display below the message.",
"items": {
"type": "object",
"properties": {
"label": {
"type": "string",
"description": "the text that is displayed to the user in each reply chip"
},
"action": {
"type": "object",
"description": "defines the text that is sent to the bot when the user clicks on the quick reply chip",
"properties": {
"text": {
"type": "string"
},
"send": {
"type": "boolean"
}
},
"required": [
"text"
]
}
},
"required": [
"label",
"action"
]
}
}
},
"required": [
"content",
"files"
]
}
Example Payload:
{
"type": "files",
"metadata": {
"content": "Please find the requested report below:",
"files": [
{
"name": "CAM_Report.pdf",
"url": "https://example.com/files/cam_report.pdf",
"mime_type": "application/pdf"
}
]
}
}
5. Interactive Forms
Generates dynamic, interactive forms within the chat, enabling organized data collection (like date pickers, sliders, and file uploads).
Schema Overview:
content(string): Instructions shown above the form.form_schema(object): A standard JSON Schema defining the fields, types, and validation rules.form_ui_schema(object): Rules dictating how the fields are visually rendered (e.g., forcing a dropdown vs. radio buttons).
Expected Metadata Schema
{
"type": "object",
"properties": {
"content": {
"type": "string",
"description": "A brief message or instruction for the user to fill out the form."
},
"form_schema": {
"type": "object",
"description": "The JSON Schema that defines the form fields and their data types.",
"properties": {
"properties": {
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": [
"string",
"number",
"integer",
"boolean",
"array",
"object"
]
},
"title": {
"type": "string"
},
"format": {
"type": "string"
},
"enum": {
"type": "array",
"items": {
"anyOf": [
{
"type": "string"
},
{
"type": "number"
},
{
"type": "integer"
},
{
"type": "boolean"
}
]
}
}
},
"required": [
"title"
],
"additionalProperties": true
}
}
}
},
"form_ui_schema": {
"type": "object",
"description": "A UI json schema allowing any number of fields with common UI properties like widget, placeholder, and help text.",
"properties": {
"properties": {
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"ui:widget": {
"type": "string",
"description": "Specifies the form's UI widget, only mention file if applicable, skip for all remaining cases."
},
"ui:placeholder": {
"type": "string",
"description": "A hint for text-based inputs."
},
"ui:help": {
"type": "string",
"description": "A help message for the field."
},
"ui:title": {
"type": "string"
}
},
"additionalProperties": true
}
}
}
}
},
"required": [
"form_schema",
"form_ui_schema"
]
}
Form Rendering Rules & Field Combinations
The platform uses rendering priority rules to convert JSON schemas into UI elements.
Priority Order:
form_schema.format(Highest Priority)form_ui_schema.ui:widget- Auto-detection from
enum,minimum/maximum, etc. (Lowest Priority)
Common Form Equivalents
| JSON Schema Configuration | UI Widget Rendered | Example |
|---|---|---|
{"type": "string"} | Text Input | Simple text field, generic input |
{"type": "string", "enum": ["A", "B"]} | Dropdown | Automatically uses dropdown if enum is provided |
{"type": "string", "format": "email"} | Email Input | Enforces email format validation |
{"type": "string", "format": "date"} | Date Picker | Calendar interface |
{"type": "integer", "format": "range"} | Range Slider | A horizontal slider (minimum/maximum must be set) |
{"type": "array", "items": {"type": "string", "enum": [...]}} | Checkbox Group | Allows multiple selection |
{"type": "array", "format": "radio"} | Radio Buttons | Forces single-selection layout |
Using UI Overrides (form_ui_schema)
You can force a specific rendering widget using the form_ui_schema:
| Override Key & Value | Result | Use Case |
|---|---|---|
"ui:widget": "textarea" | Long Text Area | Replaces standard input with a multi-line text block. |
"ui:widget": "file" | File Upload | Spawns an attachment upload dropping zone. |
"ui:placeholder": "..." | Hint Text | Adds ghost placeholder text inside empty inputs. |
Form Configuration Example:
{
"type": "form",
"metadata": {
"content": "Please fill in the below form",
"form_schema": {
"type": "object",
"properties": {
"full_name": {
"title": "Full Name",
"type": "string"
},
"email_address": {
"title": "Email Address",
"type": "string",
"format": "email"
},
"secure_password": {
"title": "Secure Password",
"type": "string",
"format": "password"
},
"date_of_birth": {
"title": "Date of Birth",
"type": "string",
"format": "date"
},
"preferred_appointment_time": {
"title": "Preferred Appointment Time",
"type": "string",
"format": "time"
},
"experience_rating": {
"title": "Rate Your Overall Experience",
"type": "integer",
"format": "range",
"minimum": 1,
"maximum": 5
},
"budget_preference": {
"title": "Budget Preference",
"type": "integer",
"format": "range",
"minimum": 0,
"maximum": 10000
},
"gender": {
"title": "Gender",
"type": "string",
"enum": [
"Male",
"Female",
"Other"
]
},
"hobbies": {
"title": "Hobbies",
"type": "array",
"items": {
"type": "string",
"enum": [
"Sports",
"Music",
"Travel",
"Reading",
"Others"
]
}
},
"country_of_residence": {
"title": "Country of Residence",
"type": "string"
},
"profile_picture": {
"title": "Profile Picture",
"type": "string"
},
"supporting_documents": {
"title": "Supporting Documents",
"type": "string"
},
"personal_website_or_linkedin": {
"title": "Personal Website or LinkedIn Profile",
"type": "string",
"format": "uri"
}
}
},
"form_ui_schema": {
"full_name": {
"ui:placeholder": "Enter your full name"
},
"email_address": {
"ui:placeholder": "Enter your email address"
},
"secure_password": {
"ui:placeholder": "Choose a secure password"
},
"date_of_birth": {
"ui:placeholder": "Pick your date of birth"
},
"preferred_appointment_time": {
"ui:placeholder": "Select your preferred appointment time"
},
"experience_rating": {
"ui:placeholder": "Rate your experience (1–5 stars)"
},
"budget_preference": {
"ui:placeholder": "Enter your budget preference (0–10,000)"
},
"gender": {
"ui:placeholder": "Select your gender"
},
"hobbies": {
"ui:placeholder": "Choose your hobbies"
},
"country_of_residence": {
"ui:placeholder": "Select your country of residence"
},
"profile_picture": {
"ui:widget": "file",
"ui:placeholder": "Upload your profile picture"
},
"supporting_documents": {
"ui:widget": "file",
"ui:placeholder": "Upload supporting documents (PDF/DOC)"
},
"personal_website_or_linkedin": {
"ui:placeholder": "Share your personal website or LinkedIn profile URL"
}
}
}
}