Quantcast
Channel: Developer Express Inc.
Viewing all articles
Browse latest Browse all 2390

DevExpress AI-Powered Extensions — Extending Text Editors with AI Capabilities — Early Access Preview (v24.2)

$
0
0

This post highlights exciting initiatives underway at DevExpress in one of the hottest and most "hyped" fields in recent years — Artificial Intelligence (AI) and Natural Language Processing (NLP).

NLP, a branch of AI, allows computers to interact with human languages. This includes the ability to understand, interpret, generate, and respond to text (and speech) in ways that are meaningful/useful. NLP-based features allow for better data analysis, personalized experiences, efficient communication and lead to more informed decision-making and increased productivity. Examples include:

  • Machine Translation
  • Text Summarization
  • Text Generation
  • Text Classification, and more...

Our goal at DevExpress is to provide a lightweight subset of APIs for seamless interaction with multiple AI services, including OpenAI, Azure OpenAI, and self-hosted models (Ollama). We’re also working to leverage these APIs and integrate NLP-based capabilities into multiple UI components, including RichEdit, MemoEdit, HtmlEditor, LayoutControl, GridControl, and more.

In this post, I’ll walk you through what’s available for testing in our Early Access Preview build (our v24.2 EAP is available to active Universal or DXperience subscribers).

Early Access and CTP builds are provided solely for early testing purposes and are not ready for production use. This build can be installed side by side with other major versions of DevExpress products. Please back up your project and other important data before installing Early Access and CTP builds. This EAP may not include all features/products we expect to ship in our v24.2 release cycle. As its name implies, the EAP offers an early preview of what we expect to ship in the upcoming months.

Getting Started with DevExpress AI-Powered Extensions

Prerequisites

DevExpress AI APIs allow you to register and use multiple AI services within your WinForms, WPF, or Blazor application. Our Early Access Preview build (v24.2.1) supports the following AI services:

  • OpenAI
  • Azure OpenAI
  • Ollama

AI services and DevExpress AI extensions described in this post follow the "bring your own key" principle. DevExpress does not offer a REST API and does not ship any built-in LLMs/SLMs. You need to have an active Azure/Open AI subscription to obtain the REST API endpoint, key, and model deployment name. These variables must be specified at application startup to register AI clients and enable DevExpress AI-Powered Extensions in your application.

To get started, you need to meet the following prerequisites:

The following table lists DevExpress AI-related NuGet packages:

Package NameDescription
DevExpress.AIIntegrationImplements core functionality for DevExpress AI-powered extensions and services.
DevExpress.AIIntegration.OpenAIImplements APIs for OpenAIClient and OpenAIAssistant.
DevExpress.AIIntegration.Azure.OpenAIImplements APIs for AzureOpenAIClient.
DevExpress.AIIntegration.OllamaImplements services for self-hosted Ollama models.
DevExpress.AIIntegration.Azure.TextAnalyticsImplements services for Azure AI Language.
DevExpress.AIIntegration.Azure.TranslationImplements services for Azure AI Translator.
DevExpress.AIIntegration.DesktopImplements a container for DevExpress AI Extensions and shared components for WinForms and WPF.
DevExpress.AIIntegration.WebImplements APIs for AI-powered features in DevExpress-powered Web applications.
DevExpress.AIIntegration.Blazor.CommonImplements a dialog window to display AI responses within Blazor applications.
DevExpress.AIIntegration.Blazor.HtmlEditorImplements APIs for AI-powered extensions in the DevExpress Blazor HTML Editor component.
DevExpress.AIIntegration.Blazor.RichEditImplements APIs for AI-powered extensions in the DevExpress Blazor RichEdit component.
DevExpress.AIIntegration.WinFormsImplements APIs for AI-powered extensions in DevExpress WinForms controls.
DevExpress.AIIntegration.WPFImplements APIs for AI-powered extensions in DevExpress WPF controls.

Register AI Clients

To use DevExpress AI-Powered Extensions in a console application, you need to install one of the following DevExpress packages and register an AI client:

  • DevExpress.AIIntegration
  • DevExpress.AIIntegration.OpenAI, DevExpress.AIIntegration.Azure.OpenAI, or DevExpress.AIIntegration.Ollama (based on your preferences)

DevExpress AI-Powered Extensions operate within an AIExtensionsContainerDefault container. This container manages registered AI clients so that DevExpress UI components can automatically leverage AI services.

The following code snippet registers an Azure OpenAI client:

using Azure.AI.OpenAI;
using DevExpress.AIIntegration;
...
//Modify the following lines to obtain and pass your personal Azure OpenAI credentails to the Register* method.
string AzureOpenAIEndpoint { get { return Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT"); } }
string AzureOpenAIKey { get { return Environment.GetEnvironmentVariable("AZURE_OPENAI_APIKEY"); } }
string DeploymentName { get { return Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENTNAME"); } }

AIExtensionsContainerDefault defaultAIContainer;
...
    
static void Main()    { 
    defaultAIContainer = new AIExtensionsContainerDefault();	
    defaultAIContainer.RegisterChatClientOpenAIService(
        new AzureOpenAIClient(new Uri(AzureOpenAIEndpoint), 
        new System.ClientModel.ApiKeyCredential(AzureOpenAIKey)), 
        DeploymentName
    );
}

To use offline Ollama models, call the RegisterChatClientOllamaAIService method. This method registers an Ollama client. In this method, the first parameter is the default Ollama API Endpoint. The second parameter specifies the model name:

defaultAIContainer.RegisterChatClientOllamaAIService("http://localhost:11434/api/chat", "llama3.1");

Once you register an AI client, you can utilize DevExpress AI-Powered Extensions as follows (DevExpress AI-Powered Extensions are listed below):

var result = await defaultAIContainer.TranslateAsync(new TranslateRequest(textToTranslate, language));
//OR
var translateExtension = defaultAIContainer.CreateTranslateExtension();
TranslateRequest request = new TranslateRequest(textToTranslate, language);
var result = await translateExtension.ExecuteAsync(request, CancellationToken.None);

Manage Long Input Text and AI Responses

Working with large text blocks can be challenging, but we've got you covered. Our AI-Powered Extensions automatically split large content into manageable chunks (including paragraphs, sentences, words, punctuation marks, and other text elements) to ensure accurate processing. Here's how you can manage large text blocks and handle AI service responses:

public async Task<string> GetTranslation(string textToTranslate, string language) {
    TextResponse result = await defaultAIContainer.TranslateAsync(new TranslateRequest(textToTranslate, language));
    if(result.IsCompleted)
        return result.Response;
    if(!result.IsRestrictedOrFailed) {
        string translatedText = result.Response;
        while(result.IsContinuationRequired){
            await result.ContinueAsync();
            translatedText += result.Response
        }
        return translatedText;
    }
    //Something unexpected happened
    switch (result.Status) {
        case ResponseStatus.MaxTokenLimitExceeded:
        case ResponseStatus.InputSizeLimitExceeded:
            return "The text you're trying to send within a request is too long and exceeds the allowed limit.";
        case ResponseStatus.ContentFiltered:
            return "Potentially harmful content was detected in your request.";
        case ResponseStatus.Error:
            return "An error occurred while processing the request.";
    }
    throw new NotSupportedException();
}

Important responses include:

  • MaxTokenLimitExceeded — Indicates that the AI service response was truncated because it exceeds the allowed limit (specifically from the model). This situation arises when combined input size (request) and output (response) exceeds the maximum number of tokens the model can process in one interaction. For example, GPT-4o LLM has a context window of 8K-128K tokens depending on the chosen plan, while Microsoft's small-language model (SLM), such as Phi-3 Mini, can handle up to 4K tokens.
  • InputSizeLimitExceeded — Indicates that AI service response was truncated by our 'engine' to prevent excessive resource consumption. This allows you to avoid instances where users might unintentionally consume all Azure credits by submitting overly large text requests. To modify this limit, you can explicitly set the ChatMaxTokensDefault property, aligning it with corporate policies and model context window size.

Use the following settings to configure size limits for text content and images:

public static partial class AIIntegration {
    //Specifies the maximum number of tokens that can be processed by a model request. Applies to all DevExpress AI-Powered extensions.
    public static int? ChatMaxTokensDefault { get; set; } = null;
    //Specifies the default temperature for chat-based AI-Powered extensions. 
    //To learn more please visit https://ai.stackexchange.com/questions/32477/what-is-the-temperature-in-the-gpt-models
    public static float? ChatTemperatureDefault { get; set; } = null;
    //Specifies the maximum allowed size of the text buffer per request, in bytes.
    public static int TextBufferMaxSize { get; set; } = 64 * 1024;//64kb
    //Specifies the maximum allowed size of the image buffer per request, in bytes.
    public static int ImageBufferMaxSize { get; set; } = 2 * 1024 * 1024;//2mb
    //Specifies the maximum allowed chunk length, in characters.
    public static int ChunkMaxLength { get; set; } = 6000;
}

DevExpress AI-Powered Extensions

Our EAP v24.2 ships with the following AI-Powered Extensions:

ExtensionAPI MethodRequest Type
Smart-AutoCompleteAutoCompleteAsyncAutoCompleteRequest
Adjust ToneToneStyleAsyncToneStyleRequest
ExpandExpandAsyncExpandRequest
ExplainExplainAsyncExplainRequest
Proofread ProofreadAsyncProofreadRequest
Rewrite (Change Writing Style)RewriteStyleAsyncRewriteStyleRequest
Shorten ShortenAsyncShortenRequest
SummarizeAbstractiveSummaryAsync
ExtractiveSummaryAsync
AbstractiveSummaryRequest
ExtractiveSummaryRequest
TranslateTranslateAsyncTranslateRequest
Get Image DescriptionGenerateDescriptionAsyncGenerateDescriptionRequest
Ask AICustomPromptAsyncCustomPromptRequest

These extensions work in a chat-like manner - where requests are sent with built-in prompts to OpenAI/Ollama models.

In addition to providing access to AI models hosted in Azure and Ollama, we have implemented APIs that interact with Azure AI Language services, including Azure AI Translator and Azure AI Language (text summarization). The choice of service depends on your budget. However, please note that when calling the following methods, text translation and summarization requests will be sent exclusively to Azure AI Language services:

string azureTranslatorEndpoint = GetEnvironmentVariable("AZURE_TRANSLATOR_ENDPOINT");
string azureTranslatorKey = GetEnvironmentVariable("AZURE_TRANSLATOR_API_KEY");
//Register an Azure.TextAnalytics client.
defaultAIContainer.RegisterTranslatorAzureAIService(new TextTranslationClient(new AzureKeyCredential(azureTranslatorKey), new Uri(azureTranslatorEndpoint)));

//Register an Azure.TextAnalytics client.
string azureTextAnalyticsEndpoint = GetEnvironmentVariable("AZURE_TEXT_ANALYTICS_ENDPOINT");
string azureTextAnalyticsKey = GetEnvironmentVariable("AZURE_TEXT_ANALYTICS_API_KEY");
defaultAIContainer.RegisterTextAnalyticsAzureAIService(new TextAnalyticsClient(new Uri(azureTextAnalyticsEndpoint), new AzureKeyCredential(azureTextAnalyticsKey)));

Customize and Extend DevExpress AI-Powered Extensions

Our AI-powered extensions ship with pre-built prompts that can be customized as needed:

  1. Derive from an AI-powered extension and override the GetSystemPrompt method:

    
        public class WilliamShakespeareStyleExtension : RewriteStyleExtension {
        	public WilliamShakespeareStyleExtension(IServiceProvider serviceProvider) : base(serviceProvider) { }
        	protected override string GetSystemPrompt(RewriteStyleRequest request) {
       			return "Rewrite this text in the William Shakespeare style.";
        	}
        }
        
  2. Register your extension within the default container (this overrides the built-in extension):

    defaultAIContainer.Register<RewriteStyleRequest, WilliamShakespeareStyleExtension>();

When you call the RewriteStyleAsync method, your custom prompt will be sent to the AI service along with user input.

TIP: To learn more about prompts, refer to the following:

At this stage, another important element of DevExpress AI-related APIs comes into play: the AIExtensionsContainerLocal class. Similar to the default extensions container, the AIExtensionsContainerLocal class stores AI-powered extensions that you implement and register manually, without replacing DevExpress built-in extensions.

The key difference is that extensions included in this container are not automatically used by DevExpress UI components. This allows you to implement different behaviors for various parts of your application. The following code snippet implements a custom AI-powered extension (notice how I use base classes):

public class AuthoredStyleExtension : ChangeTextExtension<AuthoredStyleRequest> {
    public AuthoredStyleExtension(IServiceProvider serviceProvider) : base(serviceProvider) { }
    protected override string GetSystemPrompt(AuthoredStyleRequest request) {
        return $"Rewrite this text in the {request.Author} style";
    }
}
//A custom request that processes text.
public class AuthoredStyleRequest : TextRequest {
    public AuthoredStyleRequest(string Author, string Text) : base(Text) {
        this.Author = Author;
    }
    public string Author { get; init; }
}

public static class CustomAIIntegration {
    public static AuthoredStyleExtension CreateAuthoredStyleExtension(this IAIExtensionsContainer container){
        return (AuthoredStyleExtension)container.GetExtension(typeof(AuthoredStyleRequest));
    }
}

You can implement your own TextRequest descendant to supply additional information to an AI model when constructing a prompt. Examples include built-in ChangeTone and RewriteStyle extensions that obtain enumeration values used to specify tone (e.g., Casual, Friendly, Confident) and style (e.g., Formal, Conversational). To invoke custom extensions at runtime, use the following code snippet:

var localContainer = new AIExtensionsContainerLocal(defaultAIContainer);
localContainer.Register<AuthoredStyleRequest, AuthoredStyleExtension>();

AuthoredStyleExtension extension = localContainer.CreateAuthoredStyleExtension();
var request = new AuthoredStyleRequest("Mark Twain", textToModify);

//Here you can also set up the Temperature variable that will be used to execute this particular model request
request.Options.Temperature = 0.9f;
string result = await extension.ExecuteAsync(request, CancellationToken.None);

Localization

DevExpress AI-Powered Extensions support localization. This helps ensure that AI-driven features are accessible and relevant to users across different geographies. By localizing AI-powered extensions, you can adapt the user interface and any predefined content to meet the linguistic and cultural preferences of your end users. And yes, you can also customize prompts based on regional preferences (for example, when using single-language AI models).

Enable AI-Powered Extensions in the UI

In this section, I will show you how to enable DevExpress AI-Powered Extensions in DevExpress UI components for WinForms, WPF, and Blazor. We've designed different ways to interact with our components and integrate the aforementioned AI-powered capabilities into your DevExpress-powered apps.

WinForms — AI-Powered Text Transform

AI-powered text transform extensions (behaviors) can be attached to the following DevExpress WinForms controls:

Once you install the DevExpress AI Client NuGet package, you should install the following DevExpress WinForms NuGet packages:

  • DevExpress.AIIntegration.WinForms
  • DevExpress.Win.Design (enables design-time features for DevExpress WinForms UI controls)

Once installed, you'll need to register an AI client at application startup. DevExpress AI-powered extensions operate within a AIExtensionsContainerDesktop container. This container manages all registered AI clients so that DevExpress WinForms UI controls can automatically leverage appropriate AI services:

AIExtensionsContainerDesktop.Default.RegisterChatClientOpenAIService(
    new AzureOpenAIClient(
        new Uri(azureOpenAIEndpoint), 
        new System.ClientModel.ApiKeyCredential(azureOpenAIKey)),
    deploymentName);

In the Toolbox, locate the BehaviorManager component and drop it onto a form. Open the control's smart tag menu and click "Edit Behaviors". In the Collection Editor that appears on screen, create required AI-powered behaviors, attach them to corresponding UI controls, and configure behavior settings (for example, Temperature).

WinForms MemoEdit — Add AI-Powered Behaviors, DevExpress

TIP: If you did not install the DevExpress.AIIntegration.WinForms package, the BehaviorManager's smart-tag menu displays "Register AI-Powered Behaviors". Click it to install the NuGet package (or assemblies in .NET Framework apps).

WinForms Behavior Manager — Add Behaviors

The following video demonstrates how to add the AI-powered Translate behavior to a DevExpress WinForms MemoEdit (and how to use AI-powered features at runtime):

Review the following help topic for additional information/guidance: WinForms — AI-Powered Extensions.

WinForms — Smart Paste

"SmartPaste" is an AI-driven feature that analyzes copied information and intelligently assigns the right values to appropriate data fields/row cells in the DevExpress Data Grid or to Layout Control-based forms.

The following video illustrates this capability in action: 

Review the following help topic for additional information/guidance: WinForms — AI-Powered Extensions — Smart-Paste.

WPF — AI-Powered Text Transform

For our Early Access Preview build (v24.2), AI-powered functionality can be enabled in the DevExpress WPF RichEdit, SpreadsheetControland TextEdit components with the TextWrapping property set to Wrap for multi-line editing capabilities.

Once you install a DevExpress AI Client NuGet package, you should install the following DevExpress WPF NuGet packages:

  • DevExpress.AIIntegration.Wpf
  • DevExpress.Wpf

Much like WinForms, register an AI client at application startup within the AIExtensionsContainerDesktop container:

App.xaml.cs


protected override void OnStartup(StartupEventArgs e) {
    base.OnStartup(e);
    ApplicationThemeHelper.ApplicationThemeName = "Office2019Colorful";

    AIExtensionsContainerDesktop.Default.RegisterChatClientOpenAIService(
        new AzureOpenAIClient(
        	new Uri(azureOpenAIEndpoint),
            new System.ClientModel.ApiKeyCredential(azureOpenAIKey)),
        deploymentName);
}

To enable AI-Powered functionality in DevExpress WPF components, add the following namespaces to the Window that hosts them:


xmlns:dxmvvm="http://schemas.devexpress.com/winfx/2008/xaml/mvvm"
xmlns:wpf="clr-namespace:DevExpress.AIIntegration.Wpf;assembly=DevExpress.AIIntegration.Wpf.v24.2"
xmlns:desktop="clr-namespace:DevExpress.AIIntegration.Desktop;assembly=DevExpress.AIIntegration.Desktop.v24.2"

And attach behaviors to the control in the following manner:

<dxe:TextEdit TextWrapping="Wrap">
    <dxmvvm:Interaction.Behaviors>
        <wpf:SummarizeBehavior/>
        <wpf:ExplainBehavior/>
        <wpf:ShortenBehavior/>
        <wpf:ExpandBehavior />
        <wpf:ToneStyleBehavior />
        <wpf:ProofreadBehavior/>
        <wpf:RewriteStyleBehavior/>
        <wpf:CustomRequestBehavior/>
        <wpf:TranslateBehavior>
            <desktop:LanguageInfo Culture="de-DE"/>
            <desktop:LanguageInfo Culture="es-ES"/>
            <desktop:LanguageInfo Culture="pt-BR"/>
        </wpf:TranslateBehavior>
    </dxmvvm:Interaction.Behaviors>
</dxe:TextEdit>

Blazor

Once you installed a DevExpress AI Client NuGet package, you have to install the following DevExpress Blazor NuGet packages:

  • DevExpress.AIIntegration.Web
  • DevExpress.AIIntegration.Blazor.Common
  • DevExpress.AIIntegration.Blazor
  • DevExpress.AIIntegration.Blazor.RichEdit and/or DevExpress.AIIntegration.Blazor.HtmlEditor

Call the AddDevExpressAI method at application startup to register AI services for the Blazor platform:

Startup.cs:

using DevExpress.AIIntegration;
...
string azureOpenAIEndpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");
string azureOpenAIKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY");
string deploymentName = "YourModelDeploymentName"
...
builder.Services.AddDevExpressAI((config) => {
    var client = new AzureOpenAIClient(
        new Uri(azureOpenAIEndpoint),
        new AzureKeyCredential(azureOpenAIKey));
    config.RegisterChatClientOpenAIService(client, deploymentName);
    config.RegisterOpenAIAssistants(client, deploymentName);
});

Open the Razor page with DxRichEdit and/or DxHtmlEditor and declare AdditionalSettings as follows:

DxHtmlEditor:

@using DevExpress.AIIntegration.Blazor.HtmlEditor

<DxHtmlEditor @bind-Markup="Value" CssClass="my-editor" BindMarkupMode="HtmlEditorBindMarkupMode.OnLostFocus">
    <AdditionalSettings>
        <SummaryAISettings />
        <ExplainAISettings />
        <ProofreadAISettings />
        <ExpandAISettings />
        <ShortenAISettings />
        <CustomAISettings />
        <RewriteAISettings />
        <ToneAISettings />
        <TranslateAISettings Languages="@("German, French, Chinese")" />
    </AdditionalSettings>
</DxHtmlEditor>

DxRichEdit:

@using DevExpress.AIIntegration.Blazor.RichEdit
@using DevExpress.Blazor.RichEdit

<DxRichEdit DocumentContent="DocumentContent" CssClass="my-editor">
    <AdditionalSettings>
        <SummaryAISettings />
        <ExplainAISettings />
        <ProofreadAISettings />
        <ExpandAISettings />
        <ShortenAISettings />
        <CustomAISettings />
        <RewriteAISettings />
        <ToneAISettings />
        <TranslateAISettings Languages="@("German, French, Chinese")" />
    </AdditionalSettings>
</DxRichEdit>

The following videos demonstrate DevExpress AI-Powered Extensions within a DevExpress-powered Blazor app:

User Experience with Large Input Text

To prevent resource-heavy operations (both in terms of API costs and processing time), our implementation automatically breaks large text blocks into chunks (e.g., paragraphs, sentences, words, etc.), based on the AIIntegration.ChatMaxTokensDefault property value. When a user selects text that exceeds the defined limit, they are warned that the request cannot be processed in a single operation, and the system calculates the number of stages required to process the task. Once the dialog appears, users can choose to process text sequentially and view results step by step, or complete all operations simultaneously. The following video demonstrates this functionality in action:

This functionality is implemented for all supported platforms (WinForms, WPF and Blazor).

Download a Sample Project

To help you get started, we created a couple of examples that integrate/enable DevExpress AI-Powered Extensions for WinForms, WPF, and Blazor components. Should you encounter an issue while using our Early Access Preview build (v24.2), feel free to submit a support ticket via the DevExpress Support Center. We will be happy to follow up. 

Your Feedback Counts

As always, we welcome your feedback. Please complete the following short survey and help us shape the future of AI-related development here at DevExpress.




Viewing all articles
Browse latest Browse all 2390

Trending Articles