LLM Text Complete

An LLM Text Complete task utilizes a large language model (LLM) to generate text predictions based on input context. The task configuration involves selecting an LLM provider, specifying the model, and defining the prompt and its variables. The fine-tuning parameters control the output, ensuring the generated text aligns with the desired randomness and length.

Prerequisites

Integrate the required AI model with Orkes Conductor.
Create the required AI prompt for the task.

Task parameters

Configure these parameters for the LLM Text Complete task.

Parameter Description Required/ Optional

inputParameters.llmProvider The integration name of the LLM provider integrated with your Conductor cluster.

Note: If you haven’t configured your AI/LLM provider on your Orkes Conductor cluster, go to the Integrations tab and configure your required provider. Required.

inputParameters.model The available language models within the selected LLM provider.

For example, If your LLM provider is Azure Open AI and you’ve configured text-davinci-003 as the language model, you can select it here. Required.

inputParameters.promptName The AI prompt created in Orkes Conductor.

Note: If you haven’t created an AI prompt for your language model, refer to the documentation on creating AI Prompts in Orkes Conductor. Required.

inputParameters.promptVariables For prompts that involve variables, provide the input to these variables as key-value pairs, where the key is the variable name defined in the prompt template. Values can be string, number, boolean, null, or object/array. For example: "promptVariables": {"country": "${workflow.input.country}"}. Optional.

inputParameters.temperature A parameter to control the randomness of the model’s output. Higher temperatures, such as 1.0, make the output more random and creative. A lower value makes the output more deterministic and focused.

Tip: If you're using a text blurb as input and want to categorize it based on its content type, opt for a lower temperature setting. Conversely, if you're providing text inputs and intend to generate content like emails or blogs, it's advisable to use a higher temperature setting. Optional.

inputParameters.stopWords List of words to be omitted during text generation. Supports string and object/array.

In LLM, stop words may be filtered out or given less importance during the text generation process to ensure that the generated text is coherent and contextually relevant. Optional.

inputParameters.topP

Another parameter to control the randomness of the model’s output. This parameter defines a probability threshold and then chooses tokens whose cumulative probability exceeds this threshold.

Example: Imagine you want to complete the sentence: “She walked into the room and saw a __.” The top few words the LLM model would consider based on the highest probabilities would be:

Cat - 35%
Dog - 25%
Book - 15%
Chair - 10%

If you set the top-p parameter to 0.70, the LLM model will consider tokens until their cumulative probability reaches or exceeds 70%. Here's how it works:

Add "Cat" (35%) to the cumulative probability.
Add "Dog" (25%) to the cumulative probability, totaling 60%.
Add "Book" (15%) to the cumulative probability, now at 75%.

At this point, the cumulative probability is 75%, exceeding the set top-p value of 70%. Therefore, the LLM will randomly select one of the tokens from the list of "Cat," "Dog," and "Book" to complete the sentence because these tokens collectively account for approximately 75% of the likelihood.

Optional.

inputParameters.maxTokens The maximum number of tokens to be generated by the LLM and returned as part of the result. A token is approximately four characters. Optional.

inputParameters.inputGuardrail

Scrubs or validates text before it's sent to the model.

Field	Description
`inputGuardrail.type`	The guardrail type: `JAVASCRIPT`, `HTTP`, or `WORKFLOW`.
`inputGuardrail.target`	The JS expression, URL, or workflow name to run, depending on `type`.
`inputGuardrail.version`	The workflow version to use. Applies only when `type` is `WORKFLOW`. Defaults to the latest version.
`inputGuardrail.headers`	Extra request headers to send, such as an API key. Applies only when `type` is `HTTP`.
`inputGuardrail.failureMode`	`FAIL` (default) fails the task if the guardrail fails. `WARN` logs the failure and continues with the original text.

Refer to LLM Task Guardrails for a full guide.

Optional.

inputParameters.outputGuardrail

Scrubs or validates the model's response text before it's returned.

Field	Description
`outputGuardrail.type`	The guardrail type: `JAVASCRIPT`, `HTTP`, or `WORKFLOW`.
`outputGuardrail.target`	The JS expression, URL, or workflow name to run, depending on `type`.
`outputGuardrail.version`	The workflow version to use. Applies only when `type` is `WORKFLOW`. Defaults to the latest version.
`outputGuardrail.headers`	Extra request headers to send, such as an API key. Applies only when `type` is `HTTP`.
`outputGuardrail.failureMode`	`FAIL` (default) fails the task if the guardrail fails. `WARN` logs the failure and continues with the original text.

Refer to LLM Task Guardrails for a full guide.

Optional.

The following are generic configuration parameters that can be applied to the task and are not specific to the LLM Text Complete task.

Caching parameters

You can cache the task outputs using the following parameters. Refer to Caching Task Outputs for a full guide.

Parameter	Description	Required/ Optional
cacheConfig.ttlInSecond	The time to live in seconds, which is the duration for the output to be cached.	Required if using cacheConfig.
cacheConfig.key	The cache key is a unique identifier for the cached output and must be constructed exclusively from the task’s input parameters. It can be a string concatenation that contains the task’s input keys, such as `${uri}-${method}` or `re_${uri}_${method}`.	Required if using cacheConfig.

Other generic parameters

Here are other parameters for configuring the task behavior.

Parameter	Description	Required/ Optional
optional	Whether the task is optional. If set to`true`, any task failure is ignored, and the workflow continues with the task status updated to `COMPLETED_WITH_ERRORS`. However, the task must reach a terminal state. If the task remains incomplete, the workflow waits until it reaches a terminal state before proceeding.	Optional.

Task configuration

This is the task configuration for an LLM Text Complete task.

{
     "name": "llm_text_complete",
     "taskReferenceName": "llm_text_complete_ref",
     "inputParameters": {
       "llmProvider": "openAI",
       "model": "chatgpt-4o-latest",
       "promptName": "translate",
       "temperature": 1,
       "topP": 0.8,
       "maxTokens": 150,
       "stopWords": [
         "a",
         "and",
         "the"
       ]
     },
     "type": "LLM_TEXT_COMPLETE"
}

Task output

The LLM Text Complete task will return the following parameters.

Parameter	Description
result	The completed text by the LLM.
finishReason	Indicates why the text generation stopped. Common values include STOP when the model completes naturally.
tokenUsed	Total number of tokens consumed for the request, including both prompt and completion tokens.
promptTokens	Number of tokens used to process the prompt.
completionTokens	Number of tokens generated by the model in the output text returned by the task.

Examples

Here are some examples for using the LLM Text Complete task.

Using an LLM Text Complete task in a workflow

See an example of building an AI-powered translator using the LLM Text Complete task.

LLM Text Complete

Task parameters

Task configuration

Task output

Examples

Related pages