Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# How to use the Transform Grok plugin

Parse unstructured text — log lines, error messages, raw event data — into structured fields using Grok patterns.

## Tasks

`TransformValue` applies one or more Grok patterns to a single string value passed via `from` and returns the captured named fields as a map. Use it to parse a single string already in your flow's variables.

`TransformItems` reads a file from Kestra internal storage (passed as a `kestra://` URI via `from`), applies the pattern to each line, and writes the structured results back to storage. Use it for batch log parsing or event normalization.

Both tasks accept `pattern` for a single Grok expression or `patterns` for a list of expressions tried in order — matching stops at the first successful pattern. Add custom pattern definitions with `patternDefinitions` (a map of name to regex) or point `patternsDir` at directories containing pattern files. By default only named capture groups are returned (`namedCapturesOnly: true`) and matching stops at the first successful pattern (`breakOnFirstMatch: true`).
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# How to use the Transform JSONata plugin

Query and transform JSON data using [JSONata](https://jsonata.org/) expressions — a functional language for selecting, filtering, reshaping, and aggregating JSON structures.

## Tasks

`TransformValue` evaluates a JSONata `expression` against a single JSON string passed via `from` and returns the transformed result. Use it to reshape or extract values from a task output already in your flow.

`TransformItems` reads a JSON or Ion file from Kestra internal storage (passed as a `kestra://` URI via `from`), evaluates the `expression` against each record, and writes the results back to storage. If the expression returns an array and `explodeArray` is `true` (the default), each array element is written as a separate record — set `explodeArray: false` to write the whole array as a single record instead. Use `TransformItems` for batch transformations over large datasets.
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# How to use the Transform plugin

These tasks let you reshape, filter, combine, flatten, and summarize structured data without writing custom code. They work both with in-memory records and with stored Ion files, so the same tasks can be used for small workflow variables and larger streamed datasets.

## Common Inputs And Outputs
Expand Down
Loading