Designing Flows

Overview

Visual data flows are a graphical representation of data transformation processes using a series of interconnected nodes. Each node in the pipeline represents a specific data transformation action or operation, such as filtering, sorting, or aggregating data.

The purpose of visual data pipelines is to provide a user-friendly, intuitive way to design, configure, and understand complex data processing tasks without the need for extensive programming knowledge.

Key aspects of visual data pipelines

Nodes. Nodes are the fundamental building blocks of a visual data pipeline. Each node serves a specific purpose and encapsulates a particular data transformation operation, such as filtering rows based on specific conditions, sorting columns, or adding new columns using expressions.

Connections. In a visual data pipeline, nodes are connected to define the data flow through the transformation process. The output of one node becomes the input of the next node in the sequence, allowing you to chain together multiple transformations to achieve the desired result.

Visual Interface. The primary advantage of a visual data pipeline is its user-friendly graphical interface. You can add nodes on a canvas with one click, connect them in the desired order, and configure the settings for each node using a property grid.

Real-time Preview. Our data pipelines include a real-time preview feature that allows you to see the impact of their transformations on a sample of the dataset instantly. This helps you understand the results of their configuration choices and adjust settings as necessary.