Scroll To Top

Orchestration Tasks — Dataset to Table

Introduction

Foreflow can be configured to sync data between a Dataset and a Table.

Dataset Column Names

Because of how foreflow handles internal metadata tracking, certain column names are not allowed on a Dataset that is set up to sync its data to a table. If the following column names exist in a Dataset, this type of task cannot be created:

  • id
  • created_at
  • updated_at
  • deleted_at
  • created_by
  • last_updated_by

This limitation can be overcome by providing an alias for a column with one of the above names:

    select
        id as employee_id
        ,created_at as record_created_at

    from
        employees;

Creating a Dataset to Table Task

To create a Dataset to Table Task, you will need to have already created the Dataset.

On the Tasks page, you can create a new task by clicking the "Add Task" button. You will be presented with a dialogue where you can specify a name and description for the task. To set this task up as a Dataset to Table Task, you will select "Datasets" from the Task Group dropdown, and "Dataset to Table" from the Task Type dropdown. Selecting these two items will expand the form for further configuration.

You will next select the Dataset that you would like to sync to a Table, and provide a name for the destination Table.

Syncing Dataset Changes

Foreflow will automatically sync the structure of the Dataset to the destination table. For example, if a Dataset to Table task already exists for a particular Dataset, if a column is added to the Dataset, that change will be reflected in the Table the next time the task is executed.