File

This source reads metadata from a file generated by the File sink.

Use this source to decouple metadata extraction, performed with a File sink from pushing this metadata, using this source.

Sample recipe

Example 1. File Source sample recipe.

source:
  type: "file"
  config:
    path: ./path/file.json

sink:
  # sink configuration

Parameters

The following table lists the source parameters.

Parameter Mandatory Description

Parameter	Mandatory	Description
`path`	Yes	Path to a source file or a folder containing source files. If a folder is specified, all files with the `file_extension` are processed.
`aspect`	No	Only read this aspect from the source file.
`count_all_before_starting`	No	If set to `true`, count the total number of records in the file before starting. This option is used for processing time estimation. You can disable this option to speed up startup time. Defaults to `true`.
`file_extension`	No	When a folder is specified in the `path`, optionally set this field with the file extension to process. Use `*` to process any extension. Defaults to `.json`.

path

Yes

Path to a source file or a folder containing source files. If a folder is specified, all files with the file_extension are processed.

aspect

Only read this aspect from the source file.

count_all_before_starting

If set to true, count the total number of records in the file before starting. This option is used for processing time estimation. You can disable this option to speed up startup time. Defaults to true.

file_extension

When a folder is specified in the path, optionally set this field with the file extension to process. Use * to process any extension. Defaults to .json.