Reading only a specifed range of records

The File Metadata offers the possibility to define filters, that allows to specify a range of records to process.

This can be useful to define more precisely the records to treat when loading files.

Metadata Configuration

To define the range filter:

  1. Open the File Metadata

  2. Add a Filter on the Record, or update an existing one

  3. Set the First Line and Last Line Parameters

metadata

  • You can use negative values like -10 in these parameters to filter from the end of the file

  • If the First Line parameter is not set, it will be considered as '0' (read everything from the beginning).

  • If the Last Line parameter is not set, it will be read until the last line of the file.

  • This filter only applies when reading the file, not when writing it.

That’s it! From now, when the file will be read in a Mapping or process, only the specified lines will be retrieved.