Semarchy.com Tutorials Community
Start Now

Semarchy xDI

  • Introduction to Semarchy xDI
    • xDI architecture
  • Release notes
    • xDI 2025.1 release notes
    • xDI 2024.4 release notes
    • xDI 2024.3 release notes
    • xDI 2024.2 release notes
    • xDI 2024.1 release notes
    • xDI 2023.4 release notes
    • xDI 2023.3 release notes
    • xDI 2023.2 release notes
    • xDI 2023.1 release notes
    • xDI 5.3 release notes
  • Set up Semarchy xDI
    • Install xDI
      • System requirements
      • Install the Designer
      • Install components
      • Install xDI Runtime
      • Install Analytics
      • Install the license server
    • Configure xDI
      • Configure xDI Designer
        • Manage licenses
        • Choosing your Java instance
      • Configure xDI Runtime
        • Keystores and encryption
        • Runtime access and security
        • Delivery repositories
        • External value resolvers
        • Configure Java options
        • Runtime services
        • Runtime scheduler
        • Session and debug logging
        • Change the Jython version
        • Change the temporary folder
        • Runtime parameters reference
      • Configure the License Server
    • Upgrade xDI
      • Upgrade the License Server
      • Upgrade Designer
      • Upgrade the Runtime
      • Upgrade Analytics
  • Design integration flows
    • Get started with Semarchy xDI Designer
      • Resource concepts
      • Design-time views
      • Runtime views
      • Demo databases
    • Work with projects
      • Change tracking
      • Source control
      • Advanced features
    • Connect to your data
      • Set up your environment
        • Install components
        • Install and manage modules
        • Import templates
      • Metadata and reverse engineering
        • Define a database model
        • Define a file model
        • Define an XML model
        • Work with Sub-metadata
      • Enable data quality
      • Externalize Metadata Values
    • Work with mappings
      • Create mappings
      • Filters
      • Join sources
      • Template parameters
      • Advanced Topics
        • Change Data Capture (CDC)
        • Rejects management
        • Serialize source data
        • Stage the sources
        • Deserialize source data
        • User-defined functions
    • Work with processes
      • Create a process
      • Execution flow
      • Work with sub-processes
      • Step repetition
      • Parameters
      • Linked Metadata
      • Scripting
        • Scripting in processes
        • Scripting context
        • Additional libraries
    • Work with variables
    • Work with Metadata configurations
    • Override internal resources
  • Components
    • Semarchy xDM
      • Release notes
      • Getting started
    • Actian Avalanche
      • Release notes
      • Getting started
    • Actian VectorWise
      • Release notes
      • Getting started
    • Amazon
      • Release notes
      • Amazon Redshift
        • Getting started
        • Loading data via S3 buckets
      • S3
        • Getting started
    • Amazon S3 Delivery Repository
      • Release notes
    • Amazon Secret Manager
      • Release notes
    • AMQP
      • Release notes
      • Getting started
    • Avro
      • Release notes
      • Getting started
    • Base
      • Release notes
      • Certificates and Keys
        • Getting started
      • Email
        • Getting started
        • Sending an email with Semarchy xDI
        • Sending emails with Gmail SMTP servers
        • Sending the result of a SELECT query by email
        • Reading and sending Outlook 365 emails
      • Files
        • Flat files
          • xDI Designer for Flat Files
          • Sorting data when writing a flat File
          • Using File Property Fields
          • Reading Multiple files at the same time
          • Generating files with dynamic name
        • Multirecord files
          • Getting started with hierarchical files
          • Loading data when source and target are hierarchical Files
          • Reading only a specifed range of records
          • Using Computed Fields
          • Ordering data when loading a hierarchical file
        • Scripting
          • Getting Started With Transformation Scripting
          • Removing an UTF BOM header
        • Remove Header/Footer from files concatenated with Concat Files action
        • Deleting a directory with the FileDelete Action
      • FTP
        • Getting started
        • Using certificates with FTPS
        • FTP results in execution conditions
        • Retrieve files with different masks
      • HSQL
        • Getting started
      • HTTP Security
        • Getting started
        • OAuth2 advanced options
      • JSON
        • JSON property fields
        • Reading JSON nodes with varying names
        • Reading Multiple JSON files
        • Sorting data when writing a JSON File
      • Kerberos
        • Getting started
      • LDAP
      • ODBC
        • Work with ODBC datasources
      • Proxy
        • Getting started
      • SSH
        • Getting started
      • XML
        • XML property fields
        • Reading XML nodes with dynamic names
        • Reading Multiple XML files
        • Validating an XML file with an XSD
        • Including empty or null values in XML files
        • Indenting an XML File using an XSL Transformation
        • Sorting data when writing an XML File
      • WSDL
        • Working with REST in xDI (legacy)
          • Web services reverse wizard
          • Invocation error handling
          • Tips for calling an HTTP REST service
          • Customize the HTTP verb
          • Retrieving HTTP headers
          • Retrieving HTTP response codes and messages
          • Investigating a REST invocation issue
          • Retrieving raw, unstructured data
          • Sending raw, unstructured data
    • Cassandra
      • Release notes
      • Getting started
    • Change Data Capture
      • Release notes
      • CDC for DB2/400
    • CMIS
      • Release notes
    • Couchbase
      • Release notes
      • Getting started
    • Databricks
      • Release notes
      • Getting Started
    • DBase
      • Release notes
      • Getting started
    • Elasticsearch
      • Release notes
      • Getting started
      • Search Guard security
    • Firebird
      • Getting started
      • Release notes
    • Google BigQuery
      • Release notes
      • Getting started
      • Working with partitioned tables
    • Google Cloud Platform
      • Release notes
      • Getting started
      • Cloud Storage
        • Getting started
    • Google Cloud Secret Manager
      • Release notes
    • Google Cloud Storage Delivery Repository
      • Release notes
    • Google Sheets
      • Release notes
      • Getting started
    • Greenplum
      • Release notes
      • Getting started
    • H2
      • Release notes
      • Getting started
    • Hadoop
      • Release notes
      • HBase
        • Getting started
        • Management tools
        • Importtsv tool
      • HDFS
        • Getting started
      • Hive
        • Getting started
      • Impala
        • Getting started
      • Sqoop
        • Getting started
    • HashiCorp Vault
      • Release notes
    • HTTP REST
      • Release notes
      • Getting started
      • Working with REST in xDI
        • HTTP methods
        • HTTP content types
        • Multipart data
        • Headers and responses
        • Retrieve entire responses
        • Error handling
    • HyperFile SQL
      • Release notes
      • Getting started
    • IBM DB2
      • Release notes
      • DB2 UDB
        • Getting started
      • DB2/400
        • Getting started
        • CCSID Reverse
    • Lotus Notes
      • Release notes
      • Getting started
    • Netezza
      • Release notes
      • Getting started
    • Informix
      • Release notes
      • Getting started
    • Ingres
      • Getting started
      • Release notes
    • InterSystems Caché
      • Release notes
      • Getting started
    • JMS
      • Release notes
      • Getting started
    • Kafka
      • Release notes
      • Getting started with Kafka Raw metadata
      • Getting started with Kafka Structured metadata
    • MemSQL
      • Release notes
      • Getting started
    • Microsoft Access
      • Release notes
      • Getting started
    • Microsoft Azure
      • Release notes
      • Blob Storage
        • Getting started
      • SQL Database
        • Getting started
    • Microsoft Azure Blob Storage Delivery Repository
      • Release notes
    • Microsoft Azure Cosmos DB
      • Release notes
      • Getting started
    • Microsoft Azure Key Vault
      • Release notes
    • Microsoft Azure Service Bus
      • Release notes
      • Getting started
    • Microsoft Azure Table Storage
      • Release notes
      • Getting started
    • Microsoft Excel
      • Release notes
      • Getting started
    • Microsoft SQL Server
      • Release notes
      • Getting started
      • BULK INSERT configuration
    • MonetDB
      • Release notes
      • Getting started
    • MongoDB
      • Release notes
      • Getting started
    • MySQL
      • Release notes
      • Getting started
    • Oracle
      • Release notes
      • Oracle Database
        • Getting started
        • Slowly Changing Dimensions
        • Oracle SQL*Loader
      • Oracle RDB
        • Getting started
      • Oracle BI
        • Getting started
    • Paradox
      • Release notes
      • Getting started
    • Parquet
      • Release notes
      • Getting started
    • PostgreSQL
      • Release notes
      • Getting started
    • Privacy Protect
      • Release notes
      • Getting started
    • Progress OpenEdge
      • Release notes
      • Getting started
    • Salesforce
      • Release notes
      • Getting started
      • Salesforce replicator in Incremental Mode
      • Attach files to records
      • Work with relations
    • Sampling
      • Release notes
      • Getting started
    • SAP
      • Release notes
      • Getting started
      • Set up Semarchy xDI for SAP
      • Set up SAP for xDI
        • Set up to work with tables
        • Set up to query DataSources
        • Set up to process IDocs
        • Set up to call BAPIs or RFCs
        • How to create function modules
      • Connect to SAP
      • Use SAP in xDI
      • Tips and tricks
    • SAP ASE
      • Release notes
      • Getting started
    • SAP Hana
      • Release notes
      • Getting started
    • SAP IQ
      • Release notes
      • Getting started
    • SAP S/4HANA
      • Release notes
      • Getting started
      • Set up Semarchy xDI for SAP S/4HANA
      • Set up SAP S/4HANA for xDI
        • Set up to work with tables
        • Set up to query DataSources
        • Set up to process IDocs
        • Set up to call BAPIs or RFCs
        • How to create function modules
      • Connect to SAP S/4HANA
      • Use SAP S/4HANA in xDI
      • Tips and tricks
    • SAP SQL Anywhere
      • Release notes
      • Getting started
    • Semarchy xDM
      • Release notes
      • Getting started
    • Snowflake
      • Release notes
      • Getting started
      • Snowpipe
    • Spark
      • Release notes
      • Getting started
    • SQreamDB
      • Release notes
      • Getting started
    • Stored Procedure
      • Release notes
      • Getting started
    • Teradata
      • Release notes
      • Getting started
    • Twitter
      • Release notes
      • Getting started
    • Vertica
      • Release notes
      • Getting started
      • Array and Row types
  • Deploy integration flows
    • Generate packages
    • Generate deliveries
    • Deploy deliveries
    • Run deliveries
      • Add modules to a runtime
    • Schedule deliveries
    • Publish web services
      • Getting started
      • Access API definitions
      • Configure advanced settings
      • Customize behavior at invocation
      • Temporary file retention
    • Monitor the runtime
    • Runtime clusters
      • Share delivery schedules
      • Load balancing
    • Designer CLI
      • CLI commands
      • CLI optimization
    • Runtime commands
      • General
      • Delivery management
      • Schedule management
      • Service management
      • Session management
    • Runtime REST API for deliveries
  • Manage production
    • Configure Analytics
      • Runtimes
      • Log databases
      • Profiles
      • Environments
      • Statistics
      • Preferences
    • Manage runtimes
    • Metadata configurations
    • Manage sessions
    • Delivery Projects
      • Create a Delivery Project
      • Manage Package Manager versions
      • Manage deployments
      • Advanced deployment
      • Delivery Project Supervisor
      • Source Management editor
    • Management REST API
  • Actions reference
    • File Actions
      • Move Files
      • Copy Files
      • Delete Files
      • Write a file
      • Make a directory
      • Zip files
      • BZip files
      • GZip files
      • Tar files
      • Unzip files
      • BUnzip files
      • GUnzip files
      • Untar file
      • Wait for Files
      • Concat Files
      • Xslt Transformation
    • Internet and Networking
      • Get Files with FTP
      • Send files with FTP
      • Run FTP Commands
      • Run SFTP Commands
      • Send Email
      • Read Email
      • Send files with SCP
      • Get Files with SCP
      • Run SSH Commands
      • JMS Send Message From File
      • JMS Receive Message to File
      • JMS Operation
      • AMQP Send Message
      • AMQP Receive Message
      • AMQP Operation
      • AMQP Send File
      • AMQP Receive Message to File
    • Misc
      • Sleep
      • Operating System Command
      • Start Delivery
      • Empty action
      • Raise Error
      • Variable Manager
    • Script
      • Java Native Scripting
      • Bean Scripting Framework
    • SQL
      • SQLFileExport
      • SQL Operation
      • SQL To Parameters
    • xDM Tools
      • Get LoadID
      • Submit Load
      • Get Load Status
      • Cancel Load
    • Other Tools
      • Amazon
        • Amazon S3 - operation
      • CDC
        • CDC DB2-AS400
        • CDC DB2/400 Read Journal Data
      • Google Cloud Storage
        • GCS Copy Blobs
        • GCS Create Bucket
        • GCS Create Folder
        • GCS Delete Blobs
        • GCS Delete Bucket
        • GCS Get Blobs
        • GCS Move Blobs
        • GCS Put Blobs
        • GCS Update Blobs
      • Apache Kafka
        • Kafka Reset Offsets
      • Microsoft Azure Blob Storage
        • Azure Storage Operation
      • Microsoft Azure Service Bus
        • Azure Service Bus Create Queue
        • Azure Service Bus Create Subscription
        • Azure Service Bus Create Topic
        • Azure Service Bus Delete Queue
        • Azure Service Bus Delete Subscription
        • Azure Service Bus Delete Topic
        • Azure Service Bus Receive Queue Message
        • Azure Service Bus Receive Subscription Message
        • Azure Service Bus Send Queue Message
        • Azure Service Bus Send Topic Message
      • Replicator
        • Replicator Rdbms to Snowflake
      • SAP
        • SAP Clean Temporary Table
      • Snowflake
        • Snowflake - Snowpipe Operation
        • Snowflake Snowpipe Streaming
        • Snowflake - Snowpipe Streaming Get Last Committed Offset Token
        • Snowflake - Snowpipe Streaming Reset Offset Token
        • Snowflake - Snowpipe Streaming Set Offset Token
        • Snowflake - Upload File to Storage
        • Snowflake - Warehouse Operation
  • Additional resources
    • Guides
      • Development
        • Pivot or unpivot a table
      • Deliveries
        • Runtime delivery pulling
      • Runtime
        • Communicate with a runtime
    • Manuals
      • xDI Designer
        • Customize the Project Explorer view
        • Name a workspace
        • Import a project
        • Mappings - the Mapper utility
        • Restore deleted objects
        • Use the Impact view
        • Show object dependencies
      • xDI Runtime
        • Check if Runtime is running
    • Troubleshooting
      • xDI Runtime
        • Slow startup and execution time on macOS
    • License information
Semarchy xDI 2025.1
  • Semarchy xDI
    • 2025.1
    • 2024.3
    • 2024.2
    • 2024.1
    • 2023.4
    • 2023.3
    • 2023.2
    • 2023.1
    • 5.3
  • Semarchy xDI
  • Components
  • Hadoop
2025.1 2024.3 2024.2 2024.1 2023.4 2023.3 2023.2 2023.1 5.3

Hadoop component

Overview

The Hadoop component lets you create Hadoop integration flows in Semarchy xDI.

Install the Hadoop component

If you did not install it yet, install the Hadoop component in Designer by following the component installation process.

Supported products

Product

Hadoop Distributed File System

Apache HBase

Apache Hive

Apache Impala

Apache Sqoop
Getting started Release notes

Copyright © Semarchy - all rights reserved.