Getting started with the Databricks database

This page contains information to help you start working with Databricks SQL.

Create Databricks SQL metadata

Databricks SQL metadata objects hold the same information as other SQL server metadata, such as data schemas, columns, and query folders. You can use this metadata in mappings to map data integration between fields, or in processes by using SQL actions.

To create Databricks SQL metadata:

  1. Create or open a project in xDI Designer.

  2. Either open the project context menu, or the main File menu, and select New > Metadata. The New Metadata window opens.

  3. Select Databricks SQL from the rdbms section, then click Next.

  4. Type a name for the metadata model and click Next.

  5. Select or create a Databricks SQL module. If you create a module, choose one of these options to add the necessary drivers:

    • From the Third Party Pack drop-down menu, select Databricks JDBC Driver.

    • Download the Databricks JDBC Driver yourself, and add it to the module with the + button.

  6. Click Finish to create the metadata object.

The server wizard launches after you create the metadata.

Connect to your data

You can connect to your Databricks SQL data by using the wizard, or defining metadata properties manually. The wizard allows you to test the connection.

On June 24, 2024, Databricks announced that Databricks-managed passwords will reach end of life on July 10, 2024. Databricks recommend using OAuth authentication to authenticate to Databricks APIs.

To read about OAuth tokens for Databricks, see the official Databricks documentation.

Using the wizard

If the server wizard did not open automatically, launch it from the metadata window:

  1. Open the Databricks SQL metadata.

  2. Open the context menu on the top-level node, and select Actions > Launch Server Wizard.

In the server wizard, enter required information:

  1. Give the server metadata element a display Name in Designer

  2. Make sure the Module and Driver entries are correct. If you only have one module and JDBC driver, accept the defaults.

  3. Edit the JDBC URL with the hostname and properties needed to connect to your data in Databricks SQL.

  4. Configure authentication:

  5. Test the connection using the Connect button.

  6. If the connection is successful, you can finish the wizard.

  7. Optionally, add other connection properties from the Properties window.

  8. Save your metadata.

Using the properties window

  1. Open the Databricks SQL metadata, and click on the top-level node to open its Properties window.

  2. In the Standard finger tab, configure properties:

    1. Give the server metadata element a display Name.

    2. Make sure the Module entry is correct, and choose a Driver. If you only have one module and JDBC driver, accept the defaults.

    3. Set a JDBC URL with the hostname, port, and properties needed to connect to your data in Databricks SQL.

    4. Configure authentication:

    5. Set other properties as needed.

  3. Save your metadata.

Reverse engineer your data

You can reverse-engineer the database structure and store it in the Databricks SQL metadata object. Use this metadata when designing mappings and processes to adapt your business rules according to your requirements.

Refer to Connect to your data for more information and instructions.

Work with mappings

In mappings, Databricks SQL metadata elements work the same as with other SQL metadata.

Refer to Work with mappings to learn about mappings.