Skip to main content

Connect Materialize to dbt Core

Vendor-supported plugin

Certain core functionality may vary. If you would like to report a bug, request a feature, or contribute, you can check out the linked repository and open an issue.

  • Maintained by: Materialize Inc.
  • Authors: Materialize team
  • GitHub repo: MaterializeInc/materialize
  • PyPI package: dbt-materialize
  • Slack channel: #db-materialize
  • Supported dbt Core version: v0.18.1 and newer
  • dbt support: Not Supported
  • Minimum data platform version: v0.28.0

Installing dbt-materialize

Use pip to install the adapter. Use the following command for installation:

python -m pip install dbt-materialize

Configuring dbt-materialize

For Materialize-specific configuration, please refer to Materialize configs.

Connecting to Materialize​

Once you have set up a Materialize account, adapt your profiles.yml to connect to your instance using the following reference profile configuration:

~/.dbt/profiles.yml
materialize:
  target: dev
  outputs:
    dev:
      type: materialize
      host: [host]
      port: [port]
      user: [user@domain.com]
      pass: [password]
      dbname: [database]
      cluster: [cluster] # default 'default'
      schema: [dbt schema]
      sslmode: require
      keepalives_idle: 0 # default: 0, indicating the system default
      connect_timeout: 10 # default: 10 seconds
      retries: 1 # default: 1, retry on error/timeout when opening connections

Configurations​

cluster: The default cluster is used to maintain materialized views or indexes. A default cluster is pre-installed in every environment, but we recommend creating dedicated clusters to isolate the workloads in your dbt project (for example, staging and data_mart).

keepalives_idle: The number of seconds before sending a ping to keep the Materialize connection active. If you are encountering SSL SYSCALL error: EOF detected, you may want to lower the keepalives_idle value to prevent the database from closing its connection.

To test the connection to Materialize, run:

dbt debug

If the output reads "All checks passed!", you’re good to go! Check the dbt and Materialize guide to learn more and get started.

Supported Features​

Materializations​

Because Materialize is optimized for transformations on streaming data and the core of dbt is built around batch, the dbt-materialize adapter implements a few custom materialization types:

TypeSupported?Details
sourceYESCreates a source.
viewYESCreates a view.
materializedviewYESCreates a materialized view.
tableYESCreates a materialized view. (Actual table support pending #5266)
sinkYESCreates a sink.
ephemeralYESExecutes queries using CTEs.
incrementalNOUse the materializedview materialization instead. Materialized views will always return up-to-date results without manual or configured refreshes. For more information, check out Materialize documentation.
Loading table...

Indexes​

Materialized views (materializedview), views (view) and sources (source) may have a list of indexes defined.

Seeds​

Running dbt seed will create a static materialized view from a CSV file. You will not be able to add to or update this view after it has been created.

Tests​

Running dbt test with the optional --store-failures flag or store_failures config will create a materialized view for each configured test that can keep track of failures over time.

Resources​

Was this page helpful?

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

0
Loading