Managing dbt project properties and manifest upload – SqlDBM

This feature may not be available by default and might need to be enabled upon request. Please contact your account manager for more information.

Overview

The dbt Manifest Import feature allows you to upload your dbt project's manifest.json file directly into SqlDBM, automatically aligning your SqlDBM project with your dbt project structure. By uploading this file to SqlDBM, you can automatically classify objects as dbt sources or models, populate project metadata, sync descriptions, and visualize dbt dependencies—all without manual data entry.

The same properties that are managed by the automated manifest upload can also be maitained by hand if designing bottom up and creating new models and sources in SqlDBM before exporting YAML to dbt.

Uploading a dbt Manifest

When you run dbt compile or dbt run, dbt generates a manifest.json file containing metadata about all resources in your dbt project—including models, sources, their descriptions, and dependencies. This file is located in the dbt target directory.

Note that dbt's target directory is not tracked in Git because it is regenerated with every run and compilation.

Open this file in dbt and save it locally.

Alternatively, the manifest file can be obtained programmatically using dbt's discovery API.

Import the Manifest to SqlDBM

The manifest file can be uploaded from the Database Documentation screen.

Navigate to Database Documentation from the left-hand menu, then use the "Upload dbt" option to select your manifest.json file.

File Requirements

The uploaded file must meet the following criteria:

Format: JSON only (.json extension). Other file formats are not supported.
Size limit: Maximum file size is 100 MB.
Structure: The file must conform to the standard dbt manifest JSON structure. If the file structure is malformed or cannot be parsed, the upload will fail with an error.

How Object Matching Works and What is Imported

When you upload a manifest file, SqlDBM matches objects from the manifest to existing objects in your project using their full name: schema_name.object_name. This comparison is case-insensitive.

In the manifest file, sources and models follow a naming structure: <project_name>.<package_name>.<object_name>. SqlDBM extracts the fully-qualified object name (i.e., database, schema, object) from each source/model and matches them to existing project objects. The process applies to all supported logical table objects (tables, views, materialized views, etc.).

When a match is found, SqlDBM will update the matched object with the following properties from the manifest:

Object type — classified as Source, Model, None, or empty, with accompanying icon. "None" exists to explicitly indicate that an object is maintained outside of dbt, as opposed to an empty value, which is unclassified.
Object/column description — replaced by the source/model/column description from the manifest (only if the SqlDBM description is empty and the manifest is not, [i.e., existing descriptions in SqlDBM will not be overwritten.])
Project name — the dbt project name the object belongs to
Package — the dbt package associated with the object. Unless the dbt project imports other projects or packages, its package name is the same as the project name.
Path — the original file path of the object in the dbt project

These properties are available as fields on the Database Documentation screen for review or manual editing.

Note, this feature is not to be confused with Reverse Engineering from a dbt project. Through this process, no objects are created, removed, or structurally modified in the SqlDBM project—only (dbt-related) metadata is changed.

Unmatched and Removed Objects

When a new manifest is uploaded, SqlDBM compares the project_name.package identifier to determine whether the upload belongs to the same dbt project as a previous upload. If there is a match, any objects previously marked as sources or models from that same project but no longer present in the new manifest will have their dbt project properties cleared (i.e., object type, project name, package, and path).

However, dbt object properties like tests, version, and meta values will not be affected.

Objects belonging to a different dbt project are never affected by the upload. For example, if your SqlDBM project contains objects from both dbt_project_1 and dbt_project_2, uploading a new manifest from dbt_project_1 will only clear the unmatched objects from dbt_project_1—objects from dbt_project_2 remain unchanged.

Creating dbt Lineage Dependencies (Virtual Relationships)

Based on the information contained in the Manifest file, SqlDBM will automatically create dependency virtual relationships as defined by ref() calls. These represent the data lineage and transformation flow defined in your dbt project.

For example, if model_b references source_a in your dbt project, a dependency relationship will be created from source_a to model_b in SqlDBM, visually representing the data flow on the diagram.

Key behaviors for manifest-created dependencies:

Dependencies are automatically created based on the parent_map entries in the manifest.
Dependencies created through the manifest can be manually edited (renamed, recolored, etc.) or deleted just like any other virtual relationship.
Manual edits to a dependency's name, color, description, or column pairs are preserved across subsequent manifest uploads.
If a dependency is removed from the manifest in a subsequent upload, it will also be removed from the project.
If a parent or child object in the manifest is not found in the SqlDBM project, the dependency is not created.

Manually Assigning dbt Project Properties

In addition to uploading a manifest file, dbt project properties can be manually assigned and edited on the Database Documentation screen (or its equivalent metadata bottom panel on the diagrams). This is useful when you want to classify objects, adjust values for newly created objects, or change a value set by a previous upload.

dbt Project Properties Section

A dedicated section called dbt project properties is available in the Database Documentation for each object. This section contains the following system fields:

Object type — a dropdown with three values: None (default), Source, or Model. Determines how the object is classified in the dbt context.
Project name — a text field for the dbt project name the object belongs to.
Package — a text field for the dbt package associated with the object.
Path — a text field for the original file path of the object in the dbt project.

(See the previous screenshot, above).

These fields are distinct from the existing dbt object properties section (previously called "dbt properties"), which contains standard dbt properties like tests, freshness, version, and meta. The dbt project properties section appears below the dbt object properties section.

Note that dbt project properties are system fields and cannot be edited, removed, or reconfigured through the field management interface.

Changing the dbt Object Type

Once an object has been identified as a source or model—either through a manifest upload or manual assignment—the object type can be changed at any time using the dropdown in the dbt project properties section.

The selection will also be reflected in the associated icons in the explorer menu and in the diagrams.

Excel Editing

The properties described in this article are editable through Excel upload like the rest of the dbt fields. This is possible in both the documentation and structure variants of the Excel upload.