Updated specific locations to be searchable, take a look at Las Vegas as an example.
14
I want to be able to programmatically integrate dbt into an end-to-end pipeline in python. Is this the wrong usage of dbt?
Post Flair (click to view more posts with a particular flair)
Post Body

Essentially what the title says. I have an end to end pipeline that at a high level is * extract data * do some python transformations * export data

I want to add a 4th step of

  • run the relevant dbt models

The thing is that I’m running into problems with invoking it programmatically, as the pipeline file is in a different directory than the dbt project (so it doesn’t detect the project yaml file) and I can’t use absolute paths to specify which model to run, which I need to do because the pipeline is in a different directory

This is for a personal project, so I am trying to implement dbt just for the sake of learning it. The thing is though, that the pipeline runs automatically on a cron job, and I need the model to be run at the time of the pipeline

Is this the wrong usage pattern for dbt? I suspect it is because it’s not straightforward to do so far

Author
Account Strength
100%
Account Age
7 years
Verified Email
Yes
Verified Flair
No
Total Karma
42,675
Link Karma
23,655
Comment Karma
18,693
Profile updated: 2 days ago
Posts updated: 6 months ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
1 year ago