Coming soon - Get a detailed view of why an account is flagged as spam!
view details

r/dataengineering

219,197 subscribers

Notice: This subreddit is not actively monitored, so the posts here are only a small subset of what exists on reddit.

Browse by flair

Help Career Discussion Meme Blog Open Source
Showing posts with Discussion
remove filter
Sort by: New Top
to r/dataengineering by saaggy_peneer
3 weeks ago
DMS -> S3 (CDC & Full) to SCD2 with DBT?
Discussion
to r/dataengineering by saaggy_peneer
3 weeks ago
DBT incremental merge with DMS CDC?
Discussion
to r/dataengineering by NFeruch
7 months ago
Using pandas in a data-intensive application - what the best, RAM-efficient, alternative?
Discussion
to r/dataengineering by NFeruch
7 months ago
Database design for my software app - how to handle changing fields from API response?
Discussion
to r/dataengineering by NFeruch
7 months ago
Is there such thing as a “product data engineer” who isn’t just a slave to the reporting team?
Discussion
to r/dataengineering by NFeruch
7 months ago
Why can’t I pass context between tasks in Airflow/Dagster?
Discussion
to r/dataengineering by NFeruch
7 months ago
Is doing a "dry run" of a data pipeline - so that changes to the database are either rolled back or not committed, for testing purposes - normal practice in DE?
Discussion
to r/dataengineering by NFeruch
7 months ago
What is the best SQL environment you have ever worked in?
Discussion
to r/dataengineering by FigTraditional1201
1 week ago
MS SQL / AZURE SQL data base?
Discussion
to r/dataengineering by NFeruch
7 months ago
ELI5, what are the main differences between software engineers and data engineers?
Discussion
to r/dataengineering by CutSubstantial7320
7 months ago
How do you "know" your architecture is correct?
Discussion
to r/dataengineering by NFeruch
7 months ago
What is the difference between a data engineer, a data platform engineer, and a software engineer - data platform?
Discussion
to r/dataengineering by NFeruch
7 months ago
I want to be able to programmatically integrate dbt into an end-to-end pipeline in python. Is this the wrong usage of dbt?
Discussion
to r/dataengineering by NFeruch
7 months ago
dbt seems like such a cool concept
Discussion
to r/dataengineering by NFeruch
7 months ago
What do you use for version control of your database schema?
Discussion
to r/dataengineering by Drekalo
7 months ago
What other communities do you follow for DE discussion?
Discussion
in Delaware
to r/dataengineering by Eisenstein
6 months ago
A question about filesystems and quantity of files in relation to speed
Discussion
to r/dataengineering by kryon-X
1 year ago
Which best practices do you follow to build robust & extensible ETL jobs?
Discussion
to r/dataengineering by NFeruch
7 months ago
What's the best way to learn dbt
Discussion
to r/dataengineering by NFeruch
7 months ago
Does this tool already exist?
Discussion
to r/dataengineering by relentless_bull_
1 year ago
ELT vs ETL
Discussion
to r/dataengineering by relentless_bull_
1 year ago
What does Coding in “Production” mean - Databricks
Discussion
to r/dataengineering by NFeruch
7 months ago
ROUGHLY at what point would I switch from pandas to spark?
Discussion
to r/dataengineering by NFeruch
7 months ago
Are ETL tools necessary for API querying at scale?
Discussion
to r/dataengineering by Deer-Antlers
8 months ago
How are you guys validating your data?
Discussion