Coming soon - Get a detailed view of why an account is flagged as spam!
view details

r/dataengineering

219,197 subscribers
News & discussion on Data Engineering topics, including but not limited to: data pipelines, databases, data formats, storage, data modeling, data governance, cleansing, NoSQL, distributed systems, streaming, batch, Big Data, and workflow engines.

Notice: This subreddit is not actively monitored, so the posts here are only a small subset of what exists on reddit.

Browse by flair

Help Career Discussion Meme Blog Open Source
New Top (24hr) Top (Week) Top (Month)
to r/dataengineering by saaggy_peneer
1 month ago
DMS -> S3 (CDC & Full) to SCD2 with DBT?
Discussion
to r/dataengineering by saaggy_peneer
1 month ago
DBT incremental merge with DMS CDC?
Discussion
to r/dataengineering by saaggy_peneer
3 months ago
Preview thumbnail for Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2 image
Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2
Open Source
to r/dataengineering by FigTraditional1201
3 weeks ago
Google sheets —-> ???? ——-> PowerBI
Help
to r/dataengineering by FigTraditional1201
3 weeks ago
Database solution?
Help
to r/dataengineering by cantseemelol
8 months ago
Cost of ADF using Stored Procedures vs pure ADF
Help
to r/dataengineering by izotAcario
5 months ago
Newbie question: What solutions should I use to suggest my users posts based on their preferences?
Help
to r/dataengineering by IllustriousCorgi9877
1 month ago
Kind of DE job to apply to
Career
in Delaware
to r/dataengineering by FigTraditional1201
3 weeks ago
MS SQL / AZURE SQL data base?
Discussion
to r/dataengineering by LoneSysAdm
7 months ago
ADF for Consuming REST API's
Help
to r/dataengineering by CutSubstantial7320
8 months ago
How do you "know" your architecture is correct?
Discussion
to r/dataengineering by chock-a-block
11 months ago
Apache Airflow PostgreSQL not Committing
Help
to r/dataengineering by BitterCustard26
11 months ago
Load SDMX format into a db
Help
to r/dataengineering by FigTraditional1201
3 weeks ago
Break into Data Engineering?
Career
to r/dataengineering by Drekalo
8 months ago
What other communities do you follow for DE discussion?
Discussion
in Delaware
to r/dataengineering by dehydratedcatnip
1 year ago
How do I get better in SQL
Help
to r/dataengineering by Eisenstein
7 months ago
A question about filesystems and quantity of files in relation to speed
Discussion
to r/dataengineering by kryon-X
1 year ago
Which best practices do you follow to build robust & extensible ETL jobs?
Discussion
to r/dataengineering by rahuldev07
1 year ago
Track to Choose
Career
to r/dataengineering by ZarekBro
1 year ago
AWS beginner. How would you approach this personal project?
Help
to r/dataengineering by didierCamer
11 months ago
Ingest Mainframe data to cloud
Help
to r/dataengineering by dehydratedcatnip
1 year ago
Career Shift To Data Engineering
Career
to r/dataengineering by relentless_bull_
1 year ago
ELT vs ETL
Discussion
to r/dataengineering by relentless_bull_
1 year ago
What does Coding in “Production” mean - Databricks
Discussion
to r/dataengineering by edsmart123
8 months ago
Does what kind of degree matters? Do I need MS CS?
Career
in Mississippi