Things got a little backed up - we're processing the data and things should be back to normal within the hour.

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

86
How do you "know" your architecture is correct?
Post Body

I am always wondering how can a data engineer know, and then convince product team or business that their architecture or design is correct? There is not exactly an exact science to data engineering. There are companies which are happy using out of the box tools and make it work, and there are engineers putting up with complex legacy codebases using microservices et al. Besides reading up and doing courses trying to get as close as possible to best practices, your architecture suggestions are subject to either lessons learnt from experience (takes 1-2 years for a big data project to take real shape) or googling and doing the best possible back of the envelope tradeoffs. While other non technical team members expect you to give them a silver bullet or quickly lose trust in you. How do you establish this confidence in a solution without seeming like a tinkerer?

Author
User Disabled
Account Strength
0%
Disabled 5 months ago
Account Age
2 years
Verified Email
Yes
Verified Flair
No
Total Karma
14,276
Link Karma
1,846
Comment Karma
12,214
Profile updated: 1 day ago
Posts updated: 5 months ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
11 months ago