Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

1
Is partial deployment possible on spark ?
Post Body

Spark Stand Alone Mode

What would happen if the resources required by a spark cluster deployment are not available ? By spark cluster deployment I mean starting the master and workers for a single standalone cluster with certain requirements. Will the spark cluster begin with the resources that are available and then get other resources later or will the application return an error ?

These are the few cases that I can think of. I am sure there are many more but I could not find any info on this topic.

1.Spark cluster needs 4 nodes and 3 are available.

2.Deployment requires 3 workers with 2 cores each but only 2 workers are available with 2 cores, the third one can only get one core at the moment.

3.Deployment requires 3 workers with 500 mb each but only two workers can get 500mb and the third one cannot get 500mb.

It would help me if you also provided a source of the answer.

Author
Account Strength
90%
Account Age
10 years
Verified Email
Yes
Verified Flair
No
Total Karma
1,670
Link Karma
1,326
Comment Karma
326
Profile updated: 3 days ago
Posts updated: 1 year ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
8 years ago