Coming soon - Get a detailed view of why an account is flagged as spam!
view details

This post has been de-listed

It is no longer included in search results and normal feeds (front page, hot posts, subreddit posts, etc). It remains visible only via the author's post history.

0
Large memory usage when creating ORM models in loop
Post Body

I've noticed when creating a large amount (around 25000) of new ORM objects in a loop, (in this case Treebeard MP_nodes) Django begins eating up more and more of the machines ram in gigabytes as it iterates through an in-memory dictionary and creates these models, and i can't wrap my head around why it does so.

I've run into this before when creating objects in a migration, so i don't believe its specific to the Treebeard library. I was wondering if any of you guys have run into the same issue when creating a large amount of models without bulk create.

Obviously when doing this you should use bulk_create, But this is a limitation of the library which doesn't have a method to bulk create the nodes of the tree at once. I can get around this by using some private methods, but I'd rather not unless I have to.

Author
Account Strength
90%
Account Age
7 years
Verified Email
Yes
Verified Flair
No
Total Karma
1,613
Link Karma
993
Comment Karma
620
Profile updated: 2 days ago
Posts updated: 2 weeks ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
5 years ago