Coming soon - Get a detailed view of why an account is flagged as spam!
view details
32
ParallelVec: A succinct, transparent SoA Vec implementation
Post Body

Been working quite a bit in developing the low level primitives for Rust gamedev, and got fed up with writing structure of arrays around Vec<T>s. The other options in the space soa_derive and soa-vec fell short of a satisfying implementation for me:

  • Multiple underlying Vec<T> (as done by soa_derive) duplicates the length and capacity values for every field the struct has and it exposes the finer details of the resultant implementation to the user. This is a small overhead, but it can add up when you need thousands of these in one program (i.e. animation curves in animation editors or full games).
  • User facing macros like soa_derive creates extra types which may cause extra friction when moving/copying/converting the types around.
  • soa_Vec uses multiple base types with different counts of generic type parameters just didn't sit well with me. Also makes it hard to make generic functions over all SoA buffers.
  • Existing ECS implementations like Legion and bevy_ecs are often overkill for some of the use cases where this might be useful (i.e. particle data in particle systems or static animation data). These solutions support alternative storages, long term identity tracking, runtime extendable fields (in the form of components). I just wanted a extendable buffer of parallel slices where the layout and size are known at compile time.

From the dissatisfaction with the available solutions for this curiously recurring pattern, I whipped up https://crates.io/crates/parallel_vec, a solution that I think at least partially addresses some of these issues.

From a user's perspective, ParallelVec<(T1, T2, T3, ...)> works almost like a Vec<(T1, T2, T3...)> but stores the like-typed values contiguously (i.e. ([A1, A2, A3, ...], [B1, B2, B3...]...), not [(A1, B1, ...), (A2, B2, ...), (A3, B3, ...), ...]). The rest of the API is almost 1:1 with Vec, where possible. The implementation is fairly close to the way soa_vec works, with a few improvements, and a few (hopefully temporary) drawbacks:

  • PRO: There's only one main type: ParallelVec, which allows for easier integration into generic functions and types.
  • PRO: The number of fields is dictated by a implementation of a unsafe auto trait over tuples, using associated types to enforce type safety instead of a macro. The trait is sealed to prevent users of the library from making implementations that are more unsound.
  • PRO: ParallelVec does not require nightly features for allocation, and instead uses the stabilized global allocator API instead.
  • PRO: The implementation is no_std friendly.
  • SAME: Both implementations only require one large allocation for all slices when initially allocating or resizing. Duplicated work in managing multiple lengths and capacities is removed.
  • CON: The implementation requires nightly for generic associated types support. Hoping a MVP gets stabilized soon here.
  • CON: All elements must be 'static, this is more a product of me not being able to untangle the mess of lifetimes in such an implementation than anything else.
  • CON: The generated docs are pretty opaque due to all of the associated types. Definitely needs more example driven documentation.

The source code for this can be found at https://github.com/HouraiTeahouse/parallel_vec. This is my first time directly dealing with allocators, and combining macros and unsafe. Hoping I didn't miss anything horribly unsafe here, haha. PRs welcome and highly encouraged.

Author
Account Strength
100%
Account Age
12 years
Verified Email
Yes
Verified Flair
No
Total Karma
29,703
Link Karma
15,788
Comment Karma
13,897
Profile updated: 1 day ago
Posts updated: 8 months ago

Subreddit

Post Details

We try to extract some basic information from the post title. This is not always successful or accurate, please use your best judgement and compare these values to the post title and body for confirmation.
Posted
2 years ago