It's pronounced Diane. I do data architecture, operations, and backend development. In my spare time I maintain Massive.js, a data mapper for Node.js and PostgreSQL.
All your ideas are kludges to try to mimic a relational structure.
as you identified, this becomes more difficult and less performant as the rep arrays grow. The real problem isn't retrieving a user with a long rep array though -- it's filtering that array, which you need to do a few different ways.
materializing the total count per user only helps with one specific use case, not for anything else.
this just moves the same problem to another collection.
reinventing the junction table is probably the least-bad approach. A look at neDB doesn't turn up anything about foreign key constraints, so referential integrity is on you. Indexing will be important because this collection will be by far the biggest.
might be required if indexes with #4 aren't sufficient.
Bottom line, you have relational data that can scale arbitrarily and it's always going to be an awkward fit at best in a non-relational datastore. The single best thing you could do is switch to a database that's designed to handle the information structures you're working with.
Thanks for the input. Filtering and managing an array is pretty simple though and is nothing beyond map reduce functions. I'll give a few options a try and see which works nice
It's pronounced Diane. I do data architecture, operations, and backend development. In my spare time I maintain Massive.js, a data mapper for Node.js and PostgreSQL.
It is simple! The problem is it takes time. Filtering one rep array is O(n); filtering and aggregating (for example, if you wanted to see all rep you've given) is O(n2 ). Performance may be acceptable at first but you've chosen a structure which can grow indefinitely and does not admit any shortcuts.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
All your ideas are kludges to try to mimic a relational structure.
Bottom line, you have relational data that can scale arbitrarily and it's always going to be an awkward fit at best in a non-relational datastore. The single best thing you could do is switch to a database that's designed to handle the information structures you're working with.
Thanks for the input. Filtering and managing an array is pretty simple though and is nothing beyond map reduce functions. I'll give a few options a try and see which works nice
It is simple! The problem is it takes time. Filtering one rep array is O(n); filtering and aggregating (for example, if you wanted to see all rep you've given) is O(n2 ). Performance may be acceptable at first but you've chosen a structure which can grow indefinitely and does not admit any shortcuts.