Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexes should probably not assume MODE_ATCOMMIT if queue processor is not active #61

Open
mcdonc opened this issue Jun 10, 2013 · 2 comments
Assignees

Comments

@mcdonc
Copy link
Member

mcdonc commented Jun 10, 2013

Right now, if the queue processor is inactive, indexes will presume that they should not add to the queue (e.g. MODE_DEFERRED), and instead, they should treat indexing operations as if they were MODE_ATCOMMIT. This foils bulk load scenarios where you don't actually want to index the data while the loading is happening.

However, right now, the queue is one bigass nonpersistent object. This means that if it's not popped every so often, transactions which append to it need to write a multimegabyte list on every commit.

I'm not sure how to solve this yet.

@mcdonc
Copy link
Member Author

mcdonc commented Jun 12, 2013

Portential solution: have each transaction create its own list of persistent actions and manage the lists in the _p_resolveConflict of a persistent object that keeps a set of them.

@ghost ghost assigned mcdonc Jun 12, 2013
@mcdonc
Copy link
Member Author

mcdonc commented Jul 26, 2013

Note that the bulk loading scenario is now handled by the substanced.catalogs.force_deferred flag (adecc33). But the issue still remains of reserializing the bigass nonpersistent queue on every commit that adds to it.

@cguardia cguardia modified the milestones: After First Release, Future enhancements May 12, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants