Memory overhead for subscription mtrie nodes is large
Description
Environment
None
Activity
Show:
Martin Hurton June 27, 2012 at 6:58 AM
There was a series of commits from Staffan Gimåker (6fa9ffebe5d4cf0bba051f464b8c8054cffc2b72, e6c97c5ecc3f2b9e84258cb405eb92f6b6f6ca7d, and 2cd04c54dfabd48f528636d15ccc4e1959b14432) meant to improve the memory overhead. Does this help your case?
Staffan Gimåker December 28, 2011 at 6:04 PM
To avoid duplicated work I just wanted to chime in and say that I'm working on this and making good progress.
In the mtrie, the set of pipe pointers per node combined with the large number of nodes (per-char) in the trie leads to a rather impressive memory usage (1kB+ per node in our test case).
This starts to hurt even for a relatively small number of topics.
A compressed trie implementation or some clever use of a bitset to represent the list of pipes might be the way to go (exploring this).