You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I also posted this issue here as I'm not sure which of the two is maintained.
Simple example and look at top/htop
Take this simple example where we create a tree in each iteration:
from ailist import AIList
import time
for i in range(7000000):
a = AIList()
a.add(i,i+1)
time.sleep(0.0000001)
Then when look at top/htop we clearly see that memory is not freed and gradually builds up after each iteration.
We can see memory usage with this simple example
We can clearly see this when we look at the memory usage increase after each iteration (very small example):
from ailist import AIList
import time
import numpy as np
from tqdm import tqdm
import matplotlib.pyplot as plt
def getCurrentMemoryUsage():
# from: https://stackoverflow.com/questions/938733/total-memory-used-by-python-process
with open('/proc/self/status') as f:
memusage = f.read().split('VmRSS:')[1].split('\n')[0][:-3]
return int(memusage.strip()) # in kb
r = 50000
memory_usage = np.empty(r, dtype = 'int32')
for i in tqdm(range(r)):
a = AIList()
a.add(i,i+1)
time.sleep(0.0000001)
memory_usage[i] = getCurrentMemoryUsage()
plt.plot(range(r), memory_usage)
Which shows us:
Not a garbage collection thing
When we call garbage collection within the loop (which we of course shouldn't have to) it also does not resolve the issue:
This works fine with other interval libraries, such as IntervalTree
from intervaltree import Interval, IntervalTree
r = 5000
memory_usage = np.empty(r, dtype = 'int32')
for i in tqdm(range(r)):
t = IntervalTree()
t.add(Interval(i,i+1))
time.sleep(0.0000001)
memory_usage[i] = getCurrentMemoryUsage()
plt.plot(range(r), memory_usage)
Compared to intervaltree AIlist is much faster so I would love to use this instead, however, currently we can't with the memory issue :(
The text was updated successfully, but these errors were encountered:
I also posted this issue here as I'm not sure which of the two is maintained.
Simple example and look at
top/htop
Take this simple example where we create a tree in each iteration:
Then when look at
top/htop
we clearly see that memory is not freed and gradually builds up after each iteration.We can see memory usage with this simple example
We can clearly see this when we look at the memory usage increase after each iteration (very small example):
Which shows us:

Not a garbage collection thing

When we call garbage collection within the loop (which we of course shouldn't have to) it also does not resolve the issue:
This works fine with other interval libraries, such as IntervalTree
Compared to intervaltree AIlist is much faster so I would love to use this instead, however, currently we can't with the memory issue :(
The text was updated successfully, but these errors were encountered: