Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak when Attribute.Valid is set to 0 #196

Open
ncw opened this issue Mar 20, 2018 · 2 comments
Open

Memory leak when Attribute.Valid is set to 0 #196

ncw opened this issue Mar 20, 2018 · 2 comments

Comments

@ncw
Copy link

ncw commented Mar 20, 2018

In rclone recently I decided to set the attribute caching time to 0s.

This stopped a number of file system oddities I was seeing.

However it seems to have provoked a possible bug in this library. Here is the background: rclone/rclone#2157

The reason why I think it is the library is that if I set the attribute caching to non-zero the memory leak disappears, and the pprof is quite suggestive:

profile001

(pprof) list allocMessage
Total: 1.83GB
ROUTINE ======================== github.com/ncw/rclone/vendor/bazil.org/fuse.allocMessage in /home/ncw/go/src/github.com/ncw/rclone/vendor/bazil.org/fuse/fuse.go
    1.76GB     1.76GB (flat, cum) 96.03% of Total
         .          .    419:var reqPool = sync.Pool{
         .          .    420:	New: allocMessage,
         .          .    421:}
         .          .    422:
         .          .    423:func allocMessage() interface{} {
    1.76GB     1.76GB    424:	m := &message{buf: make([]byte, bufSize)}
         .          .    425:	m.hdr = (*inHeader)(unsafe.Pointer(&m.buf[0]))
         .          .    426:	return m
         .          .    427:}
         .          .    428:
         .          .    429:func getMessage(c *Conn) *message {

Any thoughts?

It setting attribute caching to zero a really bad idea?

@tv42
Copy link
Member

tv42 commented Mar 21, 2018

That sounds like you caused a lot more FUSE requests to happen, by disabling the cache.

I'm not sure why they'd stay inuse, apart from sync.Pool doing something non-optimal (again).

I have a near-rewrite of the core bits of the FUSE library that changes how it manages memory, and it shouldn't have these kinds of issues. Just need to find enough time to get it to feature parity..

@ncw
Copy link
Author

ncw commented Mar 22, 2018

OK! I'll set the attribute timeout back to 1s in the mean time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants