Skip to content
This repository was archived by the owner on Jul 15, 2023. It is now read-only.
This repository was archived by the owner on Jul 15, 2023. It is now read-only.

hdfs-mount can't read file again after deletion followed by puting a new one #54

@liaoyw

Description

@liaoyw

version:

hdfs-mount: lastest
kernel: 4.18.8-1.el7.elrepo.x86_64
fuse-libs-2.9.2-11

how to reproduce it

  1. mount hdfs
    /hdfs-mount -root / --logLevel 2 -fuse.debug localhost:8020 /mnt &
  2. use hdfs command create file localhost:8020/tmp/x
  3. cat /mnt/tmp/x
2019/06/04 14:04:20 FUSE: <- Lookup [ID=0x4 Node=0x2 Uid=0 Gid=0 Pid=512] "x"
2019/06/04 14:04:20 FUSE: -> [ID=0x4] Lookup 0x3 gen=0 valid=1m0s attr={valid=1m0s ino=16468 size=17 mode=-rw-r--r--}
2019/06/04 14:04:20 FUSE: <- Getxattr [ID=0x5 Node=0x3 Uid=0 Gid=0 Pid=512] "security.capability" 24 @0
2019/06/04 14:04:20 FUSE: -> [ID=0x5] Getxattr error=operation not supported
2019/06/04 14:04:20 FUSE: <- Getxattr [ID=0x6 Node=0x3 Uid=0 Gid=0 Pid=512] "security.capability" 24 @0
2019/06/04 14:04:20 FUSE: -> [ID=0x6] Getxattr error=operation not supported
2019/06/04 14:04:20 FUSE: <- Open [ID=0x7 Node=0x3 Uid=0 Gid=0 Pid=512] dir=false fl=OpenReadOnly
INFO: 2019/06/04 14:04:20 Open:  /tmp/x OpenReadOnly
2019/06/04 14:04:20 FUSE: -> [ID=0x7] Open 0x1 fl=0
2019/06/04 14:04:20 FUSE: <- Read [ID=0x8 Node=0x3 Uid=0 Gid=0 Pid=512] 0x1 4096 @0x0 dir=false fl=0 lock=0 ffl=OpenReadOnly
Warning: 2019/06/04 14:04:20 FileHandleReader.go:112: [ /tmp/x ] EOF @ 17
2019/06/04 14:04:20 FUSE: -> [ID=0x8] Read 17
xyz0000000000000
2019/06/04 14:04:20 FUSE: <- Getattr [ID=0x9 Node=0x3 Uid=0 Gid=0 Pid=512] 0x1 fl=GetattrFh
2019/06/04 14:04:20 FUSE: -> [ID=0x9] Getattr valid=1m0s ino=16468 size=17 mode=-rw-r--r--
2019/06/04 14:04:20 FUSE: <- Flush [ID=0xa Node=0x3 Uid=0 Gid=0 Pid=512] 0x1 fl=0x0 lk=0xc153994ee1b5de49
2019/06/04 14:04:20 FUSE: -> [ID=0xa] Flush
2019/06/04 14:04:20 FUSE: <- Release [ID=0xb Node=0x3 Uid=0 Gid=0 Pid=0] 0x1 fl=OpenReadOnly rfl=0 owner=0x0
INFO: 2019/06/04 14:04:20 [ /tmp/x ] ReadStats: holes: 0 , cache hits: 0 , hard seeks: 0
INFO: 2019/06/04 14:04:20 [ /tmp/x ] Close/Read: err= <nil>
2019/06/04 14:04:20 FUSE: -> [ID=0xb] Release
  1. delete file use hdfs command
  2. cat /mnt/tmp/x
cat /mnt/tmp/x
2019/06/04 14:05:10 FUSE: <- Getxattr [ID=0xf Node=0x3 Uid=0 Gid=0 Pid=514] "security.capability" 24 @0
2019/06/04 14:05:10 FUSE: -> [ID=0xf] Getxattr error=operation not supported
2019/06/04 14:05:10 FUSE: <- Open [ID=0x10 Node=0x3 Uid=0 Gid=0 Pid=514] dir=false fl=OpenReadOnly
INFO: 2019/06/04 14:05:10 Open:  /tmp/x OpenReadOnly
Error: 2019/06/04 14:05:10 FileHandleReader.go:33: [ /tmp/x ] Opening:  open /tmp/x: file does not exist
2019/06/04 14:05:10 FUSE: -> [ID=0x10] Open error=EIO: open /tmp/x: file does not exist
cat: /mnt/tmp/x: Input/output error
  1. wait some time(about 1min), cat file again
 cat /mnt/tmp/x
2019/06/04 14:06:12 FUSE: <- Lookup [ID=0x17 Node=0x1 Uid=0 Gid=0 Pid=518] "tmp"
2019/06/04 14:06:12 FUSE: -> [ID=0x17] Lookup 0x2 gen=0 valid=1m0s attr={valid=1m0s ino=16395 size=0 mode=drwxrwxrwx}
2019/06/04 14:06:12 FUSE: <- Lookup [ID=0x18 Node=0x2 Uid=0 Gid=0 Pid=518] "x"
Warning: 2019/06/04 14:06:12 Dir.go:186: stat [x]: stat /tmp/x: file does not existstat /tmp/x: file does not exist
2019/06/04 14:06:12 FUSE: -> [ID=0x18] Lookup error=ENOENT
2019/06/04 14:06:12 FUSE: <- Lookup [ID=0x19 Node=0x2 Uid=0 Gid=0 Pid=518] "x"
2019/06/04 14:06:12 FUSE: <- Forget [ID=0x1a Node=0x3 Uid=0 Gid=0 Pid=0] 1
2019/06/04 14:06:12 FUSE: -> [ID=0x1a] Forget
2019/06/04 14:06:12 FUSE: -> [ID=0x19] Lookup 0x3 gen=1 valid=1m0s attr={valid=1m0s ino=16395 size=0 mode=drwxrwxrwx}
2019/06/04 14:06:12 FUSE: <- Getxattr [ID=0x1b Node=0x3 Uid=0 Gid=0 Pid=518] "security.capability" 24 @0
2019/06/04 14:06:12 FUSE: -> [ID=0x1b] Getxattr error=operation not supported
2019/06/04 14:06:12 FUSE: <- Getxattr [ID=0x1c Node=0x3 Uid=0 Gid=0 Pid=518] "security.capability" 24 @0
2019/06/04 14:06:12 FUSE: -> [ID=0x1c] Getxattr error=operation not supported
2019/06/04 14:06:12 FUSE: <- Open [ID=0x1d Node=0x3 Uid=0 Gid=0 Pid=518] dir=true fl=OpenReadOnly
INFO: 2019/06/04 14:06:12 Open:  /tmp/tmp OpenReadOnly
===> Error: 2019/06/04 14:06:12 FileHandleReader.go:33: [ /tmp/tmp ] Opening:  open /tmp/tmp: file does not exist
===> 2019/06/04 14:06:12 FUSE: -> [ID=0x1d] Open error=EIO: open /tmp/tmp: file does not exist
cat: /mnt/tmp/x: Input/output error
  1. put new file using hdfs command
  2. cat file still using open /tmp/tmp
 cat /mnt/tmp/x
2019/06/04 14:07:16 FUSE: <- Lookup [ID=0x1e Node=0x1 Uid=0 Gid=0 Pid=519] "tmp"
2019/06/04 14:07:16 FUSE: -> [ID=0x1e] Lookup 0x2 gen=0 valid=1m0s attr={valid=1m0s ino=16395 size=0 mode=drwxrwxrwx}
2019/06/04 14:07:16 FUSE: <- Lookup [ID=0x1f Node=0x2 Uid=0 Gid=0 Pid=519] "x"
Warning: 2019/06/04 14:07:16 Dir.go:186: stat [tmp]: stat /tmp/tmp: file does not existstat /tmp/tmp: file does not exist
2019/06/04 14:07:16 FUSE: -> [ID=0x1f] Lookup error=ENOENT
2019/06/04 14:07:16 FUSE: <- Lookup [ID=0x20 Node=0x2 Uid=0 Gid=0 Pid=519] "x"
2019/06/04 14:07:16 FUSE: <- Forget [ID=0x21 Node=0x3 Uid=0 Gid=0 Pid=0] 1
2019/06/04 14:07:16 FUSE: -> [ID=0x21] Forget
2019/06/04 14:07:16 FUSE: -> [ID=0x20] Lookup 0x3 gen=2 valid=1m0s attr={valid=1m0s ino=16395 size=0 mode=drwxrwxrwx}
2019/06/04 14:07:16 FUSE: <- Getxattr [ID=0x22 Node=0x3 Uid=0 Gid=0 Pid=519] "security.capability" 24 @0
2019/06/04 14:07:16 FUSE: -> [ID=0x22] Getxattr error=operation not supported
2019/06/04 14:07:16 FUSE: <- Getxattr [ID=0x23 Node=0x3 Uid=0 Gid=0 Pid=519] "security.capability" 24 @0
2019/06/04 14:07:16 FUSE: -> [ID=0x23] Getxattr error=operation not supported
2019/06/04 14:07:16 FUSE: <- Open [ID=0x24 Node=0x3 Uid=0 Gid=0 Pid=519] dir=true fl=OpenReadOnly
INFO: 2019/06/04 14:07:16 Open:  /tmp/tmp OpenReadOnly
Error: 2019/06/04 14:07:16 FileHandleReader.go:33: [ /tmp/tmp ] Opening:  open /tmp/tmp: file does not exist
2019/06/04 14:07:16 FUSE: -> [ID=0x24] Open error=EIO: open /tmp/tmp: file does not exist
cat: /mnt/tmp/x: Input/output error

expect

read the right file after put new one

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions