xmlrpc Errors

[darkalchemy]

I'm getting a lot of xmlrpx errors, started a yesterday. I've increased the network.xmlrpc.size_limit.set to 128M and 256M but that didn't help.

This is the error that happens when using this rtcontrol is_complete=yes --stop --yes but the same error occurs for every command that I compared the results of.

Details

Traceback (most recent call last):
  File "/home/username/.local/bin/rtcontrol", line 7, in <module>
    sys.exit(run())
             ^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/rtcontrol.py", line 1057, in run
    RtorrentControl().run()
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/base.py", line 184, in run
    self.mainloop()
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/rtcontrol.py", line 847, in mainloop
    matches = futures[url].get()
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/pool.py", line 774, in get
    raise self._value
  File "/usr/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/rtcontrol.py", line 838, in fetch
    matches = list(e.items(view=view, prefetch=prefetch))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/torrent/rtorrent.py", line 873, in items
    raw_items = multi_call(*tuple(multi_args))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1122, in __call__
    return self.__send(self.__name, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/util/rpc.py", line 242, in __request
    return self.__request_switch(methodname, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/util/rpc.py", line 260, in __request_switch
    return self.__request_xml(methodname, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/util/rpc.py", line 133, in __request_xml
    response = self.__transport.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/io/scgi.py", line 165, in request
    return self.parse_response(io.BytesIO(response))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/io/scgi.py", line 59, in parse_response
    return super().parse_response(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1348, in parse_response
    p.feed(data)
  File "/usr/lib/python3.11/xmlrpc/client.py", line 451, in feed
    self._parser.Parse(data, False)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 40857
Traceback (most recent call last):
  File "/home/username/.local/bin/rtcontrol", line 7, in <module>
    sys.exit(run())
             ^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/rtcontrol.py", line 1057, in run
    RtorrentControl().run()
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/base.py", line 184, in run
    self.mainloop()
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/rtcontrol.py", line 847, in mainloop
    matches = futures[url].get()
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/multiprocessing/pool.py", line 774, in get
    raise self._value
  File "/usr/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/scripts/rtcontrol.py", line 838, in fetch
    matches = list(e.items(view=view, prefetch=prefetch))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/torrent/rtorrent.py", line 873, in items
    raw_items = multi_call(*tuple(multi_args))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1122, in __call__
    return self.__send(self.__name, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/util/rpc.py", line 242, in __request
    return self.__request_switch(methodname, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/util/rpc.py", line 260, in __request_switch
    return self.__request_xml(methodname, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/util/rpc.py", line 133, in __request_xml
    response = self.__transport.request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/io/scgi.py", line 165, in request
    return self.parse_response(io.BytesIO(response))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/username/.local/pipx/venvs/pyrosimple/lib/python3.11/site-packages/pyrosimple/io/scgi.py", line 59, in parse_response
    return super().parse_response(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/xmlrpc/client.py", line 1348, in parse_response
    p.feed(data)
  File "/usr/lib/python3.11/xmlrpc/client.py", line 451, in feed
    self._parser.Parse(data, False)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 969861

I'm running rTorrent 0.15.5/0.15.5 and pyrosimple version 2.14.2. Current torrent count is 16861. I removed a few torrents, down to 16254 and no more errors.

I downgraded 1 of my servers to rTorrent 0.15.3/0.15.3 just to see if it's rtorrent that is the issue. No change.

I'm not sure of the cause since some clients with 30k torrents have no issues but a client with 17 torrents does.

Thanks

[darkalchemy]

I added a few torrents until I found what might be causing the error. It looks like non utf-8 characters are the culprit.

In the ui, when the torrent name displays something like ~ in place of this character ’ or a blank space for this character á, rtcontrol errors. There hasn't been an update in some time, so I'm guessing the issue might be with tinyxml, since swizzin is building rtorrent with it enabled.

For now, I'm deleting these problem torrents. Besides that and fixing torrents, where possible, any ideas how to correct this?

Thanks again for your work, I would have had to switch to another client without it!!

[darkalchemy]

I've found that I can ignore the encoding error and the command continues. I don't know where this would go in this code, but I've added an ignore line to /usr/lib/python3.11/xmlrpc/client.py before line 1346

data = data.decode(encoding='utf-8', errors='ignore')

Again, I don't know how or where this would go in your code, but at least rtcontrol isn't crashing with encoding errors.

The encoded strings it's crashing on appears to be windows-1252.