Cannot run full node

[Aareon]

(venv) PS C:\Users\User\crankycoin> py .\run.py full


full node starting...


DEBUG:crankycoin:queue process starting...
DEBUG:crankycoin:worker process(es) starting...
Traceback (most recent call last):
  File ".\run.py", line 218, in <module>
    main(sys.argv[1:])
  File ".\run.py", line 212, in main
    full()
  File ".\run.py", line 119, in full
    full_node.start()
  File "C:\Users\User\crankycoin\crankycoin\node.py", line 79, in start
    wp.start()
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'ConfigDict.__init__.<locals>.<lambda>'
[INFO/MainProcess] process shutting down
[DEBUG/MainProcess] running all "atexit" finalizers with priority >= 0
[INFO/MainProcess] calling join() for process Process-1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
[INFO/Process-1] child process calling self.run()
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input
[INFO/Process-2] process shutting down
[DEBUG/Process-2] running all "atexit" finalizers with priority >= 0
[DEBUG/Process-2] running the remaining "atexit" finalizers
[INFO/Process-1] process shutting down
[DEBUG/Process-1] running all "atexit" finalizers with priority >= 0
[DEBUG/Process-1] running the remaining "atexit" finalizers
Process Process-1:
Traceback (most recent call last):
  File "C:\Users\User\crankycoin\crankycoin\services\queue.py", line 18, in start_queue
    frontend.bind(cls.QUEUE_BIND_IN)
  File "zmq\backend\cython\socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
  File "zmq\backend\cython\checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Protocol not supported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Users\User\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\User\crankycoin\crankycoin\services\queue.py", line 26, in start_queue
    logger.error("could not start queue: %s", e.message)
AttributeError: 'ZMQError' object has no attribute 'message'
[INFO/Process-1] process exiting with exitcode 1
[DEBUG/MainProcess] running the remaining "atexit" finalizers

[Aareon]

The above was using Python 3.6, the following is using Python 2.7

(venv2) PS C:\Users\User\crankycoin> py .\run.py full


full node starting...


DEBUG:crankycoin:queue process starting...
DEBUG:crankycoin:worker process(es) starting...
Traceback (most recent call last):
  File ".\run.py", line 218, in <module>
    main(sys.argv[1:])
  File ".\run.py", line 212, in main
    full()
  File ".\run.py", line 119, in full
    full_node.start()
  File "C:\Users\User\crankycoin\crankycoin\node.py", line 79, in start
    wp.start()
  File "C:\Python27\Lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\Lib\multiprocessing\forking.py", line 277, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\Lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\Lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\multiprocessing\forking.py", line 67, in dispatcher
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 401, in save_reduce
    save(args)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 554, in save_tuple
    save(element)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 606, in save_list
    self._batch_appends(iter(obj))
  File "C:\Python27\Lib\pickle.py", line 639, in _batch_appends
    save(x)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 606, in save_list
    self._batch_appends(iter(obj))
  File "C:\Python27\Lib\pickle.py", line 639, in _batch_appends
    save(x)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 554, in save_tuple
    save(element)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\Lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\Lib\pickle.py", line 754, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <function <lambda> at 0x0000000004306F98>: it's not found as bottle.<lambda>
[INFO/MainProcess] process shutting down
[DEBUG/MainProcess] running all "atexit" finalizers with priority >= 0
[INFO/MainProcess] calling join() for process Process-1
[INFO/Process-1] child process calling self.run()
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\Lib\multiprocessing\forking.py", line 381, in main
    self = load(from_parent)
  File "C:\Python27\Lib\pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "C:\Python27\Lib\pickle.py", line 864, in load
    dispatch[key](self)
  File "C:\Python27\Lib\pickle.py", line 886, in load_eof
    raise EOFError
EOFError
[INFO/Process-2] process shutting down
[DEBUG/Process-2] running all "atexit" finalizers with priority >= 0
[DEBUG/Process-2] running the remaining "atexit" finalizers
ERROR:crankycoin:could not start queue:
[INFO/Process-1] process shutting down
[DEBUG/Process-1] running all "atexit" finalizers with priority >= 0
[DEBUG/Process-1] running the remaining "atexit" finalizers
Process Process-1:
Traceback (most recent call last):
  File "C:\Python27\Lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Python27\Lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\User\crankycoin\crankycoin\services\queue.py", line 18, in start_queue
    frontend.bind(cls.QUEUE_BIND_IN)
  File "zmq\backend\cython\socket.pyx", line 547, in zmq.backend.cython.socket.Socket.bind
  File "zmq\backend\cython\checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
    raise ZMQError(errno)
ZMQError: Protocol not supported
[INFO/Process-1] process exiting with exitcode 1
[DEBUG/MainProcess] running the remaining "atexit" finalizers

[Aareon]

Could possibly have something to do with #3

[Aareon]

Might also have something to do with sailfish-team/sailfish#33

[Aareon]

And also possibly related to this https://stackoverflow.com/questions/15386121/does-zeromq-support-ipc-as-a-transport-channel-on-windows

[cranklin]

Thank you for raising this issue @Aareon.

[Aareon]

@cranklin im going to try to replicate this on Linux, but I think it's Windows specific

Edit: As far as I can tell, it is Windows specific. There are quite a few errors that I had to fix on side in my Ubuntu environment, but a PR isn't viable rn

[Aareon]

@cranklin what version of Python did you design this in, and on what OS?

Edit: I've rectified the protocol part of the issue by changing

    queue_bind_in: "ipc:///tmp/crank_in"
    queue_bind_out: "ipc:///tmp/crank_out"

in config/config.yaml to

    queue_bind_in: "tcp://127.0.0.1:5555"
    queue_bind_out: "tcp://127.0.0.1:5556"

Still having issues with

Traceback (most recent call last):
  File ".\run.py", line 219, in <module>
    main(sys.argv[1:])
  File ".\run.py", line 213, in main
    full()
  File ".\run.py", line 120, in full
    full_node.start()
  File "C:\Users\Aareon\crankycoin\crankycoin\node.py", line 79, in start
    wp.start()
  File "C:\Users\Aareon\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Users\Aareon\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Aareon\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\Aareon\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Aareon\AppData\Local\Programs\Python\Python36-32\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'ConfigDict.__init__.<locals>.<lambda>'

Edit: Creating a PR to rectify the protocol issue on Windows.

[Aareon]

In my Py2.7 environment, I get the following exception

pickle.PicklingError: Can't pickle <function <lambda> at 0x0000000004B97F28>: it's not found as bottle.<lambda>

[Aareon]

Is a move to asyncio out of the question? If we were to move, we could actually remove the requirement for bottle and use the builtin asyncio web framework

[Aareon]

That and the fact that IPC isn't compatible with Windows at all, this coin is completely unusable on Windows machines

[meherett]

Hey @Aareon! for this Error
pickle.PicklingError: Can't pickle <function at 0x0000000004B97F28>: it's not found as bottle.
pip install dill
You can fix it by import dill in run.py

[cranklin]

I'm looking into using asyncio's web framework. I like this idea, but if we rely on asyncio, we're going to have to drop support for Python 2.7 (maybe that's a good idea). I chose bottle because it's so lightweight and it's great for a thin API layer, but It seems to rely on Threading. I'll test this out in a different branch.

Regarding ZeroMQ, I wanted to keep the API layer very thin and have it responsible for a very light validation up front before pushing messages onto a queue. The consumers would ideally be scaled up/down based on the individual's machine and/or needs (independent of the externally facing API). ZeroMQ seemed like the ideal solution for interprocess communication without having to spin up a separate queue/broker. I'm open to alternatives (including python sockets). I globally set queue_bind_in and queue_bind_out to use TCP rather than IPC. It was an oversight on my part as I don't have a windows machine to test on.

Did @meherett 's suggestion for including import dill work for you? I included it in the requirements and imported dill in run.py