diff --git a/CHANGES.rst b/CHANGES.rst index b41d1169..0d4ce939 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -1,6 +1,15 @@ Changes ======= +0.0.3 (2017-11-02) +------------------ + +* Implemented fixed length 1D arrays +* Refactored argument packing and unpacking code +* Plenty of cleanups based on static code analysis +* Introduced ``Python``'s ``any`` functions in a number of places +* Fixed lots of typos and grammar issues in documentation + 0.0.2 (2017-07-28) ------------------ diff --git a/README.rst b/README.rst index ab31afd1..f39f67fa 100644 --- a/README.rst +++ b/README.rst @@ -71,15 +71,15 @@ allowing you to cleanly integrate Windows applications into your desktop.* Prerequisites ============= -+--------------------+--------------------------------------------------------------------------------------------------------+ -| for usage + - `CPython`_ 3.x (tested with 3.{4,5,6}) - no additional Python packages required + -| + - `Wine`_ 2.x (tested with 2.{5,6,10,12} regular & `staging`_) - expected to be in the user's `PATH`_ + -+--------------------+--------------------------------------------------------------------------------------------------------+ -| for tests + - `pytest`_ + -| + - `mingw cross-compiler`_ - for building DLLs against which examples and tests can be run + -+--------------------+--------------------------------------------------------------------------------------------------------+ -| for documentation + - `Sphinx`_ + -+--------------------+--------------------------------------------------------------------------------------------------------+ ++--------------------+-------------------------------------------------------------------------------------------------------------+ +| for usage + - `CPython`_ 3.x (tested with 3.{4,5,6}) - no additional Python packages required + +| + - `Wine`_ 2.x (tested with 2.{5,6,10,12,18,19} regular & `staging`_) - expected to be in the user's `PATH`_ + ++--------------------+-------------------------------------------------------------------------------------------------------------+ +| for tests + - `pytest`_ + +| + - `mingw cross-compiler`_ - for building DLLs against which examples and tests can be run + ++--------------------+-------------------------------------------------------------------------------------------------------------+ +| for documentation + - `Sphinx`_ + ++--------------------+-------------------------------------------------------------------------------------------------------------+ .. _CPython: https://www.python.org/ .. _Wine: https://www.winehq.org/ @@ -118,7 +118,7 @@ Start an interactive Python session under Unix and try the following: You have just witnessed ``msvcrt.dll`` in action on Unix. Interested in more? Check the `examples`_ in zugbruecke's documentation or read `ctypes' documentation`_. -A lot of code, which was written with ``cdll``, ``windll`` or ``oledll`` +A lot of code, which was written with ``ctypes``' ``cdll``, ``windll`` or ``oledll`` in mind and which runs under Windows, should run just fine with zugbruecke on Unix (assuming it does not use Windows features not supported by Wine). diff --git a/docs/configuration.rst b/docs/configuration.rst index 48a0b5f8..abdf65b9 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -27,7 +27,7 @@ Configuring the session constructor ----------------------------------- If you chose to start your own session with ``zugbruecke.session()``, the session -constructor can be provided with a dictionaries containing parameters. +constructor can be provided with a dictionary containing parameters. Configuration files ------------------- @@ -44,7 +44,7 @@ Parameters passed directly into the *zugbuecke* session constructor will always be given priority. Beyond that, missing parameters are being looked for location after location in the above listed places. If, after checking for configuration files in all those locations, there are still parameters -left unconfigured, *zugbuecke* will fill them with its defaults. A parameter +left undefined, *zugbuecke* will fill them with its defaults. A parameter found in a location higher in the list will always be given priority over a the same parameter with different content found in a location further down the list. @@ -73,13 +73,13 @@ necessary. ``stdout`` (bool) ^^^^^^^^^^^^^^^^^ -Tells *zugbuecke* to show messages its sub-processes write to ``stdout``. +Tells *zugbuecke* to show messages its sub-processes are writing to ``stdout``. ``True`` by default. ``stderr`` (bool) ^^^^^^^^^^^^^^^^^ -Tells *zugbuecke* to show messages its sub-processes write to ``stderr``. +Tells *zugbuecke* to show messages its sub-processes are writing to ``stderr``. ``True`` by default. ``logwrite`` (bool) @@ -106,7 +106,7 @@ Default is ``win32``, even on 64-bit systems. It appears to be a more stable con The ``version`` parameter tells *zugbuecke* what version of the *Windows* *CPython* interpreter it should use. By default, it is set to ``3.5.3``. -Please not that 3.4 and earlier are not supported. In the opposite direction, at the time of +Please note that 3.4 and earlier are not supported. In the opposite direction, at the time of writing, 3.6 (and later) does not work under *Wine* due to a `bug in Wine`_. .. _bug in Wine: https://github.com/pleiszenburg/zugbruecke/issues/13 @@ -114,6 +114,6 @@ writing, 3.6 (and later) does not work under *Wine* due to a `bug in Wine`_. ``dir`` (str) ^^^^^^^^^^^^^ -This parameter defines the root directory of *zugbruecke*. It is where *zugbruecke*'s +This parameter defines the root directory of *zugbruecke*. This is where *zugbruecke*'s own *Wine* profile folder is stored (``WINEPREFIX``) and where the :ref:`Wine Python environment ` resides. By default, it is set to ``~/.zugbruecke``. diff --git a/docs/examples.rst b/docs/examples.rst index 26e30bbf..9af8d1f9 100644 --- a/docs/examples.rst +++ b/docs/examples.rst @@ -34,13 +34,13 @@ looks somewhat like this: float __stdcall __declspec(dllimport) simple_demo_routine(float param_a, float param_b) { return param_a - (param_a / param_b); } -Because of the drop-in replacement design of zugbruecke, it is possible to write -Python code which works under both Unices and Windows. +Because of the drop-in replacement design of *zugbruecke*, it is possible to write +*Python* code which works under both *Unices* and *Windows*. .. code:: python from sys import platform - if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: + if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): from zugbruecke import cdll elif platform.startswith('win'): from ctypes import cdll diff --git a/docs/interoperability.rst b/docs/interoperability.rst index 37b7d9d2..e1987842 100644 --- a/docs/interoperability.rst +++ b/docs/interoperability.rst @@ -31,7 +31,7 @@ Return value * ``path_out`` (str) Converts an absolute or relative *Unix* path into a *Windows* path. It does -not check, whether the path actually exists ot not. It uses *Wine*'s internal +not check, whether the path actually exists or not. It uses *Wine*'s internal implementation for path conversion. Method: ``path_wine_to_unix`` @@ -46,5 +46,5 @@ Return value * ``path_out`` (str) Converts an absolute or relative *Windows* path into a *Unix* path. It does -not check, whether the path actually exists ot not. It uses *Wine*'s internal +not check, whether the path actually exists or not. It uses *Wine*'s internal implementation for path conversion. diff --git a/docs/introduction.rst b/docs/introduction.rst index 4e080a66..cc9f51f1 100644 --- a/docs/introduction.rst +++ b/docs/introduction.rst @@ -63,7 +63,7 @@ Implementation During the first import of *zugbruecke*, a stand-alone *Windows*-version of the *CPython* interpreter corresponding to the used *Unix*-version is automatically downloaded and placed into the module's configuration folder (by default located at -``~/.zugbruecke/``). Next to it, also during first import, zugbruecke +``~/.zugbruecke/``). Next to it, also during first import, *zugbruecke* generates its own *Wine*-profile directory for being used with a dedicated ``WINEPREFIX``. This way, any undesirable interferences with other *Wine*-profile directories containing user settings and unrelated software are avoided. diff --git a/docs/memsync.rst b/docs/memsync.rst index 6d332d04..629ae8c4 100644 --- a/docs/memsync.rst +++ b/docs/memsync.rst @@ -14,7 +14,7 @@ The memory synchronization protocol Because *zugbruecke* runs code in a separate *Python* interpreter on *Wine*, pointers pose a special kind of problem. They can, unfortunately, not be passed from the code running on the *Unix* side to the code running on the *Wine* side -or vice versa. This is why the memory pointers are pointing to must be kept in +or vice versa. This is why the memory (to where pointers are pointing) must be kept in sync on both sides. The memory synchronization can be controlled by the user through the ``memsync`` protocol. ``memsync`` implements special directives, which do not interfere with *ctypes* should the code be required to run on @@ -48,7 +48,7 @@ Consider the following example DLL routine in C: } } -It is a really simple implementation of the "bubblesort" algorithm, which accepts +It is a simple implementation of the "bubblesort" algorithm, which accepts a pointer to an array of floats of arbitrary length and an integer with length information. The array is being sorted within its own memory, so the caller expects a sorted array at the passed pointer after the call. @@ -120,7 +120,7 @@ The complete example, which will run on *Unix* and on *Windows* looks just like .. code:: python from sys import platform - if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: + if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): from zugbruecke import windll, cast, pointer, POINTER, c_float, c_int elif platform.startswith('win'): from ctypes import windll, cast, pointer, POINTER, c_float, c_int diff --git a/docs/security.rst b/docs/security.rst index ab80d757..fce2f42a 100644 --- a/docs/security.rst +++ b/docs/security.rst @@ -24,4 +24,4 @@ The following problems also directly apply to *zugbruecke*: .. _Windows malware: https://en.wikipedia.org/wiki/Wine_(software)#Security .. _FAQ at WineHQ: https://wiki.winehq.org/FAQ#Should_I_run_Wine_as_root.3F -*zugbruecke* does not actively prohibit the its use with root privileges. +*zugbruecke* does not actively prohibit its use with root privileges. diff --git a/docs/session.rst b/docs/session.rst index e96f9758..f3ef4ae1 100644 --- a/docs/session.rst +++ b/docs/session.rst @@ -50,7 +50,7 @@ All limitations and features of those constructors apply, e.g. the possibility of referring to DLLs without their .dll-file-extension or the use of absolute or relative *Windows* paths. *zugbruecke* offers :ref:`methods for path conversion ` if required. If *ctypes* on the *Wine* side raises an error, e.g. because the DLL -could not be found, the error will be re-raised by *zugbruecke* on the Unix side. +can not be found, the error will be re-raised by *zugbruecke* on the Unix side. The third parameter is optional and allows to pass a dict with the following keys: diff --git a/docs/wineenv.rst b/docs/wineenv.rst index 54ed0fd8..4640130a 100644 --- a/docs/wineenv.rst +++ b/docs/wineenv.rst @@ -21,7 +21,7 @@ Command: ``wine-python`` This command behaves just like the regular ``python`` command in a *Unix* shell, except that it fires up a *Windows* *Python* interpreter on top of *Wine*. It works -will all regular parameters and switches, even excepts pipes and can be launched in +with all regular parameters and switches, accepts pipes and can be launched in interactive mode. You can also use it for creating executable *Python* scripts by adding the following diff --git a/examples/test_cookbook.py b/examples/test_cookbook.py index b21fd896..84477daf 100755 --- a/examples/test_cookbook.py +++ b/examples/test_cookbook.py @@ -32,9 +32,9 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import sys -import os -import time +# import sys +# import os +# import time from sys import argv, platform import timeit @@ -46,7 +46,7 @@ except: pass -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): f = open('.zugbruecke.json', 'w') if TIMING_RUN: diff --git a/examples/test_zugbruecke.py b/examples/test_zugbruecke.py index 6377de70..cbefcb2c 100755 --- a/examples/test_zugbruecke.py +++ b/examples/test_zugbruecke.py @@ -32,17 +32,17 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import sys -import os -import time +# import sys +# import os +# import time from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): f = open('.zugbruecke.json', 'w') f.write('{"log_level": 10}') f.close() - + import zugbruecke ctypes = zugbruecke diff --git a/examples/test_zugbruecke_perf.py b/examples/test_zugbruecke_perf.py index ac7efd9e..684cedaa 100755 --- a/examples/test_zugbruecke_perf.py +++ b/examples/test_zugbruecke_perf.py @@ -32,13 +32,13 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import sys -import os -import time +# import sys +# import os +# import time import timeit from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): f = open('.zugbruecke.json', 'w') f.write('{}') diff --git a/makefile b/makefile index 633644d1..7e9445d0 100644 --- a/makefile +++ b/makefile @@ -60,3 +60,9 @@ test: wine-pytest -rm tests/__pycache__/*.pyc pytest + +test_quick: + -rm tests/__pycache__/*.pyc + wine-pytest + -rm tests/__pycache__/*.pyc + pytest diff --git a/setup.py b/setup.py index 2b55b1c7..6a7ac61a 100644 --- a/setup.py +++ b/setup.py @@ -46,7 +46,7 @@ # Bump version HERE! -_version_ = '0.0.2' +_version_ = '0.0.3' # List all versions of Python which are supported diff --git a/src/zugbruecke/core/arg_contents.py b/src/zugbruecke/core/arg_contents.py index d5f374fa..6e431271 100644 --- a/src/zugbruecke/core/arg_contents.py +++ b/src/zugbruecke/core/arg_contents.py @@ -50,346 +50,242 @@ class arg_contents_class(): - def client_pack_arg_list(self, argtypes_d_sub, args): - """ - TODO Optimize for speed! - Can call itself recursively! - """ - - # Shortcut for speed - args_package_list = [] + def client_pack_arg_list(self, args_tuple, argtypes_def_dict): # Step through arguments - for arg_index, argtype_d in enumerate(argtypes_d_sub): - - # Fetch current argument by index from tuple or by name from struct/kw - if type(args) is list or type(args) is tuple: - arg = args[arg_index] - else: - arg = getattr(args, argtype_d['n']) - - # TODO: - # append tuple to list "args_package_list" - # tuple contains: (argtype_d['n'], argument content / value) - # pointer: arg.value or arg.contents.value - # (value: Append value from ctypes datatype, because most of their Python equivalents are immutable) - # (contents.value: Append value from ctypes datatype pointer ...) - # by value: just "arg" - - try: - - arg_value = arg # Set up arg for iterative unpacking - for flag in argtype_d['f']: # step through flags - - # Handle pointers - if flag == FLAG_POINTER: - - # There are two ways of getting the actual value - if hasattr(arg_value, 'value'): - arg_value = arg_value.value - elif hasattr(arg_value, 'contents'): - arg_value = arg_value.contents - else: - raise # TODO - - # Handle arrays - elif flag > 0: - - arg_value = arg_value[:] - - # Handle unknown flags - else: - - raise # TODO - except: - - self.log.err(pf(arg_value)) + args_package_list = [] + for arg_index, arg_raw in enumerate(args_tuple): + args_package_list.append(self.__pack_item__(arg_raw, argtypes_def_dict[arg_index])) - self.log.err(' abc') - self.log.err(pf(arg_value)) + # Return parameter message list - MUST WORK WITH PICKLE + return args_package_list - # Handle fundamental types - if argtype_d['g'] == GROUP_FUNDAMENTAL: - # Append argument to list ... - args_package_list.append((argtype_d['n'], arg_value)) + def client_unpack_return_list(self, old_arguments_list, args_package_list, argtypes_d): - # Handle structs - elif argtype_d['g'] == GROUP_STRUCT: + # Step through arguments + new_arguments_list = [] + for arg_index, arg in enumerate(args_package_list): + self.__sync_item__( + old_arguments_list[arg_index], + self.__unpack_item__(arg[1], argtypes_d[arg_index]), + argtypes_d[arg_index] + ) - # Reclusively call this routine for packing structs - args_package_list.append((argtype_d['n'], self.client_pack_arg_list( - argtype_d['_fields_'], arg - ))) - # Handle everything else ... likely pointers handled by memsync - else: + def server_pack_return_list(self, args_tuple, argtypes_def_dict): - # Just return None - will (hopefully) be overwritten by memsync - args_package_list.append(None) + # Step through arguments + args_package_list = [] + for arg_index, arg_raw in enumerate(args_tuple): + args_package_list.append(self.__pack_item__(arg_raw, argtypes_def_dict[arg_index])) # Return parameter message list - MUST WORK WITH PICKLE return args_package_list - def client_unpack_return_list(self, argtypes_d, args, return_dict): - """ - TODO Optimize for speed! - """ - - # Get arguments' list - arguments_list = return_dict['args'] + def server_unpack_arg_list(self, args_package_list, argtypes_d): # Step through arguments - for arg_index, arg in enumerate(args): - - # Fetch definition of current argument - argtype_d = argtypes_d[arg_index] - - # Handle fundamental types - if argtype_d['g'] == GROUP_FUNDAMENTAL: - - # Start process with plain old argument - arg_value = args[arg_index] - # New value is: arguments_list[arg_index] - - # Step through flags - for flag in argtype_d['f']: - - # Handle pointers - if flag == FLAG_POINTER: + arguments_list = [] + for arg_index, arg in enumerate(args_package_list): + arguments_list.append(self.__unpack_item__(arg[1], argtypes_d[arg_index])) - # There are two ways of getting the actual value - # if hasattr(arg_value, 'value'): - # arg_value = arg_value.value - if hasattr(arg_value, 'contents'): - arg_value = arg_value.contents - else: - arg_value = arg_value + # Return args as list, will be converted into tuple on call + return arguments_list - # Handle arrays - elif flag > 0: - arg_value = arg_value + def __pack_item__(self, arg_raw, arg_def_dict): - # Handle unknown flags - else: + arg_value = arg_raw # Set up arg for iterative unpacking + for flag in arg_def_dict['f']: # step through flags - raise # TODO + # Handle pointers + if flag == FLAG_POINTER: + # There are two ways of getting the actual value if hasattr(arg_value, 'value'): - arg_value.value = arguments_list[arg_index] + arg_value = arg_value.value + elif hasattr(arg_value, 'contents'): + arg_value = arg_value.contents + elif hasattr(arg_value, '_fields_'): + # HACK it's likely just a struct passed "as is", + # configured as a pointer in argtypes, + # but without the intention of letting the routine change it. + # ctypes does not mind ... (?) + pass else: - arg_value = arguments_list[arg_index] - - # # If by reference ... - # if argtype_d['p']: - # # Put value back into its ctypes datatype - # args[arg_index].value = arguments_list[arg_index] - # # If by value - # else: - # # Nothing to do - # pass - - # Handle everything else (structures and "the other stuff") - else: + raise # TODO - # HACK TODO - pass + # Handle arrays + elif flag > 0: + arg_value = arg_value[:] # TODO arrays of arrays (fixed length) - def server_pack_return_list(self, argtypes_d, args): - """ - TODO: Optimize for speed! - """ + # Handle unknown flags + else: - # Start argument list as a list - args_package_list = [] + raise # TODO - # Step through arguments - for arg_index, arg in enumerate(args): + # Handle fundamental types + if arg_def_dict['g'] == GROUP_FUNDAMENTAL: - # Fetch definition of current argument - argtype_d = argtypes_d[arg_index] + # Append argument to list ... + return ( + arg_def_dict['n'], + self.__pack_item_fundamental__(arg_value, arg_def_dict) + ) - arg_value = arg # Set up arg for iterative unpacking - for flag in argtype_d['f']: # step through flags + # Handle structs + elif arg_def_dict['g'] == GROUP_STRUCT: - # Handle pointers - if flag == FLAG_POINTER: + # Reclusively call this routine for packing structs + return ( + arg_def_dict['n'], + self.__pack_item_struct__(arg_value, arg_def_dict) + ) - # There are two ways of getting the actual value - if hasattr(arg_value, 'value'): - arg_value = arg_value.value - elif hasattr(arg_value, 'contents'): - arg_value = arg_value.contents - else: - pass - # self.log.err(pf(arg_value)) - # raise # TODO + # Handle everything else ... likely pointers handled by memsync + else: - # Handle arrays - elif flag > 0: + # Just return None - will (hopefully) be overwritten by memsync + return (None, None) - arg_value = arg_value[:] - # Handle unknown flags - else: + def __pack_item_fundamental__(self, arg_raw, arg_def_dict): - self.log.err('ERROR in __pack_return__, flag %d' % flag) - raise # TODO + if hasattr(arg_raw, 'value'): + return arg_raw.value + else: + return arg_raw - self.log.err(' efg') - self.log.err(pf(arg_value)) - # Handle fundamental types by value - if argtype_d['g'] == GROUP_FUNDAMENTAL: + def __pack_item_struct__(self, struct_raw, struct_def_dict): - if hasattr(arg_value, 'value'): - arg_value = arg_value.value - args_package_list.append(arg_value) - - # # If by reference ... - # if argtype_d['p']: - # # Append value from ctypes datatype (because most of their Python equivalents are immutable) - # args_package_list.append(arg.value) - # # If by value ... - # else: - # # Nothing to do ... - # args_package_list.append(None) - - # Handle everything else (structures etc) - else: + # Shortcut for speed + fields_package_list = [] - # HACK TODO - args_package_list.append(None) + # Step through fields of dict + for field_def_dict in struct_def_dict['_fields_']: - return args_package_list + fields_package_list.append(self.__pack_item__( + getattr(struct_raw, field_def_dict['n']), field_def_dict + )) + # Return parameter message list - MUST WORK WITH PICKLE + return fields_package_list - def server_unpack_arg_list(self, argtypes_d, args_package_list): - """ - TODO Optimize for speed! - """ - # Start argument list as a list (will become a tuple) - arguments_list = [] + def __sync_item__(self, old_arg, new_arg, arg_def_dict): - # Step through arguments - for arg_index, arg in enumerate(args_package_list): + # Handle fundamental types + if arg_def_dict['g'] == GROUP_FUNDAMENTAL: - # Fetch definition of current argument - argtype_d = argtypes_d[arg_index] + # HACK let's handle pointers to scalars like before + if len(arg_def_dict['f']) == 1 and arg_def_dict['f'][0] == FLAG_POINTER: - # Handle fundamental types - if argtype_d['g'] == GROUP_FUNDAMENTAL: - - try: + if hasattr(old_arg, 'contents'): + old_arg_ref = old_arg.contents + else: + old_arg_ref = old_arg + if hasattr(new_arg, 'contents'): + new_arg_ref = new_arg.contents + else: + new_arg_ref = new_arg + if hasattr(new_arg_ref, 'value'): + new_arg_value = new_arg_ref.value + else: + new_arg_value = new_arg_ref + if hasattr(old_arg_ref, 'value'): + old_arg_ref.value = new_arg_value + else: + old_arg_ref = new_arg_value - # Start process with plain argument - arg_rebuilt = getattr(ctypes, argtype_d['t'])(arg[1]) + # HACK let's handle 1D fixed length arrays + elif len(arg_def_dict['f']) == 2 and arg_def_dict['f'][0] == FLAG_POINTER and arg_def_dict['f'][1] > 0: - # Step through flags - for flag in argtype_d['f']: + if hasattr(old_arg, 'contents'): + old_arg_ref = old_arg.contents + else: + old_arg_ref = old_arg + if hasattr(new_arg, 'contents'): + new_arg_ref = new_arg.contents + else: + new_arg_ref = new_arg + old_arg_ref[:] = new_arg_ref[:] - if flag == FLAG_POINTER: + else: - pass # Nothing to do? + pass # TODO struct ... - elif flag > 0: - raise + def __unpack_item__(self, arg_raw, arg_def_dict): - else: + # Handle fundamental types + if arg_def_dict['g'] == GROUP_FUNDAMENTAL: - raise + return self.__unpack_item_fundamental__(arg_raw, arg_def_dict) - # # By reference - # if argtype_d['p']: - # # Put value back into its ctypes datatype - # arguments_list.append( - # getattr(ctypes, argtype_d['t'])(arg[1]) - # ) - # # By value - # else: - # # Append value - # arguments_list.append(arg[1]) + # Handle structs + elif arg_def_dict['g'] == GROUP_STRUCT: - arguments_list.append(arg_rebuilt) + return self.__unpack_item_struct__(arg_raw, arg_def_dict) - except: + # Handle voids (likely mensync stuff) + elif arg_def_dict['g'] == GROUP_VOID: - self.log.err('ERROR in __unpack_arguments__, fundamental datatype path') - self.log.err(traceback.format_exc()) + # Return a placeholder + return 0 - # Handle structs - elif argtype_d['g'] == GROUP_STRUCT: + # Handle everything else ... + else: - # Generate new instance of struct datatype - struct_inst = self.struct_type_dict[argtype_d['t']]() + # HACK TODO + self.log.err('__unpack_item__ NEITHER STRUCT NOR FUNDAMENTAL?') + self.log.err(str(arg_def_dict['g'])) + return None - # Unpack values into struct - self.server_unpack_arg_struct_dict(argtype_d['_fields_'], struct_inst, arg[1]) - # Append struct to list - arguments_list.append(struct_inst) + def __unpack_item_fundamental__(self, arg_rebuilt, arg_def_dict): - # Handle everything else ... - else: + # Handle scalars, whether pointer or not + if arg_def_dict['s']: + arg_rebuilt = getattr(ctypes, arg_def_dict['t'])(arg_rebuilt) - # HACK TODO - arguments_list.append(0) + # Step through flags in reverse order + for flag in reversed(arg_def_dict['f']): - # Return args as list, will be converted into tuple on call - return arguments_list + if flag == FLAG_POINTER: + arg_rebuilt = ctypes.pointer(arg_rebuilt) - def server_unpack_arg_struct_dict(self, argtypes_d_sub, struct_inst, args_list): - """ - TODO Optimize for speed! - Can be called recursively! - """ + elif flag > 0: - # Step through arguments - for arg_index, arg in enumerate(args_list): + # TODO does not really handle arrays of arrays (yet) + arg_rebuilt = (flag * getattr(ctypes, arg_def_dict['t']))(*arg_rebuilt) - # Get current argument definition - argtype_d = argtypes_d_sub[arg_index] + else: - # Handle fundamental types - if argtype_d['g'] == GROUP_FUNDAMENTAL: + raise - # Put value back into its ctypes datatype - setattr( - struct_inst, # struct instance to be modified - arg[0], # parameter name (from tuple) - getattr(ctypes, argtype_d['t'])(arg[1]) # ctypes instance of type with value from tuple - ) + return arg_rebuilt - # TODO pointers and arrays - # Handle structs - elif argtype_d['g'] == GROUP_STRUCT: + def __unpack_item_struct__(self, args_list, struct_def_dict): - # Generate new instance of struct datatype - struct_sub_inst = self.struct_type_dict[argtype_d['t']]() + # Generate new instance of struct datatype + struct_inst = self.struct_type_dict[struct_def_dict['t']]() - # Unpack values into struct - self.__unpack_arguments_struct__(argtype_d['_fields'], struct_sub_inst, arg[1]) + # Fetch fields for speed + struct_fields = struct_def_dict['_fields_'] - # Append struct to struct TODO handle pointer to structs! - setattr( - struct_inst, # struct instance to be modified - arg[0], # parameter name (from tuple) - struct_sub_inst # value from tuple - ) + # Step through arguments + for arg_index, arg in enumerate(args_list): - # Handle everything else ... - else: + setattr( + struct_inst, # struct instance to be modified + arg[0], # parameter name (from tuple) + self.__unpack_item__(arg[1], struct_fields[arg_index]) # parameter value + ) - # HACK TODO - setattr( - struct_inst, # struct instance to be modified - arg[0], # parameter name (from tuple) - 0 # least destructive value ... - ) + return struct_inst diff --git a/src/zugbruecke/core/arg_definition.py b/src/zugbruecke/core/arg_definition.py index cdaafc27..a8c316a8 100644 --- a/src/zugbruecke/core/arg_definition.py +++ b/src/zugbruecke/core/arg_definition.py @@ -198,11 +198,15 @@ def __pack_definition_dict__(self, datatype, field_name = None): type_name = datatype.__name__ group_name = type(datatype).__name__ + # Flag pure scalars as, well, pure scalars (for speed) + flag_scalar = len([flag for flag in flag_list if flag > 0]) == 0 + # Fundamental ('simple') C types if group_name == 'PyCSimpleType': return { 'f': flag_list, + 's': flag_scalar, 'n': field_name, # kw 't': type_name, # Type name, such as 'c_int' 'g': GROUP_FUNDAMENTAL @@ -211,8 +215,13 @@ def __pack_definition_dict__(self, datatype, field_name = None): # Structs elif group_name == 'PyCStructType': + # Keep track of datatype on client side + if type_name not in self.struct_type_dict.keys(): + self.struct_type_dict[type_name] = datatype + return { 'f': flag_list, + 's': flag_scalar, 'n': field_name, # kw 't': type_name, # Type name, such as 'c_int' 'g': GROUP_STRUCT, @@ -226,6 +235,7 @@ def __pack_definition_dict__(self, datatype, field_name = None): return { 'f': flag_list, + 's': flag_scalar, 'n': field_name, # kw 't': type_name, # Type name, such as 'c_int' 'g': GROUP_VOID # Let's try void diff --git a/src/zugbruecke/core/dll_client.py b/src/zugbruecke/core/dll_client.py index da33efc5..0edbb435 100644 --- a/src/zugbruecke/core/dll_client.py +++ b/src/zugbruecke/core/dll_client.py @@ -31,9 +31,6 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import ctypes -from pprint import pformat as pf - from .routine_client import routine_client_class diff --git a/src/zugbruecke/core/dll_server.py b/src/zugbruecke/core/dll_server.py index b048ddca..234a6a5a 100644 --- a/src/zugbruecke/core/dll_server.py +++ b/src/zugbruecke/core/dll_server.py @@ -31,8 +31,7 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import ctypes -from pprint import pformat as pf +import traceback from .lib import get_hash_of_string from .routine_server import routine_server_class diff --git a/src/zugbruecke/core/interpreter.py b/src/zugbruecke/core/interpreter.py index d6271476..dcf3dc6c 100644 --- a/src/zugbruecke/core/interpreter.py +++ b/src/zugbruecke/core/interpreter.py @@ -35,7 +35,6 @@ import signal import subprocess import threading -import time # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ diff --git a/src/zugbruecke/core/log.py b/src/zugbruecke/core/log.py index f70d29e5..2df0748a 100644 --- a/src/zugbruecke/core/log.py +++ b/src/zugbruecke/core/log.py @@ -32,9 +32,7 @@ # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ import json -import re import sys -import threading import time from .lib import get_free_port @@ -125,7 +123,6 @@ def __append_message_to_log__(self, message): def __compile_message_dict_list__(self, message, pipe_name, level): message_lines = [] - message_line_max = 80 for line in message.split('\n'): if line.strip() != '': diff --git a/src/zugbruecke/core/routine_client.py b/src/zugbruecke/core/routine_client.py index f6da3607..cbc7d299 100644 --- a/src/zugbruecke/core/routine_client.py +++ b/src/zugbruecke/core/routine_client.py @@ -125,14 +125,14 @@ def __handle_call__(self, *args): # Actually call routine in DLL! TODO Handle kw ... return_dict = self.__handle_call_on_server__( - self.client_pack_arg_list(self.argtypes_d, args), mem_package_list + self.client_pack_arg_list(args, self.argtypes_d), mem_package_list ) # Log status self.log.out('[routine-client] ... received feedback from server, unpacking ...') # Unpack return dict (for pointers and structs) - self.client_unpack_return_list(self.argtypes_d, args, return_dict) + self.client_unpack_return_list(args, return_dict['args'], self.argtypes_d) # Unpack memory self.client_unpack_memory_list(return_dict['memory'], memory_transport_handle) diff --git a/src/zugbruecke/core/routine_server.py b/src/zugbruecke/core/routine_server.py index 6df4c46e..2aaa88bc 100644 --- a/src/zugbruecke/core/routine_server.py +++ b/src/zugbruecke/core/routine_server.py @@ -31,7 +31,6 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import ctypes from pprint import pformat as pf import traceback @@ -78,7 +77,7 @@ def __handle_call__(self, arg_message_list, arg_memory_list): self.log.out('[routine-server] Trying call routine "%s" ...' % self.name) # Unpack passed arguments, handle pointers and structs ... - args_list = self.server_unpack_arg_list(self.argtypes_d, arg_message_list) + args_list = self.server_unpack_arg_list(arg_message_list, self.argtypes_d) # Unpack pointer data memory_handle = self.server_unpack_memory_list(args_list, arg_memory_list, self.memsync_d) @@ -96,7 +95,7 @@ def __handle_call__(self, arg_message_list, arg_memory_list): arg_memory_list = self.server_pack_memory_list(memory_handle) # Get new arg message list - arg_message_list = self.server_pack_return_list(self.argtypes_d, args_list) + arg_message_list = self.server_pack_return_list(args_list, self.argtypes_d) # Log status self.log.out('[routine-server] ... done.') diff --git a/src/zugbruecke/core/session_client.py b/src/zugbruecke/core/session_client.py index 9d32650e..ce40593c 100644 --- a/src/zugbruecke/core/session_client.py +++ b/src/zugbruecke/core/session_client.py @@ -66,7 +66,7 @@ def __init__(self, parameter = {}, force = False): self.__init_stage_1__(parameter, force) - def ctypes_FormatError(code = None): + def ctypes_FormatError(self, code = None): # If in stage 1, fire up stage 2 if self.stage == 1: @@ -76,7 +76,7 @@ def ctypes_FormatError(code = None): return self.client.ctypes_FormatError(code) - def ctypes_get_last_error(): + def ctypes_get_last_error(self): # If in stage 1, fire up stage 2 if self.stage == 1: @@ -86,7 +86,7 @@ def ctypes_get_last_error(): return self.client.ctypes_get_last_error() - def ctypes_GetLastError(): + def ctypes_GetLastError(self): # If in stage 1, fire up stage 2 if self.stage == 1: @@ -96,7 +96,7 @@ def ctypes_GetLastError(): return self.client.ctypes_GetLastError() - def ctypes_set_last_error(value): + def ctypes_set_last_error(self, value): # If in stage 1, fire up stage 2 if self.stage == 1: @@ -106,7 +106,7 @@ def ctypes_set_last_error(value): return self.client.ctypes_set_last_error(value) - def ctypes_WinError(code = None, descr = None): + def ctypes_WinError(self, code = None, descr = None): # If in stage 1, fire up stage 2 if self.stage == 1: diff --git a/src/zugbruecke/core/session_server.py b/src/zugbruecke/core/session_server.py index 617e337e..1c293146 100644 --- a/src/zugbruecke/core/session_server.py +++ b/src/zugbruecke/core/session_server.py @@ -32,9 +32,6 @@ # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ import ctypes -import os -from pprint import pformat as pf -import sys import traceback from .dll_server import dll_server_class diff --git a/src/zugbruecke/core/wineenv.py b/src/zugbruecke/core/wineenv.py index a70ea751..03c472de 100644 --- a/src/zugbruecke/core/wineenv.py +++ b/src/zugbruecke/core/wineenv.py @@ -35,7 +35,6 @@ import os import shutil import subprocess -import tempfile import urllib.request import zipfile @@ -103,7 +102,7 @@ def setup_wine_python(arch, version, directory, overwrite = False): if preexisting and overwrite: # Delete folder - shutil.rmtree(path) + shutil.rmtree(python_exe_path) # Make sure the target directory exists if not os.path.exists(directory): diff --git a/tests/test_avg.py b/tests/test_avg.py index 5b73a99d..2756c20e 100644 --- a/tests/test_avg.py +++ b/tests/test_avg.py @@ -34,7 +34,7 @@ import pytest from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): import zugbruecke ctypes = zugbruecke elif platform.startswith('win'): diff --git a/tests/test_bubblesort.py b/tests/test_bubblesort.py index 08ed908e..4e9ab9f6 100644 --- a/tests/test_bubblesort.py +++ b/tests/test_bubblesort.py @@ -34,7 +34,7 @@ import pytest from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): import zugbruecke ctypes = zugbruecke elif platform.startswith('win'): diff --git a/tests/test_devide.py b/tests/test_devide.py index a8bab3de..8f19fd3b 100644 --- a/tests/test_devide.py +++ b/tests/test_devide.py @@ -31,10 +31,10 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import pytest +# import pytest from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): import zugbruecke ctypes = zugbruecke elif platform.startswith('win'): diff --git a/tests/test_distance.py b/tests/test_distance.py index a9419cb1..83815f0a 100644 --- a/tests/test_distance.py +++ b/tests/test_distance.py @@ -34,7 +34,7 @@ import pytest from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): import zugbruecke ctypes = zugbruecke elif platform.startswith('win'): diff --git a/tests/test_gdc.py b/tests/test_gdc.py index ebcfb7ce..52991ac4 100644 --- a/tests/test_gdc.py +++ b/tests/test_gdc.py @@ -31,10 +31,10 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import pytest +# import pytest from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): import zugbruecke ctypes = zugbruecke elif platform.startswith('win'): diff --git a/tests/test_in_mantel.py b/tests/test_in_mantel.py index a6a96787..6ad1b8be 100644 --- a/tests/test_in_mantel.py +++ b/tests/test_in_mantel.py @@ -31,10 +31,10 @@ # IMPORT # +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -import pytest +# import pytest from sys import platform -if True in [platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]: +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): import zugbruecke ctypes = zugbruecke elif platform.startswith('win'): diff --git a/tests/test_mix_rgb_colors.py b/tests/test_mix_rgb_colors.py new file mode 100644 index 00000000..e0d51439 --- /dev/null +++ b/tests/test_mix_rgb_colors.py @@ -0,0 +1,81 @@ +# -*- coding: utf-8 -*- + +""" + +ZUGBRUECKE +Calling routines in Windows DLLs from Python scripts running on unixlike systems +https://github.com/pleiszenburg/zugbruecke + + tests/test_mix_rgb_colors.py: Tests 1D fixed length arrays with ints + + Required to run on platform / side: [UNIX, WINE] + + Copyright (C) 2017 Sebastian M. Ernst + + +The contents of this file are subject to the GNU Lesser General Public License +Version 2.1 ("LGPL" or "License"). You may not use this file except in +compliance with the License. You may obtain a copy of the License at +https://www.gnu.org/licenses/old-licenses/lgpl-2.1.txt +https://github.com/pleiszenburg/zugbruecke/blob/master/LICENSE + +Software distributed under the License is distributed on an "AS IS" basis, +WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the +specific language governing rights and limitations under the License. + + +""" + + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# IMPORT +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +# import pytest + +from sys import platform +if any([platform.startswith(os_name) for os_name in ['linux', 'darwin', 'freebsd']]): + import zugbruecke as ctypes +elif platform.startswith('win'): + import ctypes + + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# CLASSES AND ROUTINES +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +class sample_class: + + + def __init__(self): + + self.__dll__ = ctypes.windll.LoadLibrary('tests/demo_dll.dll') + + # void mix_rgb_colors(int8_t [3], int8_t [3], int8_t *) + self.__mix_rgb_colors__ = self.__dll__.mix_rgb_colors + self.__mix_rgb_colors__.argtypes = ( + ctypes.c_ubyte * 3, ctypes.c_ubyte * 3, ctypes.POINTER(ctypes.c_ubyte * 3) + ) + + + def mix_rgb_colors(self, color_a, color_b): + + color_type = ctypes.c_ubyte * 3 + ctypes_color_a = color_type(*tuple(color_a)) + ctypes_color_b = color_type(*tuple(color_b)) + mixed_color = color_type() + self.__mix_rgb_colors__( + ctypes_color_a, ctypes_color_b, ctypes.pointer(mixed_color) + ) + return mixed_color[:] + + +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +# TEST(s) +# +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +def test_mix_rgb_colors(): + + sample = sample_class() + + assert [45, 35, 30] == sample.mix_rgb_colors([10, 20, 40], [80, 50, 20])