Skip to content

Commit b326edf

Browse files
jimmodpgeorge
authored andcommitted
all: Remove MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE.
This commit removes all parts of code associated with the existing MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the -mcache-lookup-bc option to mpy-cross. This feature originally provided a significant performance boost for Unix, but wasn't able to be enabled for MCU targets (due to frozen bytecode), and added significant extra complexity to generating and distributing .mpy files. The equivalent performance gain is now provided by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has been enabled on the unix port in the previous commit). It's hard to provide precise performance numbers, but tests have been run on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V, xtensa) and they all generally agree on the qualitative improvements seen by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE. For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined with MAP_LOOKUP_CACHE is: diff of scores (higher is better) N=2000 M=2000 bccache -> attrmapcache diff diff% (error%) bm_chaos.py 13742.56 -> 13905.67 : +163.11 = +1.187% (+/-3.75%) bm_fannkuch.py 60.13 -> 61.34 : +1.21 = +2.012% (+/-2.11%) bm_fft.py 113083.20 -> 114793.68 : +1710.48 = +1.513% (+/-1.57%) bm_float.py 256552.80 -> 243908.29 : -12644.51 = -4.929% (+/-1.90%) bm_hexiom.py 521.93 -> 625.41 : +103.48 = +19.826% (+/-0.40%) bm_nqueens.py 197544.25 -> 217713.12 : +20168.87 = +10.210% (+/-3.01%) bm_pidigits.py 8072.98 -> 8198.75 : +125.77 = +1.558% (+/-3.22%) misc_aes.py 17283.45 -> 16480.52 : -802.93 = -4.646% (+/-0.82%) misc_mandel.py 99083.99 -> 128939.84 : +29855.85 = +30.132% (+/-5.88%) misc_pystone.py 83860.10 -> 82592.56 : -1267.54 = -1.511% (+/-2.27%) misc_raytrace.py 21490.40 -> 22227.23 : +736.83 = +3.429% (+/-1.88%) This shows that the new optimisations are at least as good as the existing inline-bytecode-caching, and are sometimes much better (because the new ones apply caching to a wider variety of map lookups). The new optimisations can also benefit code generated by the native emitter, because they apply to the runtime rather than the generated code. The improvement for the native emitter when LOAD_ATTR_FAST_PATH and MAP_LOOKUP_CACHE are enabled is (same Linux environment as above): diff of scores (higher is better) N=2000 M=2000 native -> nat-attrmapcache diff diff% (error%) bm_chaos.py 14130.62 -> 15464.68 : +1334.06 = +9.441% (+/-7.11%) bm_fannkuch.py 74.96 -> 76.16 : +1.20 = +1.601% (+/-1.80%) bm_fft.py 166682.99 -> 168221.86 : +1538.87 = +0.923% (+/-4.20%) bm_float.py 233415.23 -> 265524.90 : +32109.67 = +13.756% (+/-2.57%) bm_hexiom.py 628.59 -> 734.17 : +105.58 = +16.796% (+/-1.39%) bm_nqueens.py 225418.44 -> 232926.45 : +7508.01 = +3.331% (+/-3.10%) bm_pidigits.py 6322.00 -> 6379.52 : +57.52 = +0.910% (+/-5.62%) misc_aes.py 20670.10 -> 27223.18 : +6553.08 = +31.703% (+/-1.56%) misc_mandel.py 138221.11 -> 152014.01 : +13792.90 = +9.979% (+/-2.46%) misc_pystone.py 85032.14 -> 105681.44 : +20649.30 = +24.284% (+/-2.25%) misc_raytrace.py 19800.01 -> 23350.73 : +3550.72 = +17.933% (+/-2.79%) In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options: - are simpler; - take less code size; - are faster (generally); - work with code generated by the native emitter; - can be used on embedded targets with a small and constant RAM overhead; - allow the same .mpy bytecode to run on all targets. See micropython#7680 for further discussion. And see also micropython#7653 for a discussion about simplifying mpy-cross options. Signed-off-by: Jim Mussared <[email protected]>
1 parent 60c6d55 commit b326edf

32 files changed

+42
-260
lines changed

docs/reference/mpyfiles.rst

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,6 @@ If importing an .mpy file fails then try the following:
6666
print('mpy flags:', end='')
6767
if arch:
6868
print(' -march=' + arch, end='')
69-
if sys_mpy & 0x100:
70-
print(' -mcache-lookup-bc', end='')
7169
if not sys_mpy & 0x200:
7270
print(' -mno-unicode', end='')
7371
print()

examples/embedding/mpconfigport_minimal.h

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,6 @@
4545
#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_NONE)
4646
#define MICROPY_STREAMS_NON_BLOCK (0)
4747
#define MICROPY_OPT_COMPUTED_GOTO (0)
48-
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
4948
#define MICROPY_CAN_OVERRIDE_BUILTINS (0)
5049
#define MICROPY_BUILTIN_METHOD_CHECK_SELF_ARG (0)
5150
#define MICROPY_CPYTHON_COMPAT (0)

mpy-cross/README.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,7 @@ by the target MicroPython runtime (eg onto a pyboard's filesystem), and then
1717
imported like any other Python module using `import foo`.
1818

1919
Different target runtimes may require a different format of the compiled
20-
bytecode, and such options can be passed to the cross compiler. For example,
21-
the unix port of MicroPython requires the following:
22-
23-
$ ./mpy-cross -mcache-lookup-bc foo.py
20+
bytecode, and such options can be passed to the cross compiler.
2421

2522
If the Python code contains `@native` or `@viper` annotations, then you must
2623
specify `-march` to match the target architecture.

mpy-cross/main.c

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,6 @@ STATIC int usage(char **argv) {
108108
"Target specific options:\n"
109109
"-msmall-int-bits=number : set the maximum bits used to encode a small-int\n"
110110
"-mno-unicode : don't support unicode in compiled strings\n"
111-
"-mcache-lookup-bc : cache map lookups in the bytecode\n"
112111
"-march=<arch> : set architecture for native emitter; x86, x64, armv6, armv7m, armv7em, armv7emsp, armv7emdp, xtensa, xtensawin\n"
113112
"\n"
114113
"Implementation specific options:\n", argv[0]
@@ -205,7 +204,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
205204

206205
// set default compiler configuration
207206
mp_dynamic_compiler.small_int_bits = 31;
208-
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 0;
209207
mp_dynamic_compiler.py_builtins_str_unicode = 1;
210208
#if defined(__i386__)
211209
mp_dynamic_compiler.native_arch = MP_NATIVE_ARCH_X86;
@@ -264,10 +262,6 @@ MP_NOINLINE int main_(int argc, char **argv) {
264262
return usage(argv);
265263
}
266264
// TODO check that small_int_bits is within range of host's capabilities
267-
} else if (strcmp(argv[a], "-mno-cache-lookup-bc") == 0) {
268-
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 0;
269-
} else if (strcmp(argv[a], "-mcache-lookup-bc") == 0) {
270-
mp_dynamic_compiler.opt_cache_map_lookup_in_bytecode = 1;
271265
} else if (strcmp(argv[a], "-mno-unicode") == 0) {
272266
mp_dynamic_compiler.py_builtins_str_unicode = 0;
273267
} else if (strcmp(argv[a], "-municode") == 0) {

mpy-cross/mpconfigport.h

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,6 @@
5757
#define MICROPY_COMP_TRIPLE_TUPLE_ASSIGN (1)
5858
#define MICROPY_COMP_RETURN_IF_EXPR (1)
5959

60-
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
61-
6260
#define MICROPY_READER_POSIX (1)
6361
#define MICROPY_ENABLE_RUNTIME (0)
6462
#define MICROPY_ENABLE_GC (1)

ports/cc3200/mpconfigport.h

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,6 @@
5353
#define MICROPY_LONGINT_IMPL (MICROPY_LONGINT_IMPL_MPZ)
5454
#define MICROPY_FLOAT_IMPL (MICROPY_FLOAT_IMPL_NONE)
5555
#define MICROPY_OPT_COMPUTED_GOTO (0)
56-
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
5756
#define MICROPY_READER_VFS (1)
5857
#ifndef DEBUG // we need ram on the launchxl while debugging
5958
#define MICROPY_CPYTHON_COMPAT (1)

ports/nrf/mpconfigport.h

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,6 @@
6666
#endif
6767

6868
#define MICROPY_OPT_COMPUTED_GOTO (0)
69-
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
7069
#define MICROPY_OPT_MPZ_BITWISE (0)
7170

7271
// fatfs configuration used in ffconf.h

ports/stm32/mpconfigport.h

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,6 @@
5959
#ifndef MICROPY_OPT_COMPUTED_GOTO
6060
#define MICROPY_OPT_COMPUTED_GOTO (1)
6161
#endif
62-
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (0)
6362
#ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH
6463
#define MICROPY_OPT_LOAD_ATTR_FAST_PATH (1)
6564
#endif

ports/unix/Makefile

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -269,7 +269,6 @@ ifneq ($(FROZEN_MANIFEST)$(FROZEN_MPY_DIR),)
269269
CFLAGS += -DMICROPY_QSTR_EXTRA_POOL=mp_qstr_frozen_const_pool
270270
CFLAGS += -DMICROPY_MODULE_FROZEN_MPY
271271
CFLAGS += -DMPZ_DIG_SIZE=16 # force 16 bits to work on both 32 and 64 bit archs
272-
MPY_CROSS_FLAGS += -mcache-lookup-bc
273272
endif
274273

275274
ifneq ($(FROZEN_MANIFEST)$(FROZEN_DIR),)
@@ -285,9 +284,7 @@ endif
285284
CXXFLAGS += $(filter-out -Wmissing-prototypes -Wold-style-definition -std=gnu99,$(CFLAGS) $(CXXFLAGS_MOD))
286285

287286
ifeq ($(MICROPY_FORCE_32BIT),1)
288-
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-mcache-lookup-bc -march=x86'
289-
else
290-
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-mcache-lookup-bc'
287+
RUN_TESTS_MPY_CROSS_FLAGS = --mpy-cross-flags='-march=x86'
291288
endif
292289

293290
ifeq ($(CROSS_COMPILE),arm-linux-gnueabi-)

ports/unix/mpconfigport.h

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -79,9 +79,6 @@
7979
#endif
8080
#define MICROPY_STREAMS_POSIX_API (1)
8181
#define MICROPY_OPT_COMPUTED_GOTO (1)
82-
#ifndef MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE
83-
#define MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE (1)
84-
#endif
8582
#ifndef MICROPY_OPT_LOAD_ATTR_FAST_PATH
8683
#define MICROPY_OPT_LOAD_ATTR_FAST_PATH (1)
8784
#endif

0 commit comments

Comments
 (0)