Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

READY FOR REVIEW - Fix #171. Pass newbytes's native value to its parent constructor. #173

Closed
wants to merge 2 commits into from
Closed

READY FOR REVIEW - Fix #171. Pass newbytes's native value to its parent constructor. #173

wants to merge 2 commits into from

Conversation

posita
Copy link
Contributor

@posita posita commented Oct 1, 2015

Fix #171. Pass newbytes's native value to its parent constructor.

CAVEAT: without monkey-patching Python 2's native str's constructor, I do not know how to handle this case:

Python 2.7.10 (default, Sep 24 2015, 10:13:45)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> nativebytes = bytes ; nativestr = str ; from builtins import *
>>> nativebytes(bytes(b'asdf'))
"b'asdf'" # Whoops!
>>> # This means you can't pass newbytes in many contexts, such as:
>>> from urllib import urlencode
>>> urlencode({ bytes(b'a'): 1, bytes(b'b'): 2 })
'b%27a%27=1&b%27b%27=2'
>>> # :o(

But this works now:

Python 2.7.10 (default, Sep 24 2015, 10:13:45)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
>>> nativebytes = bytes ; nativestr = str ; from builtins import *
>>> bytes(str(u'asdf'), u'utf_8')
b'asdf'

Some things just won't work (and are not addressed by this PR). Consider:

Python 2.7.10 (default, Sep 24 2015, 10:13:45)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> nativebytes = bytes ; nativestr = str ; from builtins import *
>>> from urllib import urlencode
>>> urlencode({ bytes(b'a'): 1, bytes(b'b'): 2 }) # the bytes get double-encoded ...
'b%27a%27=1&b%27b%27=2'
>>> urlencode({ str(b'a'): 1, str(b'b'): 2 }) # ... but the strs do okay ...
'a=1&b=2'
>>> str(b'a') # ... because of this
'a'
>>> urlencode({ u'asdf': u'1' }) # this works though
'asdf=1'
>>> urlencode({ bytes(str(b'a')): 1, bytes(str(b'b')): 2 }) # as expected
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../python-future/src/future/types/newbytes.py", line 85, in __new__
    raise TypeError('unicode string argument without an encoding')
TypeError: unicode string argument without an encoding
>>> urlencode({ str(bytes(b'a')): 1, str(bytes(b'b')): 2 }) # weird, but same as Py 3
'b%27b%27=2&b%27a%27=1'
>>> str(bytes(b'a')) # this explains the immediately previous output
"b'a'"

Now consider:

Python 3.4.3 (default, Sep  3 2015, 05:14:09)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlencode
>>> urlencode({ bytes(b'a'): 1, bytes(b'b'): 2 }) # now the bytes work ...
'b=2&a=1'
>>> urlencode({ str(b'a'): 1, str(b'b'): 2 }) # ... but the strs don't ...
'b%27b%27=2&b%27a%27=1'
>>> str(b'a') # ... because of this
"b'a'"
>>> urlencode({ u'asdf': u'1' }) # this works though
'asdf=1'
>>> urlencode({ bytes(str(b'a')): 1, bytes(str(b'b')): 2 }) # yup
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: string argument without an encoding
>>> urlencode({ str(bytes(b'a')): 1, str(bytes(b'b')): 2 }) # same as in Py 2
'b%27b%27=2&b%27a%27=1'
>>> str(bytes(b'a')) # same as in Py 2
"b'a'"

The way to fix one of those inconsistencies is to change newstr's constructor to treat Python 2's native strs as bytes, which probably means more checking at future/types/newstr.py, line 95, but this might break other things.

@posita
Copy link
Contributor Author

posita commented Oct 1, 2015

By the way, I'm not sure if anyone cares, but it looks like there are some debug-by-print(f) statements being output when running tests:

% python setup.py \test -q
running test
running egg_info
writing src/future.egg-info/PKG-INFO
writing top-level names to src/future.egg-info/top_level.txt
writing dependency_links to src/future.egg-info/dependency_links.txt
writing entry points to src/future.egg-info/entry_points.txt
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.au' under directory 'tests'
warning: no files found matching '*.gif' under directory 'tests'
warning: no files found matching '*.txt' under directory 'tests'
writing manifest file 'src/future.egg-info/SOURCES.txt'
running build_ext
............x....x.......x.....................................s..........x.........xxx.....x......E.s..x......x........sAdding an item
..x....x..................................xx....xx.x...........x......x..........x.x..xx.xsss.....s...x.x.x..x.......s..x....x...x.....x.....x.RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: ws_comma
RefactoringTool: No files need to be modified.
..............................................................................................................................................s................................x.......x.x....object
xx.......x....................................................x...x.....<super: <class 'Singleton'>, <Singleton object>>
............x....xxRefactoringTool: No files need to be modified.
..x...x.......................Creating /var/folders/vx/pghvp8k129s2xwlx3g43t6ym0000gn/T/tmpraUC_F/test_imports_future_stdlib.py ...
Requests doesn't seem to be available. Skipping requests test ...
.Creating /var/folders/vx/pghvp8k129s2xwlx3g43t6ym0000gn/T/tmpr5nNmx/test_imports_future_stdlib.py ...
Requests doesn't seem to be available. Skipping requests test ...
...........x..x./.../lib/python2.7/tempfile.pyc
/.../lib/python2.7/copy.pyc
/.../lib/python2.7/textwrap.pyc
....Import succeeded!

.......x...x..s.................x.....s............................s..xxsss.........x...x.x...x.......s......................................................................................................................................s.......................................................s.xxxxsxxs......xx......s...s.........................................................................................................................s...........................Succeeded!
.Succeeded!
.Succeeded!
.Succeeded!
.Succeeded!
.Succeeded!
.Hello from a Python 2-style print statement!
Succeeded!
.Succeeded!
.
======================================================================
ERROR: test_open_default_encoding (test_future.test_builtins.BuiltinTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/.../python-future/tests/test_future/test_builtins.py", line 1217, in test_open_default_encoding
    self.write_testfile()
  File "/.../python-future/tests/test_future/test_builtins.py", line 1182, in write_testfile
    fp = open(TESTFN, 'w')
LookupError: unknown encoding:

----------------------------------------------------------------------
Ran 1075 tests in 45.515s

FAILED (errors=1, skipped=23, expected failures=65)

Regarding that failed test, I did not see that on Travis, but I do see it locally when running on OS X. It was present both before and after my changes.

@edschofield edschofield mentioned this pull request May 22, 2016
Open
@posita
Copy link
Contributor Author

posita commented Jun 14, 2016 •
edited
Loading

Closing this in light of #171 (comment) et seq.

@posita posita closed this Jun 14, 2016
@posita posita deleted the posita/171-fix-native-from-bytes branch June 14, 2016 20:48
@pjw91 pjw91 mentioned this pull request Mar 2, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@posita