Skip to content

Commit

Permalink
Handle decimal serialization internally
Browse files Browse the repository at this point in the history
With this change, user applications does not need to depend on simplejson.

The precision of the decimal is lost during serialization. There are
several reasons for this choice:
- first, an exact level of precision is unnecessary;
- second, modern versions of python json are more protective, and the
  2 hacks found in
  http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
  does not work anymore.

Additionnal info can be found at:
- http://bugs.python.org/issue16535;
- http://bugs.python.org/issue18264.
  • Loading branch information
gberaudo committed Apr 28, 2016
1 parent 8636040 commit a767081
Show file tree
Hide file tree
Showing 4 changed files with 11 additions and 14 deletions.
1 change: 0 additions & 1 deletion dev_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,4 @@ coverage
WebTest
psycopg2
pyramid_handlers
simplejson
flake8
4 changes: 0 additions & 4 deletions docs/geojson_renderer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,6 @@ implement the Python Geo Interface.

Notes:

* The GeoJSON renderer requires simplejson 2.1 or higher. Indeed, to be able to
deal with ``decimal.Decimal`` values, which are common when using SQLAlchemy,
we set ``use_decimal`` to ``True`` when calling the ``dumps`` function, and
only simplejson 2.1 and higher support that argument.
* The GeoJSON renderer supports `JSONP <http://en.wikipedia.org/wiki/JSONP>`_.
The renderer indeed checks if there's a ``callback`` parameter in the query
string, and if there's one it wraps the response in a JavaScript call and
Expand Down
10 changes: 4 additions & 6 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ Protocol defines a HTTP interface for creating, reading, updating, and deleting
Papyrus includes lower-level objects, like the GeoJSON renderer, that may come
in handy for your apps even if you don't need or want MapFish views.

Papyrus GeoJSON encoder converts decimal types to float numbers, which
may incur an infinitesimal loss of precision.

Installing
----------

Expand All @@ -26,11 +29,6 @@ Papyrus can be installed with ``easy_install`` or ``pip``::

(Installing Papyrus in an isolated ``virtualenv`` is recommended.)

You may want to install ``simplejson`` as well. This dependency is not required
but it provides more functionnalities than the ``json`` module from the
standard library like the possibility to parse float numbers as Decimal.
``papyrus`` can use such options if ``simplejson`` is present.

Most of the time you'll want to make Papyrus a dependency of your Pyramid
application. For that add ``papyrus`` to the ``install_requires`` list defined
in the Pyramid application ``setup.py``. Example::
Expand Down Expand Up @@ -66,7 +64,7 @@ Papyrus includes unit tests. Most of the time patches should include new tests.

To run the Papyrus tests, in addition to Papyrus and its dependencies the
following packages need to be installed: ``nose``, ``mock``, ``psycopg2``,
``simplejson``, ``pyramid_handlers``, ``coverage``, and ``WebTest``.
``pyramid_handlers``, ``coverage``, and ``WebTest``.

For these packages to install correctly, you have to have header files for
``PostgreSQL``, ``Python``, and ``GEOS``. On Debian-based systems install the
Expand Down
10 changes: 7 additions & 3 deletions papyrus/geojsonencoder.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import datetime
import decimal
import functools

from sqlalchemy.ext.associationproxy import _AssociationList
Expand All @@ -9,7 +10,7 @@

class GeoJSONEncoder(PyGFPEncoder):
# SQLAlchemy's Reflecting Tables mechanism uses decimal.Decimal
# for numeric columns and datetime.date for dates. simplejson
# for numeric columns and datetime.date for dates. Python json
# doesn't deal with these types. This class provides a simple
# encoder to deal with objects of these types.

Expand All @@ -18,11 +19,14 @@ def default(self, obj):
return obj.isoformat()
if isinstance(obj, _AssociationList):
return list(obj)
if isinstance(obj, decimal.Decimal):
# The decimal is converted to a lossy float
return float(obj)
return PyGFPEncoder.default(self, obj)


dumps = functools.partial(_dumps, cls=GeoJSONEncoder, use_decimal=True)
dumps = functools.partial(_dumps, cls=GeoJSONEncoder)
"""
A partial function for ``geojson.dumps`` that sets ``cls`` to
:class:`GeoJSONEncoder` and ``use_decimal`` to ``True``.
:class:`GeoJSONEncoder`.
"""

0 comments on commit a767081

Please sign in to comment.