Skip to content

Commit

Permalink
Add old reST posts for archival.
Browse files Browse the repository at this point in the history
  • Loading branch information
fperez committed Sep 13, 2012
1 parent cc7ba25 commit 354b5fe
Show file tree
Hide file tree
Showing 7 changed files with 733 additions and 0 deletions.
44 changes: 44 additions & 0 deletions old/090907-cse09-siam-news.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
=============================
CSE'09 article in SIAM News
=============================


This isn't fresh-off-the-oven news, but it's still on the main SIAM News page,
so I'll mention it as I am trying to do a better job of keeping up with this
blog...

As I mentioned on a `previous post`_, our three_ part_ minisymposium_ on Python
at the SIAM `CSE09 conference`_ was fairly well received. But even better,
SIAM asked us to write up an article for publication in the `SIAM News
bulletin`_. I was really pleased with this, as SIAM News reaches a very broad
international audience and is actually read by people. I often find very
interesting material in it, as the publication hits a very good balance of
quality and not being overly specialized (as we all drown in work, we tend to
focus only on very narrow publication lists for everyday reading,
unfortunately).

Randy, Hans-Petter and I drafted up an article, and we received great feedback
from all of the presenters at the minisymposium, including figure contributions
by `John Hunter`_ and `Hank Childs`_ (who was at LLNL at the time, and I'm
delighted to see is just up the hill from me at LBL).

Our article is now available in HTML `online at the SIAM site`_, and I also
have a `PDF version`_ that is more suitable for printing in case you are
interested (note that my version is missing the very last corrections from the
SIAM editor, but the differences should be minor).

.. Links
.. _previous post: http://fdoperez.blogspot.com/2009/03/python-at-siam-cse09-meeting.html
.. _three: http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=8044
.. _part: http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=8045
.. _minisymposium: http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=8046
.. _CSE09 conference: http://www.siam.org/meetings/cse09/

.. _SIAM News bulletin: http://www.siam.org/news/news.php
.. _John Hunter: http://matplotlib.sf.net
.. _Hank Childs: http://www.lbl.gov/cs/CSnews/CSnews073109c.html

.. _online at the SIAM site: http://www.siam.org/news/news.php?id=1595
.. _PDF version: https://cirl.berkeley.edu/fperez/papers/siam_news_py4science.pdf

141 changes: 141 additions & 0 deletions old/091105-guido-visit.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
==============================================
Guido van Rossum at UC Berkeley's Py4Science
==============================================

**Update:** Quick links

* Video_ of the entire session. The talks cover the first 50 minutes, and
Guido's part starts at the 54 minute mark.
* A `page with all the slides`_ on my site.
* Guido also blogged_ his impressions.
* Blog posts by `Jarrod Millman`_ and `Matthew Brett`_.

.. _page with all the slides: http://fperez.org/py4science/2009_guido_ucb
.. _blogged: http://neopythonic.blogspot.com/2009/11/python-in-scientific-world.html
.. _Jarrod Millman: http://jarrodmillman.blogspot.com/2009/11/visit-from-guido-van-rossum.html
.. _Matthew Brett: http://nipyworld.blogspot.com/2009/11/guido-van-rossum-talks-about-python-3.html

On November 4 2009, we had a special session of our informal Py4Science_
seminar where Guido van Rossum visited for an open discussion regarding the
uses of the Python language in scientific research. Guido had expressed his
interest in discussing the work that various scientists do with Python but
mentioned that instead of a formal talk, he would prefer a format that allowed
for more interaction with the audience. We agreed that a good plan would be
for us to present a rapid-fire sequence of very short talks highlighting
multiple projects so that he could get a good "high altitude" view of the
scientific python landscape, leaving then ample time for discussions with the
audience.

.. _Py4Science: https://cirl.berkeley.edu/view/Py4Science

Guido has already posted `his impressions`_ of the visit, and so have my
colleagues `Jarrod Millman`_ and `Matthew Brett`_, so I'll try to provide a
complementary view here without too much repetition.

.. _his impressions: http://neopythonic.blogspot.com/2009/11/python-in-scientific-world.html
.. _Jarrod Millman: http://jarrodmillman.blogspot.com/2009/11/visit-from-guido-van-rossum.html
.. _Matthew Brett: http://nipyworld.blogspot.com/2009/11/guido-van-rossum-talks-about-python-3.html

We gathered for lunch first with a small group and had a very interesting
discussion on various topics; we had a chance to talk in some detail about the
transition to Python 3 for Numpy, something a number of people have started to
think about seriously. Numpy is pretty much a 'root dependency' for so many
scientific projects, that until it makes the jump it will be very difficult for
anyone else in science to seriously consider Python 3. Understandably, Guido
would like to see some movement from our community in this direction, and he
offered useful guidance. In particular, he said that in the core Python-dev
team there might be enough interest that if we ask for help there, we might
find developers willing to pitch in and provide some assistance. He also
expressed some disappointment that `PEP 3118`_, which was accepted with our
interests in mind, still hadn't been fully implemented. Limited manpower is
the simple reason for this situation, but fortunately Jarrod mentioned that
there's a good plan to address this in the near future.

I had a chance to quiz Guido about something I've wondered for a while: Python
has unusually good number types in its core (arbitrary length integers,
extended precision decimals and complex numbers), but the integers divide
either into the integers (the truncating behavior of Python 2.x) or into the
floats (in 3.x). While the 3.x division is an improvement, I would have really
liked to see Python go to native rationals; for one thing, this would make the
Sage 'language' (by which I mean the extensions Sage makes to pure Python)
behave like Python in algorithms involving integers, eliminating a recurring
source of confusion between the two. I also happen to think it would be a
'better' behavior, though there are valid reasons also for someone to expect a
more 'calculator-like' answer to divisions like 1/2, who might be annoyed if
they get 1/2 back instead of 0.5. While obviously such changes will not be on
the table for a long while (and I should say here that I am very happy with the
planned moratorium to core language changes, as I hope that will allow a more
focused effort on the needs of the standard library), it was interesting to
hear Guido's approach to this question as one that could be handled via
overloadable literals rather than a change of integer semantics. I'd never
thought of that, but it's an intriguing idea... Something to think about for
when we start looking at Python 4000 :)

.. _PEP 3118: http://www.python.org/dev/peps/pep-3118

We then headed over to the official presentation, where we managed to cram 14
talks in 50 minutes and then had a full hour of open conversation with Guido,
where the audience asked him questions on a number of topics. You can see a
complete video_ of the entire session: after 50 minutes of talks we have a
transition, and Guido's section starta at the 54 minute mark. On my website I
have posted_ a page with all the slides for these mini-talks.

I presented an overview_ introduction and material on behalf of 4 others who
were not present locally, but coincidentally, `William Stein`_ of Sage_ fame
was on campus to give a talk in the same building almost at the same time, and
he could present the Sage slides directly. `Ondrej Certik`_ from SymPy_ was
able to make the trip from Reno, completing our out-of-town speakers. The
other 7 presentations were from a number of local speakers (from various
departments at UC Berkeley and Lawrence Berkeley National Laboratory, just up
the hill from us).

.. _overview: http://fperez.org/py4science/2009_guido_ucb/fperez_py4science_overview.pdf
.. _video: http://www.archive.org/details/ucb_py4science_2009_11_04_Guido_van_Rossum
.. _posted: http://fperez.org/py4science/2009_guido_ucb/index.html

.. _William Stein: http://wstein.org
.. _sage: http://sagemath.org
.. _Ondrej Certik: http://ondrejcertik.blogspot.com/
.. _sympy: http://code.google.com/p/sympy

I have received very good feedback from several people, and I am really
thankful to all the speakers for being so attentive to the time constraints,
which let us pack a lot of material while leaving ample time for the discussion
with Guido. My intention with this was to really provide Guido with a broad
understanding of how significant Python's penetration has been in scientific
computing, where many different projects from disciplines ranging from computer
science to astronomy are relying heavily on his creation. I wanted both to
thank him for creating and shepherding such a high-quality language for us
scientists, and to establish a good line of communication with him (and
indirectly the core python development group) so that he can understand better
what are some of the use patterns, concerns and questions we may have regarding
the language.

I have the impression that in this we were successful, especially as we had
time after the open presentations for a more detailed discussion of how we use
and develop our tools. Most of us in scientific computing end up spending an
enormous amount of time with open interpreter sessions, typically IPython_ ones
(I started the project in the first place because I wanted a *very good*
interactive environment, beyond Python's default one), and in this work mode
the key source of understanding for code are good docstrings. This is an area
where I've always been unhappy about the standard library, whose docstrings are
typically not very good (and often they are non-existent). We showed Guido the
fabulous Numpy/Scipy `docstring editor`_ by Pauli Virtanen and Emmanuelle
Gouillart, as well as the fact that Numpy has an actual `docstring standard`_
that is easy to read yet fairly complete. I hope that this may lead in the
future to an increase in the quality of the Python docstrings, and perhaps even
to the adoption of a more detailed docstring standard as part of `PEP 8`_,
which I think would be very beneficial to the community at large.

In the end, putting all this together took me a lot more time than I'd
originally planned (I think I've had this same problem before...), but I am
very pleased with the results. Python has become a central tool for the work
many of us do, and I am really happy to establish a good dialogue with Guido
(and hopefully other core developers), which I'm sure will have benefits in
both directions.

.. _IPython: http://ipython.scipy.org
.. _docstring editor: http://docs.scipy.org/numpy/Front%20Page/
.. _docstring standard: http://projects.scipy.org/numpy/wiki/CodingStyleGuidelines#docstring-standard
.. _pep 8: http://www.python.org/dev/peps/pep-0008

71 changes: 71 additions & 0 deletions old/091215-claremont-py4science.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
=====================
Py4science teaching
=====================

It's pretty clear to me that Python is rapidly growing in acceptance as a
computational platform at universities everywhere. I recently heard from `Josh
Bloom`_ at UC Berkeley astronomy that his proposal for a short 'boot camp'
course at the beginning of the Fall semester was approved. This is excellent
news, last year I taught `something similar`_ for neuroscience students and
postdocs, and I'm glad to see the campus adopting python further as a key
component of the computational training the science students receive.

.. _Josh Bloom: http://astro.berkeley.edu/~jbloom/
.. _something similar: http://fperez.org/py4science/workshop_berkeley_2008.html



John Hunter and I just completed a few days ago teaching another such workshop
at the Claremont Colleges, supported by an NSF grant that `John Milton`_,
(J. Hunter's PhD advisor) has for exposing undergraduates to a number of
research-related experiences. This grant supports summer research internships
where two undergrads visit together a research lab elsewhere to work
independently on a project, which has already resulted in some really neat
projects, as well as our teaching of scientific python tools (we were also
there in 2008 and hopefully will continue next year).

.. _John Milton: http://faculty.jsd.claremont.edu/jmilton/

I have to say that I really enjoy teaching this type of workshop, especially
now that the core tools are fairly mature, installation isn't the problem it
used to be, and we have a chance of presenting interesting examples to the
students from the very beginning. John Hunter is a phenomenal lecturer, with a
real knack for illustrating interesting concepts on the fly, in a very natural
manner that is honestly similar to how exploratory work is *actually* done in
real research, where you run a bit of code, plot some results, print a few
things, code some more, etc. In the picture above, he was working through some
of the `matplotlib image tutorial
<http://matplotlib.sourceforge.net/users/image_tutorial.html>`_, which gave us
an opportunity to find and fix a small bug in how the luminance histogram was
being computed (the current form is correct). Every one of these students
probably has a digital camera these days (if nothing else, in a cell phone), so
an example like this is a great way to connect something they are used to with
mathematical and programming concepts.


This one is from a great illustration of random numbers and simple statistics.
John built up interactively a simulation of random walks, working up from a
single (one-dimensional) walker to a group of them, comparing the simulation
results with the analytical predictions for various quantities (like mean and
variance), and also explaining to the students how these squiggly lines could
be considered a model of price fluctuations over time. In this manner, he
connected the somewhat abstract statistical concepts to something the students
could relate to, namely the risk of an investment making a profit or a loss
after a certain period.

We also talked about FFTs, dynamical systems, error analysis, data formats, and
a few other things. It was very encouraging to have most of the students
return on the second day, considering how this was a completely optional
activity for them, covering an entire weekend (with morning lectures both days)
right before they had to start diving into their finals. But they were a smart
and enthusiastic bunch, and I hope that the workshop gave them some useful
starting points they can then develop on their own as they get involved in
research projects.

These are just two examples of how we are now seeing Python's acceptance in
university computing growing. We have a lot of work still ahead of us, but
it's really encouraging to see how far we've come from the days when building
scipy was a black art, IPython was little more than a different prompt for the
python shell and matplotlib could only do basic line plots on top of libgd. We
now have tools that provide a complete computational environment for teaching
and research, and things are only getting better...
36 changes: 36 additions & 0 deletions old/091215-vision-hpc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@


http://www.abhishek-tiwari.com/2009/08/visual-high-performance-computing-in.html

From jose, re the python bof at SC 2009

The BoF was very well attended -- standing room only. There were probably about
100 people in the room. Spotz from Sandia Labs was there and it was a very
interesting presentation regarding NVIDIA "copperhead" python-to-GPU
programming. It's still very early in development, but looks very promising.

No word on a corresponding workshop, though. The SC09 committee surveys the
attendance of each of these BoF's and uses that as input for subsequent related
events -- tutorials, workshops, etc. There is a lot of interest and some of the
topics you brought up with your meeting with Guido surfaced -- especially GIL.

Anyway, given the strength of the attendance, hopefully this will result in
subsequent Python for HPC material at the next conference, which is in New
Orleans.

Just parenthetically, I also attended the high-level languages series, which
included Matlab. Outside of the ability to distribute an array on multiple
nodes, I really didn't see anything that was light years ahead of what you all
have already implemented in IPython. In fact, during the question session, some
professor brought up the fact that it was too expensive for his institution to
continue to teach parallel programming using Matlab. Obviously, they weren't
too happy to discuss that publicly.


###
http://www.paratools.com/ptoolsrte.php

The Centos build (which includes associated binaries) has IPython in it.

Hopefully, ptoolsrte will become part of the standard software installation for HPCMP and will thereby have IPython in it by default. That would be huge (~6000 DoD users ) at four major computational centers around the country would have this.

Loading

0 comments on commit 354b5fe

Please sign in to comment.