Graphite Early Feedback

A successful software package must address the needs of its users. To ensure that potential needs have not been missed, we solicited feedback on the early design of Graphite. A summary and analysis is available.

The following message (or minor variations thereof) was sent to the Python plot-SIG mailing list, the comp.lang.python newsgroup, and a handful of individuals previously identified as interested in Python plotting packages.

I'm part of a small team working on a new package called Graphite.  It's an
open-source plotting package written entirely in Python, designed primarily
for the following requirements:

1. cross-platform and cross-media (i.e., native onscrene graphics for
Windows/X-Windows/MacOS, and output to a variety of formats including
print-quality PostScript and PDF)

2. fully portable; no reliance on C libraries with limited portability

3. professional-quality graphing power (e.g., similar to Matlab)

We're still in the design/prototyping phase, and we'd like to get your
input.  We know what our needs are, but we may not understand your needs --
we're relying on you to tell us.

If you're willing, please go to the Graphite web site:

And check out the Vision Statement and Requirements documents.  The others
provide additional design details if you're interested.  Do the
requirements we've outlined meet your needs?  Have we omitted any major
functionality?  Do you forsee problems with the design?  In short, does
this look like a tool you will use, and if not, why not?  Of course no
package can be all things to all people, and there are some excellent
existing graphing packages that work well on some systems.  But your
particular reasons for preferring one over another may be informative.

Thanks very much for whatever input you're able to give.  We'll do our best
to reward your investment by producing a great plotting package we all can

Best regards,
-- Joe Strout

|    Joseph J. Strout           Biocomputing -- The Salk Institute |
|                 |

Early responses are reproduced below. Return addresses and names have been removed to protect privacy.

Wow, this sounds really good! I am very much interested in the kind of
package you are developing. I come from a MATLAB background and it
certainly sounds like you are planning on emulating the majority of
its features.

I generate many 3D plots so your decision to incorporate 3D functionality
from the outset is a good one (for me). One problem I had with 3D plots
in other packages (e.g., PLPlot) is that they would not encode heights
above the plane using a user-defined colormap. This is essential for
good visualization.

The output format issue is going to be sticky as you're going to have
many different formats requested. I, for example, use FrameMaker which
works best with encapsulated PostScript with embedded TIFF preview.
This can probably wait for a while but the more output formats you can
support, the greater the range of users you'll support.

Another thought on 3D...make sure you support integer-valued (and even
byte-valued) input arrays WITHOUT converting them to floats first! These
arrays can be really huge and doing a MATLAB mesh plot is frequently
a pain because of the huge temporary memory usage.

Have you thought about how you're going to handle fonts for placing
text on figures? I imagine that this might require some platform-dependent
handling (I don't really know if Tk abstracts this out, as I haven't
had a chance to use it yet).

Thanks for soliciting my feedback. I'd be pleased to help out with any
more info that would be of use.

I'm encouraged by your Graphite project. Although I haven't had time to look
through all of your docs, it looks like a good start.

One of the things that I'd like to see a graphing library do is follow some
of the precepts that Tufte lays out in "The Visual Display of Quantitative
Information", particularly the materal about Data-Ink Maximization. Are you
familiar with this? Although flexibility in a package like this is good,
giving too much rope for people to hang themselves with may not be; that's
more of a Perl thing to do ;-)  In any case, good defaults for the graph
layouts, etc are key.

Also, one of the bigger things I've been thinking about doing is coming up
with a graphing library that can output anti-aliased GIF files. Any
possibility of that in Graphite?

I keep my thumbs pressed that Graphite would succeed, because I can see a
need for it in the LAMDI project: Open Source Anesthesia software to capture
data gathered in the Operating Room, and visualise independant from the
commercial monitors.

I will follow with interest.

The other area where I could see such a package be used...
well that is a childish dream, but it has been collected as a prayer:

Yes, Yes, Yes, Yes.
Very interested in such a package.  
I have been using Igor Pro, Matlab, IDL, Mathematica and Excel to produce
plots of data for analysis, presentations, and papers.  In some cases I
have had to a combination of all of  them to get everything I wanted.

Glad to hear about PIDDLE/Graphite.  How will this compare to the Snow project.

It looks like it will provide a combination of features that are mixing the
above mentioned packages.

Things I like:
Output formats
Real time viewing
Native backends
Open Source
Interactive plots

Things Missing:
Log-Log plot type

Suggestion.  If you have access to an IDL package.  Take a look at Insight.
 The Gui may be something to model, once you start there. And the idea of a
plug-in system that it uses warrants some merit.
Hopefully, it will be more flexible than Matlab.:-)

If you need any help testing, I use a Macintosh and PC.

This looks fantastic.  You are definitely on the right track this is
exactly where we need to go.  I'm willing to help in anyway I can.
Because I need this functionality.  I'd love to see a GTK back end.  One
thing I'm concerned about is speed-if I can only use the Tk back end will
it be fast enough with a large data set?  Can speed be improved with a
compiled-code back-end?

Testing Konrad Hinsen's TkPlotCanvas with a data set of 1000 points shows
it to be somewhat slower than I would be happy with.

One Plot Type that I use often is the equivalent of imagesc in MATLAB or
the pli() function in the gist module from LLNL.  It produces an "image"
(really a cell-array of patches) with an axes and title.  I couldn't tell
if this kind of graph would be available.   This is a must-have plot type
for me.

Another graph type over the horizon might be a surface plot.  Simliar to a
mesh but with patches and surface shading available.  This is not a
must-have and definitely a later add on.

A final question.  I'm sure you've seen David Ascher's Snow project.  Is
there any collaboration going on there?  The goals seem similar.  I think
your idea to start with 3-D is important and worthwhile, which is a major
difference with SNOW.

Way to go, thanks for a great effort.  I'm on pins and needles waiting...

This would really take Python over the hump....

Thanks for your response.  A number of users may want to build
on-the-fly jpeg plots for web servers.  FYI: There's a potential
conflict between PIL and Zope ( I recall,
because PIL doesn't use the Python package format.

Things I need:

- Interactive use from the Python prompt. This may be difficult to
acheive using Tk since the tkinter mainloop blocks input from the
prompt. I currently use Gnuplot to do my plotting and access it through
pipes (via an abstract interface). Having the plotter running as a
separate process completely avoids blocking problems when using the
plotters from the prompt.

- Multiple overlayed 2D plots in different styles (e.g. point and lines
on the same graph). For this reason, I'd prefer to see the PlotFormats
as attributes of the data class or at least have someway of grouping
this two together.

- Support for animation (even if this is in the form of resending new
data and plot commands).

- Surface plots

- Hidden line removal for 3D plots

Things I'd like:

- Wire meshes for surface plots (Gnuplot, for example seems to insist
that data for 3D plots comes in neatly aligned points along the same
plane). It'd be nice to be able to provide 3D data point in an arbitrary
order and provide arbitrary x and y values and still get a nice looking
surface out.

- Intersections of 3D data plotted in 2D

- Unordered input:

 [(x1,y1), (x2,y2) ..]   x1x2 possible

- Input from mapping objects:
   {x1:y1,  x2:y2,  ...}
   {x1:(y1,y1err),  x2:(y2,y2err),  ...}
   {(x1,y1):z1,  (x2,y2):z2,  ...}

- Image output without the user interface. I'd be happier if you can
(optionally) use the package without even needing to import any UI
modules. This would allow quicker batch production of graphs and allow
it run on platforms without UI support.

This sounds like an excellent project. The thing that's put me off using
many plotting packages in the overcomplicated APIs. You practically have
to learn a new language to make use them. If you can provide a nice easy
to follow interface, I think you'd be onto a winner.

I have looked over your Design Documents on the Web Site.  I heartedly
agree you are on the right track.  If you need a beta tester, please let
me know.

My primary plotting requirements are 2D X-Y scatter plots with lines or
curves fitting the points and 3D  mesh plots.

Things I need:

1. labeling using different fonts, especially the Greek font with super-
and subscripts.

2. contour plots for irregularly spaced data points. Very few publicly
available plotting programs can do that (I've not found a single one). One
way to accomplish this is to first triangulate the data points and then do
inverse interpolation over the triangles to get the coordinates of the
contour lines.

I would like to see mesh/surface plots not be restricted to
regularly-spaced data points.

For engineering plotting, I found the lack of the above features makes
most of the commercial pacakges and all the publicly avaialble ones 
useless to our research use. I'll be happy to help if it is needed.

Thanks for emabarking on a very worthwhile project.

A further thought I had on fonts is that it is highly desirable to mix
fonts (and format information) in a single text string. In MATLAB, they
support a rudimentary TeX-like preprocessor which lets you insert symbol-
font characters in an ordinary plain-text string, and also allows
formatting information like superscript/subscript, italic, etc. This
makes for more professional graphs and I'm hoping you'll allow this level
of flexibility rather than just having a static font attribute for each
string (as it was in the bad old days of MATLAB).

In response to your clp request, I reviewed material posted.  Main lack I
noticed in diagram is any class for symbols.  Error bars (both with top and
bottom bars and mid-bar (as with stock hi/lo/close)).  Did notice reference
to yerr series, which would allow for first type of error bar, but suggest
also yhi,ylo pair as alternative.  This would allow for asymmetrical bars
on log plots (as well as stock/etc applications).

Even more complicated are the boxplots invented by Tukey and used by many
statisticians, where the 'symbol' incorporates info on median,  5,25,75,95
percentiles, and outliers.  Something like:

I  I

Wishing for your success (would definitely use package).
Last Updated: 4/25/99 . . . . . .