Skip to content

Latest commit

 

History

History
788 lines (690 loc) · 26.7 KB

TODO

File metadata and controls

788 lines (690 loc) · 26.7 KB

MathMap TODO

-*- org -*-

1.3.6

GIMP Plug-In

TODO Register example filters with proper plug-in arguments in GIMP

That would make them work with GAP, Script-Fu and other stuff.

TODO MathMap crashes if a drawable is selected that was already closed in GIMP

TODO Make first input image be selectable, too

TODO Put icons into tree views

http://arstechnica.com/open-source/guides/2009/08/howto-an-introduction-to-gtk-treeviews-and-autocompletion.ars

TODO Sharing of filters via CouchDB

General

TODO Rework the whole coordinate-transformation thing

Use a -1 to 1 coordinate system natively, but for square pixels. For non-square images, make the shorter edge go from -1 to 1, the longer edge will go beyond.

That way, we can get completely rid of the whole resizing stuff. Although, maybe we’ll want an image attribute which makes the longer edge be -1 to 1, which again would require resizing. Leave it out at first, though.

The only way in which a filter’s xy would now depend on the size of the canvas or its image arguments would be for pixel filters, for which xy would be xy*max(pixel_width*pixel_height)

Language

TODO sqrt for complex

Compiler

TODO The pixel-size simplifier must be updated

Simply use the canvas size. The “compositing with opacity” filters should then simplify to only fetch two pixels and should be just as fast as the native versions.

DONE Top-level filters taking images should never use the pixel-size internals

Instead they should bind IMAGE_PIXEL_WIDTH/HEIGHT(first_image) to __canvasPixelW/H. Only if a filter doesn’t take an input image should it use those builtins directly. That will allow the compiler to further simplify more code, like the “compositing with opacity” filters.

Wrong, we must always use the canvas pixel-size internals, because our brain-dead coordinate system stretches pixels so that the canvas always goes from -1 to 1 in both directions. If we use a different pixel-size for a closure, its pixels are stretched differently which screws things up.

DONE Add pixel-size arguments to closure

That simplifies cc.c and llvm.c (they both with deal with the pixel-size issue separately), and it makes the simplifier trivial.

Wrong, see above.

TODO Loop-invariant code motion does not honor non-pure ops

TODO Transform as many optimizations to use the simplifier

TODO Simplify coordinate stuff (non-stretched ident filter)

TODO don’t produce functions for filters which have been optimized away

DONE We need something to handle loop-invariant conditional closures

Obsoleted by *caching of native filter results.

Consider this code:

while ... do
  cond = ...;
  if cond then
    img = gaussian_blur(abc)
  else
    img = gaussian_blur(def)
  end;
  p = img(xy)
end

If cond is not loop-invariant then we cannot move the if statement out of the loop, and neither of the invocations of gaussian_blur, which means that for every pixel we will render either one or the other.

We need a lazy evaluation operation that makes sure that the rendering happens only once. Something that transforms the above code to:

img_true = lazy(gaussian_blur(abc));
img_false = lazy(gaussian_blur(def));
while ... do
  cond = ...;
  if ... then
    img = force(img_true)
  else
    img = force(img_false)
  end;
  p = img(xy)
end

The lazy operation would create a kind of closure which force would invoke at most once and then store and reuse the result. Of course lazy would only be used for expensive operations.

DONE Generator for simplifiers

DONE optimize IMAGE_PIXEL_WIDTH/HEIGHT(closure)

TODO passing of colors, curves and gradients to filters

Testing

TODO missing test cases

For most of the missing ones it’s required that compositions work on the command-line.

Native filters

TODO We’re superfluously copying floatmaps in Gauss

When Gauss isn’t given a floatmap, it renders it into a new floatmap, and then that floatmap is copied again.

Command Line Tool

TODO Make it work with multiple threads

TODO Compositions should work, too

TODO Report error positions

TODO Colors as definable user values

Packaging

TODO Check whether tags are pushed to github

1.3.x

Windows

TODO why does syntax highlighting not work completely in WinGimp?

It’s very confusing and is an issue with both WinGimp as well as the self-compiled GIMP in MSys. If the self-compiled GIMP is started from the MathMap source directory, it works. If it (or WinGimp) is started from the WinGimp’s ~/.gimp-2.6/plug-ins directory (with complete MathMap installation), it doesn’t work. If either one is started within MSys from /bin, they work. If /bin is copied to /bim, neither one works there.

TODO Why is random blur so slow?

GIMP Plug-In

TODO Use GtkInfoBar to report error messages?

TODO Implement our own curve widget. Use curve stuff from GEGL

TODO We have a one-pixel offset between fast and slow preview again

Just try ident filter and turn fast preview on and off.

TODO speed up rendering of GIMP drawables by processing tile for tile

TODO make preview input drawables to be floatmaps

TODO UI via GtkBuilder/Glade

General

TODO tree_vector_node_t shouldn’t be a union - don’t alloc a pointer array for the leaves

TODO move everything from invocation to frame that belongs there

TODO pass through frame and t everwhere

TODO use -O3

TODO do we need uservals, image size in invocation when we have them in the closure?

Language

TODO Voronoi cell function which gives distance to center

The function should return at least two numbers, namely the coordinates of the center of the cell that was queried. Ideally it would also return a number “identifying” the cell, so that one could for example give it a color manually via a gradient. The identifier should probably be between 0 and 1.

A better alternative to the identifier might be to get something equivalent via a solid noise function, though.

The coordinates should probably be three-dimensional.

TODO builtin == for tuples of length >1

we’ll need additional features for that in builtins.lisp

TODO builtin for getting size of an image

TODO real gamma function buggy

TODO check out LCH color space

http://github.com/schani/mathmap/issues/#issue/3

Compiler

TODO We put loops into const code even if they’re not necessary

For example, Gaussian Blur->Random Blur without loop invariant code motion.

TODO number*image must trigger an error!

This means that the compiler will have to know about types other than tuples.

TODO Make exprtree and scanner use compiler pools as well

TODO calculate R in the filter code

TODO replace resize with a simple internal filter

TODO don’t use x_consts and y_consts structs in filter_func

TODO make two different filter_XXX functions

one for calling in closure, one without closure, so as not to require building the closure

TODO make solve ops and ell-jac foldable, also tuple-nth, tree-vector-nth

requires pools in the folders

TODO make internals typed

TODO short circuit execution for && etc in compiler

LLVM

TODO implement mathfuncs

TODO pass t properly (test with recursive filters)

TODO remove pools in root closure image once mathfuncs work

TODO implement comparison ops with float args

we only have comparison ops with float args, so we always promote ints, even in loops:

filter loop ()
  g = 0;
  for i = 0 .. 10 do
    g = g + (x + y)/30
  end;
  grayColor(g)
end

Packaging

TODO Generate still and animation examples for homepage automatically

TODO Email address of Andy Thomas in contributor credits

TODO Generate keyword list in sourceview spec file from builtins.lisp

TODO rewrite HACKING and include it in distro

TODO ship documentation in package

When the user presses “Help”, those files should be used, unless they don’t exist.

Testing

TODO make random function testable

Rendering

TODO strip resize images manually

We don’t strip resize, so we take the slow rendering path. This would be obsolete if we used a filter for resizing.

TODO specifiable size

TODO antialiasing from rendered images

TODO do rendering multi-threaded if possible

Designer

TODO save current user values as defaults in the design

TODO give better error reports when loading a design fails

TODO do migration with save/load

TODO free widget_data in designs

TODO akklimatisierungsbug (bertl)

When loading a composition and then switching to the composer view, the edges are wrong first and then jump into the correct configuration.

TODO when bigger fonts are used the nodes look funny

Filters

DONE sharp blur

TODO math filters with constants for composer

TODO color gimp layer mode

Native filters

TODO Images in the native filter cache are never freed

Use some form of reference counting.

TODO figure out how much faster 2^n ffts are

TODO do gaussian multithreaded if possible

TODO see if there are faster gauss implementations available (some intel stuff?)

TODO take aspect ratio of kernel into consideration - don’t stretch it

TODO normalize argument for convolve should not be bool but be the value to normalize to

TODO image segmentation

TODO edge detection

Command Line Tool

TODO Make input drawables be floatmaps

1.3.y

Designer

TODO catch and forbid recursive compositions

GIMP Plug-In

TODO auto-completion

TODO Histogram

TODO ask to save before quitting or selecting another expression or composition

TODO Different fast image source size per drawable

Would be trivial if fast image sources were floatmaps.

TODO Adapt fast image source size to preview size

TODO zoom in preview

Do we really need this?

TODO Get preview background gray from GtkStyle

TODO figure out how to pass user values to gimp so that Filters->Repeat works properly

We can do this by encoding them in a string.

Language

TODO provide syntactic sugar for returning images instead of colors

TODO indirectly recursive filters

TODO Hypercomplex-Complex multiplication

TODO XYZ, Lab color space support

TODO filters as first class values

General

TODO Logarithmic sliders

TODO Debug tuples

TODO Remove i18n?

Rendering

TODO lazy rendering

Packaging

TODO Real dependencies

Compiler

TODO check why lots of ops (like RED_FLOAT) are not foldable

TODO pass colors as tuples

TODO more arithmetic optimizations (like a*c1+a*c2 -> a*(c1+c2))

Implement simplifier generator first.

TODO catch duplicate filter names

TODO filter call argument checking

TODO optimization for filters which use the “current” pixel

TODO Remove dead whiles

TODO don’t use const vars for int/float constants

What does that mean?

long term

GIMP Plug-In

TODO Preview in thread

TODO UI names for user values

General

TODO Edge behaviour as image attributes

probably never

Blender

TODO noise, gamma, vector/matrix division do not work

TODO edge behaviour is not implemented

TODO antialiasing and supersampling are not implemented

Ideas for world domination

Make MathMap create plug-ins for GStreamer

GEGL

MathMap could create plug-ins for GEGL

MathMap could optimize GEGL graphs

We could take a GEGL graph, and if we have MathMap filters for all the filters in the graph involved, we can create one single MathMap filter that does the work of the whole graph and should be much faster.

Links

http://aviary.com/tools/peacock

http://www.sf.edu/jhlavacek/gallery.html http://www.mai.liu.se/~halun/complex/complex.html http://www.ukuug.org/events/linux2001/papers/html/TRathborne_mathmap-fractals/Paper.html

Various filters

Konrad Stania

winkelrotz=0;
winkelrothor=0;
shiftx=0;
shifty=0;
fa=1;
fb=0;
fc=0;
fd=0;
rotmz=m3x3:[cos(-winkelrotz),-sin(-winkelrotz),0,sin(-winkelrotz),cos(-winkelrotz),0,0,0,1];
rotmh=m3x3:[1,0,0,0,cos(-winkelrothor),-sin(-winkelrothor),0,sin(-winkelrothor),cos(-winkelrothor)];
focus=max(X,Y)*5;
rad=r/R;
trafomat=rotmz*rotmh;
koord1ra=ra:[R*(fa*rad+fb*rad^2+fc*rad^3+fd*rad^4),a];
koord1x=nil:[koord1ra[0]*cos(koord1ra[1])];
koord1y=nil:[koord1ra[0]*sin(koord1ra[1])];
koord1z=0;
koordtraf=[koord1x,koord1y,koord1z]*trafomat;
origValXY(koordtraf[0]/(koordtraf[2]+focus)*focus-X*shiftx,koordtraf[1]/(koordtraf[2]+focus)*focus-Y*shifty)


Remarks:
winkelrotz   : rotate the image (clockwise)
winkelrothor : perspectively tilt the image around horizont.
               In architectural fotography used to make vertical lines strictly paralell
shiftx/shifty: shift the resulting imgae
fa ... fd    : to scale image and remove optical distortion
               in polar koordinates radius = fa * radius + fb * radius ^ 2 + fc * radius ^ 3 + fd * radius ^ 4

make vertically seamless

# Thanks to Martin Weber
origValXY(x,y-sign(y)*Y)*abs(y)/Y+origVal(xy)*(1-(abs(y)/Y))

make horizontally seamless

# Thanks to Martin Weber
origValXY(x-sign(x)*X,y)*abs(x)/X+origVal(xy)*(1-(abs(x)/X))

Unknown

g = user_slider("granularity",0,50);
nxy = xy/R*g;
n = noise3([nxy[0],nxy[1],t*g]);
grayColor(n)*0.5+0.5

Unknown

g = user_slider("granularity",0,50);
nxy = xy/R*g;
point = [nxy[0],nxy[1],t*g];
offset = (point + 0.5) / user_slider("offset granularity",0,50);
n1y = user_slider("n1y",0,10);
n1z = user_slider("n1z",0,10);
offset = [noise1(offset[0]),noise1(offset[1]),noise1(offset[2])] * user_slider("distortion",0,10);
#grayColor(offset[0]*0.5+0.5)
n = noise3(point+offset);
grayColor(n)*0.5+0.5

Unknown

dx=-5;
dy=-5;
sum=rgba:[0,0,0,0];
num=0;
while dy < 6 do
    while dx < 6 do
        sum = sum + origVal(xy+xy:[dx,dy]);
        num = num + 1;
        dx = dx + 1
    end;
    dy = dy + 1
end;
sum / num

Unknown

alpha = user_slider("alpha",0,360);
dir = xy:[cos(alpha),sin(alpha)];
ndir = xy:[-dir[1],dir[0]];
p = xy / m2x2:[dir[0],-ndir[0],
               dir[1],-ndir[1]];
pt = dir * p[0];
vec = xy - pt;
dist = -p[1] / R;
pos = 0.5 + p[0] / R / 2;
lower = user_curve("lower",pos) * 2 - 0.5;
upper = user_curve("upper",pos) * 2 - 0.5;
f = ((dist + 1) / 2 - lower) / (upper - lower);
origVal(pt + ndir * (f*2-1) * )R

Fractal Noise

z = user_slider("z",0,10);
depth = floor(user_slider("depth",1,10));
pers = user_slider("persistence",0,1);
g = user_slider("granularity",0,50);
nxy = xy / R * g;
xyz = [nxy[0],nxy[1],z];
i = 1;
sum = 0;
max = 0;
while i < depth + 1 do
    sum = sum + noise(xyz*i) * (pers^i);
    max = max + (pers ^ i);
    i = i + 1
end;
grayColor(sum / (max * 2) + 0.5)

Fractal Noise Displace

z = user_float("z",0,10);
depth = user_int("depth",1,10);
pers = user_float("persistence",0,1);
g = user_float("granularity",0,50);
strength = user_float("strength",0,50);
nxy = xy / R * g;
xyz = [nxy[0],nxy[1],z];
i = 1;
sum = xy:[0,0];
max = 0;
while i < depth + 1 do
    persi = pers ^ i;
    sum = sum + xy:[noise(xyz*i),noise(xyz*i+[3.1415,2.71234,5.4322])] * persi;
    max = max + persi;
    i = i + 1
end;
val = sum / max;
origVal(xy + val * strength)

Smooth Mugl

sl=30;
nx=floor(x/sl-0.5); ny=floor(y/sl-0.5);
alpha=[0,0,0,0];
radii=[0,0,0,0];
phii=[0,0,0,0];
xc=[0,0,0,0];
yc=[0,0,0,0];
heuslfaktor=sl/user_slider("size",1.2,5);
i=0; while i < 4 do
    ix=nx+i%2;
    iy=ny+floor(i/2);
    xc[i]=(ix+0.5)*sl;
    yc[i]=(iy+0.5)*sl;
    alpha[i]=(noise([ix*0.3,iy*0.3,0])+1)*180+t*360;
    kurde=toRA(xy:[x-xc[i],y-yc[i]])+ra:[0,pmod(alpha[i],360)];
    phii[i]=kurde[1];
    radii[i]=heuslfaktor*(1+noise([xc[i]+cos(kurde[1]),yc[i]+sin(kurde[1]),0]*2)*0.7);
    i = i + 1
end;
i=0;found=0;
while !found && i<4 do
    heuslkurde=toRA(xy:[x-xc[i],y-yc[i]])+ra:[0,alpha[i]];
    if heuslkurde[0]<radii[i] then
        found = 1
    else
        i=i+1
    end
end;
if i >= 4 then
    rgba:[0,0,1,1]
else
    if heuslkurde[0] < radii[i]-1.5 then
        origVal(toXY(heuslkurde)+xy:[xc[i],yc[i]])
    else
        rgba:[0,0,0,1]
    end
end

Unknown

dur = user_slider("duration",0,5);
amp = exp(-t / dur);
front = t * R * 2;
dropxy = xy:[0,0];
dropr = toRA(xy - dropxy)[0];
disp = sin(dropr * 360 / user_slider("length",0,100) + t * 360 * user_slider("speed",0,100)) * cos(clamp(if front == 0 then 90 else dropr/front end * 90,0,90))^2 * user_slider("amp",0,100) * amp;
dispxy = toXY(ra:[disp,0]);
origVal(xy+dispxy)

Unknown

dur = user_slider("duration",0,5);
amp = exp(-t / dur);
front = t * R * 2;
n = user_slider("num drops",1,20);
lastt = user_slider("t of last drop",0,1);
i = 0;
dispxy = xy:[0,0];
while i < n do
    dropt = scale(noise([n * pi, 0 * pi, 0]),-1,1,0,lastt);
    dropxy = xy:[noise([n * pi, pi, 0]),noise([n * pi, 2 * pi, 0])] * XY;
    dropr = (toRA(xy - dropxy))[0];
    disp = sin(dropr * 360 / user_slider("length",0,100) + t * 360 * user_slider("speed",0,100)) * cos(clamp(if front == 0 then 90 else dropr/front end * 90,0,90))^2 * user_slider("amp",0,100) * amp;
    dispxy = dispxy + toXY(ra:[disp,0]);
    i = i + 1
end;
origVal(xy+dispxy)

Frame Diff

(origVal(xy,frame)-origVal(xy,frame-1))/2+rgba:[1/2,1/2,1/2,1]

Fade

origVal(xy,image:0,frame)*(1-t)+origVal(xy,image:1,frame)*t

Movie Spiral

q=sin((r-a*0.1)*10+t*5*360)*0.5+0.5;
origVal(xy,image:0,frame)*(1-q)+origVal(xy,image:1,frame)*q

Slide (left->right) Transition

if (x+X)/W < t then origVal(xy,image:0,frame) else origVal(xy,image:1,frame) end

Unknown

st=15;p=clamp((x+X-t*W)/st+0.5,0,1);origVal(xy,image:0,frame)*(1-p)+origVal(xy,image:1,frame)*p

Craig Kaplan

Okay. Well, part of the talk was about how some of Escher’s patterns with shapes that diminish towards the centre of the plane are just mathematical transformations of simple Euclidean patterns. I demonstrated this fact by carrying the transformation out on a couple of patterns. You can see the results here:

http://www.cgl.uwaterloo.ca/~csk/tmp/butterflies.png http://www.cgl.uwaterloo.ca/~csk/tmp/butterflies_merc.png http://www.cgl.uwaterloo.ca/~csk/tmp/snakes.jpg http://www.cgl.uwaterloo.ca/~csk/tmp/snakes_merc.png

For reference, the transformation is to put the coordinate axes of the complex plane at the center of the circular picture and to pass the image through the function f(z)=log(z) in the complex plane. More precisely, I used this function in MathMap:

y = y/H*360.0*1.6;
cy = cos( ay );
sy = sin( ay );
ax =-0.8+(x+X)/W*9;
ex = exp( ax );
mx = ex*cy;
my = ex*sy;

ssx = mx*3;
ssy = my*3;

origVal( xy:[ssx+4,ssy-1] )

Language extension ideas

defun map ((nil:1)->nil:1 f, _:_ x)
    construct(x.tag:x.length, lambda (nil:1 i) f(x[i]) end)
end

defun map ((nil:1, nil:1)->nil:1 f, T:L x, T:L y)
    construct(x.tag:x.length, lambda (nil:1 i) f(x[i],y[i]) end)
end

defun __add (ri:2 a, ri:2 b)
    ri:[a[0]+b[0],a[1]+b[1]]
end

defun __add (T:L a, _:1 b)
    map(lambda (nil:1 x) x+b end, a)
end

defun __add (T:L a, T:L b)
    map(lambda (nil:1 x, nil:1 y)
            x+y
        end, a, b)
end

defun __mul (ri:2 a, ri:2 b)
    ri:[a[0]*b[0] - a[1]*b[1], a[0]*b[1] + b[0]*a[1]]
end

defun __div (ri:2 a, ri:2 b)
    if b == ri:[0,0] then
        ri:[0,0]
    else
        q = b[0] * b[0] + b[1] * b[1];
        ri:[(a[0]*b[0] + a[1]*b[1]) / q, (-a[0]*b[1] + b[0]*a[1]) / q]
    end
end

1.3.5

GIMP Plug-In

DONE The main input image can be overwritten

Copying over uservals from one filter to the next even overwrites the first image userval if the names match. This is an issue if the new filter names its first input image after a non-first input image of the last filter.

Designer

DONE The Sharp-Blur filter can’t be loaded into the expression db

DONE Filters can be included twice

When a sub-designer-filter includes a filter and the main filter (or probably some other sub-filter) uses them, too, they are included twice. See Blur->Sharp Blur.

DONE make order of inputs in composer deterministic

General

DONE What’s userval_t->type for?

Not needed anymore.

Packaging

DONE check out Robin Norwood’s spec file changes (for Fedora)

DONE update homepage

Filters

DONE Dissolve doesn’t work properly

It probably shouldn’t be a pixel filter, and the opacity control misses the 0 and 1 marks.

DONE Remove “Addition with Opacity Native”

Native Filters

DONE Cache convolution filters as well

DONE Implement caching of native filter results

Compiler

DONE Make min/max builtins use min/max ops

DONE The compiler should complain when two filters have the same name

DONE We still have some bugs in the coordinate transformations

This does not work as expected if the aspect ratios of in1 and in2 are different:
filter bla_ident (image in)
  in(xy)
end

filter bla (image in, image in2)
  bla_ident(in2,xy)
end

It turns out the problem here is that closures get their pixel size from their first image, which is wrong. It should always be the same for all closures, namely the canvas size.

DONE Still still bugs in the coordinate transformations

Doesn’t work if the two images have different aspect rations:
stretched filter util_ident (stretched image in)
  in(xy)
end

filter __composer_filter__ (image comp_burn_in1, image util_ident_in)
  util_ident(util_ident_in, xy)
end

Actually no, this isn’t a bug, but a consequence of our half-hearted coordinate transformation fixes. The stretched ident filter now hands coordinates straight through to its input image, but the coordinate it gets are in the aspect ratio of the main filters first input image.

DONE Profile the compilation of Gaussian Blur->Spin Zoom->Droste

Most of the time is spent in GCC compiling the C code. Without the backend we compile in 0.4s, with the GCC backend in over 3s. Just compiling the C code on the command-line takes about 2.5s. With LLVM we’re much faster - about 0.8s including the backend.

DONE Gaussian Blur->Droste still has some loop issues

Will be solved by *caching of native filter results.

DONE Gaussian Blur->Spin Zoom->Droste crashes

So does Gaussian Blur->Random Blur->Droste

Junk - file properly

unload c code in openstep library

macos: gruen fuer ints, floats und konstanten duenkler “Informational Text” in library? tool tips library at startup

manual: WH,W,H statt wh,w,h koordinatensystem erklaeren

> (i) Die Moeglichkeit, bei ‘user_slider(name,minimum,maximum)’ als ‘maximum’ > und ‘minimum’ Ausdruecke anzugeben, die von den Variablen ‘X(nil:1)’ und > ‘Y(nil:1)’ abhaengen. > > (ii) Einen voreingestellten Wert fuer ‘userslider’ festzulegen (mit der > gleichen Methode wie unter (i)).

macos x: movies (realtime!)

gimptool in Makefile statt gimp-config verwenden radio buttons NaN und Infty ausprobieren init optimization save expression in image

normalisierung von matrizen (matrix/determinante) inverse matrix pixel konvertieren schneller machen abs cool machen (abs([-3,4]) sollt 5 sein, ned [3,4])