Now what?


Mostly more thoughts on degriddifying, and emphasizing the target content vs. letting tile content show through.


One tweak just for speed is this: since I quantize the wavelet coefficients all the way down to {positive, negative, ignorably small}, there's certainly no need to save and compare them as floats.

So, the first degriddifying attempt was weighting the layers with the sin^2(x)*sin^2(y) at the frequency that sent it to zero at all the edges in the layer.

The next thing to do with that is to experiment further with adjusting the amplitudes of the weights to tone down the highest-frequency spots. Also, perhaps tiling all the way down to 16x16 pixel subsquares is just too far some times. OTOH, perhaps there's value in going down even further (downweighted strongly) for sheer punk rock multiresolution points.

It would be extremely convenient to update the triage (collage/montage) code to collect the layer information (perhaps write it out to a file now) and take a first shot at processing it, and then accept some slider-bar/radio-button input.

Have to find out how much of the tile edges' standing out is a result of the problem with using a wavelets transform with periodic boundary conditions.

Another direction would be: matching images to placements on the grid for a cheap first pass, and then let them wiggle around trying to slide down to a better match. Since there's one target and many tiles, it's not a big investment to take wavelet transforms of manyx by manyy grid squares (the tiling size), nor to stagger in x, then y, then both, by half a squareside -- or even a finer grain. Running the top40 wavelet coefficients of the potential tile images past top40s of this set of usual suspects is still cheap. Now you expect each one to travel much less far than that (otherwise it'd've started out there) as it homes in with some kind of gradient descent-type search...

Weight the small tiles more heavily than the large ones along Finded Edges of the image (this might well not be useful). Or weight layers more harshly by score. Or, now that the expensive job of placing the tiles has been done efficiently with the wavelet procedures, the weighting can refer to the accuracy of the pixel data matches.

For example, with cluster-weighted modeling! Oddly enough, this should make both the tile and target content more clear than in a straightforward averaging. The gaussians seek out the positions where a particular layer accounts for the target especially well; in that region, its layer will show through with little interference, and trail off at the edges.

Make a drawing program environment such that the user can go in with e.g. a "darken" brush to draw layers upwards that match better in that manner -- say, to point out important details, like edges, or the arctic fox's little nosey.

If there were some sense of foreground regions in the target image that the user wanted to have accurately (identifiably!) represented, and a background where it would be more acceptable that tile content dominate, the program could be made to accept a (BW or grayscale) mask, and the weighting at each point would include a function of that value.

Just towards making this a more usable, directable tool, it would be nice to allow user tweaking of regions of the images produced. Once in a while, some tile that proves visually distracting or semantically jarring ends up prominently featured. There's no shame in asking for an alternate tile, or wanting to decrease that layer's weighting in that region. Perhaps you could mouse-click on the point of concern and it would cycle through the layers, revealing the image that appears at that point in each, and its filename. Once, to get rid of some of the more rancidly darling images I never wanted to see again, I deleted the files and rebuilt the database. The second time, pressed for time, I wrote a tilefile-copying program that would leave out another set of offenders. But there's still the gentler art sense of "this just doesn't work right here."

Actually a wavelets-based drawing program where painting a line was a little like raking leaves (or fingerpaints, or dragging iron filings with a magnet) would be stunningly interesting. The existing (large) coefficients would adjust themselves to suit, but not appear out of nowhere or increase unsmoothly. That might beat an airbrush tool all to hell. Maybe.

Oh, and one little plan for when the faster machine arrives Monday: try out some other image collections.


ebeth@media.mit.edu