Technical implementations of upscaling

Question

So I've seen bicubic and fractal listed as methods of upscaling photos. What's the technical details behind how these work and why/when is one better than the other?

Asked by rfusca

Answer

First off, image enlargement involves moving pixels apart by a fixed amount, and filling in the space between previously neighboring pixels with similar content. This can be done through basic pixel fabrication (bicubic filtering), or via more complex means, such as converting the image into a vector representation, and scaling in vector space.

Bicubic filtering, along with Bilinear filtering, use a relatively simple function curve to blend the colors of neighboring pixels together when you enlarge an image. Bilinear uses a linear function curve, while Bicubic uses a cubic spline function curve (cspline). Bicubic filtration generally produces smoother results, however both algorithms are ultimately blending the values of neighboring pixels together via a function curve to "fill in the gaps" when enlarging an image. It should be noted that bicubic scaling is generally lossless, in that only the space between original pixels is really generated. Depending on the nuances of specific implementations, the original pixels may change slightly, however for the most part, original data is preserved and new data is fabricated to fill in as an image is scaled up. Excellent scaling can be achieved up to around 200% or so, however beyond that, visible edge softening begins to exhibit.

The fractal approach to image scaling takes an entirely different approach. More complex algorithms are employed to analyze the contents of an image, identify edges and "objects", and ultimately convert the image into a fractal vector format. Once the image is vectorized, it can be scaled "lossless" in vector space, then re-rendered at a larger size. Fractal algorithms, such as Genuine Fractals, use a fractal vector algorithm to scale an image while maintaining smooth, sharp edges. This approach is useful when your image initially has sharp and recognizable edges as a key factor, and maintaining clear sharpness on those edges is important.

Additionally, Genuine Fractals attempts to maintain non-edge detail via the concept of "self similarity", preserving detail by considering the image content to be fractal in nature, and re-rendering non-edge content from fractal algorithms. This has the presumed benefit of, rather than simply fabricating information from neighboring pixels, new content can be generated from patterns composed of many pixels. This approach can work great when upscaling by around 200%, however the fundamental vector nature of scaling becomes more apparent when scaling to larger sizes. It should also be noted that this scaling method is not lossless, and some fine pixel detail may be discarded as the algorithm tries to find fractal patterns that can be replicated. Extreme upscaling can result in visible pattern replication, and fine edge detail may be wiped away in the effort to maintain smooth, sharp edges at all image sizes.

BenVista also provides a proprietary algorithm dubbed S-Spline. There is little information about the specifics of this algorithm, however it does seem like another function curve based algorithm. The S-Spline Max scaling in PhotoZoom Pro, like Genuine Fractals, does a superb job maintaining edge definition. This algorithm is also capable of scaling to around 200% or so, however pushing this algorithm much farther results in visible non-edge detail degradation and smoothing. It seems the general compromise in scaling algorithms is either maximim edge definition, or maximum detail preservation. It should also be noted that most third-party scaling algorithms, including Genuine Fractals and BenVista PhotoZoom Pro, automatically apply an unsharp mask to the final image by default. Any comparisons between bicubic and a third-party algorithm should always be done AFTER applying a modest unsharp mask to the bicubic version.

It is possible to use bicubic in an iterative fashion. The fundamental crux of bicubic scaling is that it fabricates more information the larger you scale, at the ultimate cost of sharpness. By performing bicubic scaling in 3-5% incrments, you preserve far more original or near-original detail, and fabricate much less of the image at each step. The final results of an iterative (or stepped) bicubic scaling can maintain considerably greater edge definition, without losing fine detail. The cost, however, is far greater personal investment in scaling, as at the moment, there are no prefabricated applications that will do this for you. You need to manually calculate the new width and height of the image at each step by multiplying the previous width or height by your scaling percentage, and plugging that number into your image editing program's bicubic scaling tool. The end results can be superb, and as sharp as a bicibic image w/ unsharp masking. The limits for how much larger you can scale an image without visible detriment is much greater than other algorithms, at least 400%, possibly more.

Answered by jrista

Check more discussion of this question.