Shrinking an image is a question of what to throw away. Enlarging one is a harder question: what to make up. Double the width and height of a photograph and the result has four times the pixels, so three out of every four were never captured by anything. Every upscaling method is a rule for inventing them, and the methods differ only in what that rule knows.

The classical kernels: a weighted average of the neighbours

The traditional methods all work the same way. For each new pixel, look at the source pixels nearby and take a weighted average. What changes is how many neighbours are consulted and how the weights fall off with distance.

  • Nearest neighbour copies the closest source pixel. No averaging at all, so every original pixel becomes a hard-edged block. Ugly on a photograph, and exactly right for pixel art, screenshots of pixel-aligned UI, and anything else where the blocks are the content.
  • Bilinear blends the four nearest pixels. The blocks disappear, and so does the crispness: edges become ramps, and the image looks like it is behind frosted glass.
  • Bicubic uses a four-by-four neighbourhood and a curve that slightly overshoots at edges. That overshoot reads as sharpness, which is why it was the default in image editors for decades.
  • Lanczos uses a wider, windowed-sinc kernel. It is the closest of these to the mathematically ideal reconstruction filter, keeps the most edge definition, and pays for it with ringing — faint light and dark halos echoing either side of a high-contrast edge.

Those trade-offs are real, and choosing among them is worth doing. But they are trade-offs within one family, and the family has a ceiling.

The ceiling: an average cannot contain what its inputs do not

A kernel can only redistribute the information already in the source. Fine detail — the weave of a fabric, individual hairs, the grain of wood — lives in rapid changes between neighbouring pixels. If the source was sampled too coarsely to record a change, that change is not in the data, and no weighting of the data will put it back. This is the sampling theorem stated as a disappointment: the best possible interpolation of a low-resolution image is a smooth version of it, never a detailed one.

Which is why enlarged images look soft regardless of the kernel. Better kernels move the softness around; none of them remove it.

Sharpening is not the fix it looks like

The usual remedy is to sharpen afterwards. An unsharp mask increases contrast across edges, which the eye reads as detail, and on a modest enlargement it helps. It does not add information either. It exaggerates whatever transitions exist, and on a real-world source those include sensor noise and the block boundaries left by JPEG compression. Sharpen a heavily compressed upload after a 4x enlargement and the artefacts become the most visible thing in the picture.

What a learned model does differently

Super-resolution networks start from the observation that the missing detail is not random. Grass has a texture, so does skin, so does brick. If you have seen enough photographs, you can make a far better guess at what is between two pixels of grass than an average of those two pixels.

Training makes that concrete. Take a large set of high-resolution photographs, shrink each one, and train a network to recover the original from the shrunk copy. Across enough examples it learns the statistics of real images — what an edge looks like at full resolution, which textures go with which colours and shapes — and applies them to images it has never seen. Where a kernel has a fixed formula, the network has a prior: an informed expectation of what photographs contain.

Two ways to be trained, and why the result looks the way it does

Early networks were trained to minimise the pixel-by-pixel difference from the original. That is a sensible-sounding goal with a revealing failure. When the model is unsure whether a patch of texture should have its stripes here or one pixel over, the answer that minimises average error is the blend of both, which is a smear. Those models scored well on numerical benchmarks and still looked soft.

The ESRGAN family, which is what most practical upscalers today descend from, adds a second network during training whose only job is to tell real high-resolution photographs from upscaled ones. The upscaler is rewarded for fooling it. A smear is easy to spot, so the model learns to commit: to draw a plausible texture rather than the average of all plausible textures. Later variants also train on inputs deliberately damaged with blur, noise and compression, so the model learns to clean up a real-world upload rather than amplify its flaws — the exact case where sharpening fails worst.

The honest catch: plausible is not the same as true

"Commit to a plausible texture" is the feature and the limitation in one sentence. The detail an AI upscaler produces is not recovered. It is generated — consistent with the low-resolution input, and typical of the photographs the model was trained on, and possibly not what was actually there.

For a product shot, a background, a photograph of a landscape, that is fine: nobody needs the blades of grass to be the real blades, only for them to look like grass. It is not fine when the specific pixels carry meaning:

  • Text and numbers. A model will sharpen an illegible licence plate or serial number into a legible one. The characters it draws are a guess, rendered with confidence.
  • Faces of real people. At large factors on small sources, the model fills in facial detail from its training data. The result can look convincingly like a person who is not quite the person photographed.
  • Anything used as evidence or for measurement — documentation, medical or scientific imagery, insurance claims. An upscaled image there should be treated as an illustration, not a record.

None of this is a reason to avoid the technique. It is a reason to be clear about what it makes: a better-looking picture, not a more accurate one.

Choosing a method

In practice the decision is short:

  • Making an image smaller — which is most resizing — never needs a model. Nothing is being invented; a good kernel and correct gamma handling is the whole job.
  • Pixel art and hard-edged graphics: nearest neighbour at whole-number factors. Anything smoother destroys the thing you are enlarging.
  • A small enlargement of a decent source: a classical kernel. If the source is only slightly short of the size you need, the softness is mild and a network is expensive overkill.
  • A real gap — a catalogue of old 600px product photos going onto a high-density display, user uploads that came through a messaging app, a legacy archive that predates large screens — is where a learned model earns its cost. Twofold and fourfold enlargements are where the difference from a kernel is no longer subtle.
  • The source matters and the pixels are evidence: neither. Find a better original, or show the small one honestly.

How ai_upscale behaves here

The parameter takes 2x or 4x — the factors super-resolution models are trained for. Combine it with the other parameters as usual: ?ai_upscale=2x&w=1200 enlarges the source with the model, and any cropping or resizing you ask for works from the reconstructed detail rather than from a blurred enlargement.

Output area grows with the square of the factor — a 4 megapixel source at 4x is a 64 megapixel result — so very large sources are refused with a clear error rather than attempted. Upscaling is billed per factor step, and the result is cached like any other rendition: you pay once per image and size, not once per request.