Big news! PictSure: Pretraining Embeddings Matters for In-Context Learning Image Classifiers has been accepted at the German Conference on Artificial Intelligence (GCAI) 2026. You can read the updated paper on arXiv.
Alongside the publication, we’re releasing a new family of PictSure models that puts our own central finding to work: it’s the quality of the frozen encoder — not the fusion transformer — that drives in-context learning performance.
What’s new since the first release
Our first Hugging Face release shipped two models built around a custom-pretrained ResNet18 and a ViT trained in-house with a triplet-loss objective. For this release, we instead plugged well-established, off-the-shelf pretrained encoders straight into the same ICL fusion transformer:
- PictSure-ResNet — a supervised ResNet backbone pretrained on ImageNet-1K
- PictSure-CLIP — a CLIP ViT encoder, contrastively pretrained on 400M image-text pairs (with only the frozen image encoder used)
- PictSure-DINOv2 — a self-distilled DINOv2 ViT, trained on 142M images with no labels at all
- PictSure-DINOv2-large — the larger DINOv2 variant, for the strongest out-of-domain generalization we’ve measured so far
That’s five models in total — the original PictSure-ViT (triplet-loss) plus four new backbones — all sharing the exact same four-block, eight-head fusion transformer, with only the embedding backbone changing between them.
Why it matters
The paper’s central result carries directly into this release: encoder pretraining dominates ICL performance, far more than how the fusion transformer itself is trained. We found that widening the fusion transformer’s training data from ImageNet-21K alone to a 16-dataset, multi-domain mixture barely moved the needle — the fusion layer already reads well-structured embedding spaces reliably. The lever that actually matters is the encoder.
Concretely, across our benchmarks (tieredImageNet, PlantDoc, Brain Tumor MRI, OCTMNIST):
- DINOv2 and CLIP variants consistently outperform the ResNet-based variant, most clearly on tieredImageNet and PlantDoc.
- PictSure stays highly competitive against RFS, PMF, CAML, and even Qwen3.5 122B in a pure ICL setting, while being substantially smaller than CAML and Qwen3.5.
- Purely visual embeddings continue to show a clear edge on out-of-domain medical imagery (Brain Tumor, OCTMNIST), where language-aligned encoders like CLIP tend to lose their advantage.
Also new: an MCP server
To make PictSure easier to drop into agentic pipelines, we now ship an MCP server that exposes PictSure as a callable tool. This lets LLM-based agents perform few-shot image classification directly, without any custom integration code.
Get the new models
All four new variants are open-sourced on Hugging Face:
Code, weights, and the MCP server are all available at github.com/PictSure.
Next steps
- Extend class coverage beyond the current 10-way setup
- Explore broader encoder families and improved pretraining curricula
- Grow the MCP tooling for agentic, few-shot vision workflows
Citation
@article{schiesser2025pictsure,
title={PictSure: Pretraining Embeddings Matters for In-Context Learning Image Classifiers},
author={Schiesser, Lukas and Wolff, Cornelius and Haas, Sophie and Pukrop, Simon},
journal={arXiv preprint arXiv:2506.14842},
year={2025}
}