Bethge Lab
University of Tübingen (Tübingen AI Center) · Business
Bethge Lab is an AI research group at the Tübingen AI Center, led by Matthias Bethge, that studies continual self-improvement in agents and releases its benchmarks, code and datasets openly.
About Bethge Lab
What Is Bethge Lab
Bethge Lab is the machine learning group of Matthias Bethge at the University of Tübingen, based at the Tübingen AI Center. It's academic and non-profit, and it works in the open alongside Open Athena, a non-profit that gives academia access to frontier AI capabilities. The group publishes papers, code, benchmarks and data rather than selling a product.
Their research sits on one question: how do AI systems keep learning after deployment? A model frozen at release stops improving, so the lab puts post-training at the center and treats long-horizon experience as the place where models find something new. Work is organized into three lines: recursive self-improvement, human learning in the age of AI, and foundation models for neuroscience.
The limits are worth knowing before you plan around them. There's no hosted service, no support contract and no private roadmap. Output arrives on conference schedules, individual repositories differ in how well they're maintained, and the lab studies a narrow set of questions on purpose. You get research-grade code and honest evaluation, not a vendor relationship.
Getting Started
- Open bethgelab.org and pick the research line closest to your problem, since each one lists its own selected publications.
- Read the paper behind the benchmark you care about, then clone its repository and check the license before you build on it.
- Run the benchmark on your own model or agent scaffold, using the sample-level scores rather than a single average.
- Cite the paper and report the version you used, because several of these benchmarks keep growing as new models arrive.
- For collaboration or hiring, use the lab's application form instead of email, and expect a monthly review cycle.
Product Information
A quick look at Bethge Lab's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- ML researchers
- Evaluation and benchmark teams at AI labs
- Neuroscience and vision scientists
Tasks
- Checking whether an agent can post-train a model
- Stress-testing agent evaluations
- Finding open training data
Scenarios
- Picking an eval harness before a model release
- Literature review on continual learning
- Teaching a graduate seminar
Key features
PostTrainBench for Automated Post-Training
PostTrainBench asks whether an LLM agent can take a base model and post-train it without a human in the loop. Agents get a fixed compute budget, and the resulting model is scored on held-out tasks, so the gain shows up as a number rather than a claim. Teams use it to compare agent scaffolds on equal footing. It's the closest thing here to a stress test for autonomous ML work.
Sample-Level Benchmarking With ONEBench
ONEBench scores individual samples instead of collapsing a run into one average. That matters when a model gets better on easy prompts and quietly breaks on hard ones. The benchmark is built to grow as new models and tasks arrive, so old scores don't freeze in place. You can add your own test sets and keep the comparison fair.
Process Checks for Long-Running Agents
AlgoTune tests whether a model can write genuinely faster numerical code, and QVal measures which steps in a long agent run actually helped. Both target the part of agent work that final-answer accuracy hides. If your agents run for dozens of steps, these give you a cheaper signal than full human review of every trace.
Open Datasets for Vision and Language
DataComp-VLM provides curated image-text data with public baselines, and LAION-BVD adds 10 million hours of open video for multimodal pretraining. Both are built in the open, with filtering decisions documented. Researchers reach for them when licensing or opaque curation blocks the experiment they want to run.
CiteME and Research Agent Checks
CiteME tests whether a language model can find the paper behind a scientific claim. It's a narrow task with a verifiable answer, which makes it useful for spotting models that sound confident and cite nothing. The lab also studies how capabilities follow from training data, so a wrong citation can be traced rather than just counted.
Gaze Prediction With DeepGaze
DeepGaze3.5-VL predicts where people look by treating scanpaths as token sequences and leaning on vision-language models. Earlier releases in the same line are still used for saliency work, and the underlying datasets are public. UX and vision teams use it when they need a model of attention instead of a heatmap from a small sample.
openretina and Shared Retina Models
openretina is a collaborative project that trains retina models across datasets, labs and species. Instead of one lab hoarding recordings, the models and data are shared so results can be compared. Neuroscience groups use it as a common baseline, and the broader goal is a cumulative account of biological vision.
Pros and cons
Pros
- Everything ships open. Papers, code, benchmarks and datasets are public, so you can check a claim instead of trusting a leaderboard.
- Evaluation is unusually honest. Sample-level scoring and process metrics expose regressions that a single accuracy number would hide.
- The research agenda is focused. Three lines, each with current publications, make it easy to tell whether your problem fits.
- Benchmarks stay alive. ONEBench and its neighbors are designed to absorb new models, so a score from last year still means something.
Cons
- There's no product to buy. No hosted API, no dashboard and no support line, which means you own the setup, the compute and the debugging.
- Repository quality varies. Some projects are actively maintained and others are a paper artifact, so check the last commit before you plan a dependency.
- The scope is deliberately narrow. If your question sits outside agent evaluation, post-training or computational vision, the lab won't have much for you.
- Output follows academic timing. A result can appear in a paper months before the code is clean enough to reuse.
Frequently asked questions
No. It's an academic, non-profit research group at the University of Tübingen, based at the Tübingen AI Center and led by Matthias Bethge. There's no commercial arm selling access to the work.
Related content
Explore related tools, skills, and articles for Bethge Lab.
Bethge Lab Alternatives
Claude for Google Workspace
Anthropic · Writing · Productivity · BusinessClaude for Google Workspace puts Claude in a sidebar inside Google Docs, Sheets, and Slides, where it reads the file you have open and edits it in place.
KloudMate 2.0
KloudMate · Coding · BusinessKloudMate 2.0 is an AI observability platform that keeps logs, metrics, traces, and incidents in one place, with KloudMate Assistant investigating alerts and proposing the fix.
Cekura Bench
Cekura · Voice & Language · BusinessCekura Bench is a public benchmark that runs speech-to-speech models and deployed voice agents through 82 simulated phone calls, three times each, then publishes pass rates, latency and cost.