flowchart TB
bbs["BBS<br/>source tarballs"] --> master["master.bioconductor.org"]
ru["r-universe<br/>builds packages"] -- "most binaries" --> master
master -- "bioc-sync, hourly" --> mir[("R2: the mirror")]
site["bioc-website<br/>static build, CI"] --> b[("R2: site/<sha>")]
mir --> wk["bioc-edge Worker<br/>build first, else mirror"]
b --> wk
wk --> v(["bioconductor.org"])
ru -- "polled every 15 min" --> reg["bioc-registry<br/>the propagation gate"]
reg -. "own workers.dev host" .-> cran["installable R repository"]
reg -. "DuckDB" .-> pq["Parquet archive"]
reg -- "build-time fetch, /next/ pages only" --> site
reg -. "planned: registry switch" .-> mir
Bioconductor Infrastructure
This site documents the infrastructure behind bioconductor.org: what runs today, what is replacing it, and the contracts between the pieces. It exists because none of it was written down anywhere a newcomer — or a future maintainer — could find in one place.
Start here
| You want to | Read |
|---|---|
| Install packages, or check whether anything changed for you | Installing packages |
| Report something broken, slow or missing | Reporting a problem |
| Run a mirror | Running a mirror |
| Understand how a request is answered and how content is published | How bioconductor.org works, then this page |
| Operate the stack, or know who does | Operations: who runs what, runbooks, top risks |
| See what is waiting on a decision | Decisions pending and the roadmap |
| Know why it was built this way | The ADRs |
TL;DR
Since 2026-09-28, bioconductor.org is served by the new stack. The replacement estate is a set of repos that meet at data contracts rather than shared machines, which is why each can proceed — or be replaced — without stopping the others.
| Repo | What it is | Retires | Status |
|---|---|---|---|
| bioc-registry | The data plane: an r-universe observer plus the propagation gate that decides which build is fit to publish and what is true about a package, exposed as an installable repository and a Parquet archive. It publishes at its own host; bioconductor.org does not serve from it yet (registry switch) | Planned: the three builders and the legacy build system. Today bioconductor.org’s packages still come from BBS and r-universe through master (how it works) |  |
| bioc-website | The builder: a Python pipeline and an Astro static build that render package truth into ~93,000 pages, published immutably by CI | The master and staging webservers |  |
| bioc-edge | The serving side: a Cloudflare Worker and the R2 mirror behind bioconductor.org, owner of the route table that decides what is served where |
CloudFront, the S3 bucket, the Open Storage Network buckets |  |
| bioc-traffic | Downstream of the request logs. Today: the daily check that yesterday’s logs arrived, log analysis, and a first (v0) traffic classification (bioc-traffic#5, still open). Planned: download statistics from it. Published statistics still come from the legacy pipelines (download stats) | The legacy download-stats pipelines | Not yet producing statistics |
| bioc-build and bioc-manifest | The standalone build system for the packages r-universe doesn’t build, and the reviewed manifest that authorizes what it builds (ADR 0010) | The legacy build machines, for data and workflow packages | Proof of concept |
| bioc-infrastructure | This documentation site | Nothing. It is the map. |  |
Two prototypes are archived (read-only, and private) since 2026-09-29: bioc-cloudflare, whose history continues in bioc-edge, and bioc-origin. Older pages and ADRs link to their issues, which remain readable; a mention of bioc-cloudflare as where code lives means bioc-edge today.
The health and freshness badges are daily probes of the live system, so red means something is down or stale. The site and docs badges are build workflows; red means the last build failed. More in CI & scheduled jobs. Where this is all going, milestone by milestone: the roadmap. How a request is answered and how content gets published today, with the terms these pages use: how bioconductor.org works.
TipLive since 2026-09-28
bioconductor.org itself is served by the new stack, so there is nothing to configure: BiocManager::install() works as it always has. If anything behaves differently from how it used to, that’s a bug we want to hear about. What has changed, with numbers: What the new site has changed.
Where to file issues
Every report goes through the bioc-infrastructure issue forms and is triaged by hand; suspected security problems go by email. Details: reporting a problem.
How the pieces meet
Solid lines to bioconductor.org are the path that serves it today: packages come from master through the hourly sync, and the registry is not in that path. Dashed lines are the registry’s own outputs and the planned switch. Detail: how bioconductor.org works.
The organising principle is one idea applied at two scales: define where the data comes from once, and let every consumer render it. Inside bioc-website that shows up as the astro/data/ seam between its pipeline and its renderer. Across the estate it shows up as the table above. The website is one consumer of package truth. It is deliberately not the place where “what is true about a package” gets decided.
The contracts
Everything above holds together at five seams. They are worth stating in one place, because each is a place where a change in one repo can quietly break another.
1. astro/data/ — pipeline to renderer, inside bioc-website. JSON written to disk by a Python pipeline, read by an Astro build. Nothing downstream knows where the data came from; nothing upstream knows how it is rendered. That seam is what let the same component render both the legacy-parity pages and the new data-plane-fed pages unchanged.
2. The registry’s HTTP API — data plane to website, fetched at build time from /data/prop/{universe}/index.json, /data/state/{universe}/latest, and /prop/{universe}/cas/{sha256}. This is what the /next/ pages render: what propagated, not whatever r-universe last built.
3. The R2 object layout — build to serving. The website’s CI writes immutable site/<sha>/ builds, a site/latest pointer, and preview/pr-<n>/ for every pull request (served at /_pr/<n>/). The Worker reads them. The build side publishes; the serving side decides what is served — write access to the website repo never implies the power to change production.
4. The route table — _routes.json, an object in R2, not code. Flipping a route from mirrored legacy content to a static build is a data write; rolling it back is the same write in reverse. No deploy, no rebuild. An empty table means the mirror serves everything, which is the safe default. Since the cutover the table routes / to the latest build: the build serves any path it has a file for, and the mirror copy in R2 serves the rest.
5. Request logs — serving to statistics. The Worker emits one structured record per request in the same field vocabulary as the CloudFront logs it replaces, delivered by Logpush to Google Cloud Storage, so both eras can be one query rather than a migration project. Analysis lives in bioc-traffic. The record keeps the raw client address; de-identification is a view, and nothing published carries the address (ADR 0012).
Where this stands
The serving replacement took over bioconductor.org on 2026-09-28: nameservers moved to Cloudflare in the afternoon and the site followed that evening (DNS runbook). The legacy servers still run behind it, and still produce the content it mirrors, until milestone M2 retires them. Package landing pages exist for 35 releases: 3.23 and 3.24 are regenerated, and the 33 older releases are a frozen snapshot (ADR 0006, site build). The build-system track is earlier. Software source tarballs are still built by BBS and most binaries by r-universe; serving them from the registry instead is the registry switch. Annotation, experiment-data and workflow packages have no replacement builder running yet (bioc-build is a proof of concept).
Numbers throughout this site are point-in-time measurements, not live dashboards — check the date on each page before relying on specifics.
Two costs, not one
Two figures appear across these pages and they are not competing estimates of the same thing:
- ~$5,000/month is the AWS estate being retired — CloudFront, the S3 bucket, and the master and staging webservers — before counting the builder hardware or the staff time to keep three operating systems patched.
- $240-500/month is the estimate for the Cloudflare stack replacing it, from production traffic on 2026-09-28. Storage is about $77 of that (5.1 TB in R2); the rest is per-request charges. R2 charges no egress fees, and egress is most of the difference (details).
Known weak points
Stated here rather than discovered later:
- The website reads the registry’s stored JSON shape directly, not a versioned API. An unreachable endpoint fails the build loudly, which is right; a renamed field renders a blank section on a page and fails nothing. The convention is “never change a shape in place, add fields and tolerate absences” — it is a convention, with no mechanism behind it.
astro/data/has no written schema. Its shape is implied by the code that writes it and the code that reads it, and by nothing else.- The 15-minute poll has no fast alerting. A silent stop is caught by bioc-registry’s daily freshness check, which fails when the latest observation is more than 36 hours old (the freshness badge above).
- One person operates the new stack, with no named backup, on a personal Cloudflare account and one host. This and the other operational risks are on operations.
- Some facts are written in several places — release numbers, repository lists, display labels — and at least one set of display labels already disagrees with itself.
How this site is organised
The sidebar groups pages by reader and by how they are kept:
- Using and How it works describe the system today and are kept current: a stale statement there is a bug.
- Operating is for whoever runs the stack: operations and CI & scheduled jobs.
- Plans carry a status line that should match their tracking issues: the roadmap, the registry switch and the Cloudflare account migration.
- Records are dated snapshots of the legacy estate and the transition, including the executed DNS cutover. They are corrected if they were wrong at the time, not updated to track the system; a status banner points to the current page. What the new site has changed is the one record that is appended to.
- Decisions (ADRs) record the reasoning, including decisions later reversed. They are not edited; a new ADR supersedes an old one.
Note
This site is deliberately public, so that the people affected by this work can read it without needing an account or an invitation. Findings are shared with the Bioconductor core team as they land, so read these pages as work in progress rather than settled conclusions.