Skip to contents

Download HTAN files

Source: R/download.R
htan_download.Rd

Fetches open-access HTAN files, routing each by its access tier from htan_files():

Usage

htan_download(
  files,
  dest = getOption("HTAN.download_dir"),
  overwrite = FALSE,
  max_bytes = getOption("HTAN.max_download_bytes", 2e+10),
  token = Sys.getenv("SYNAPSE_AUTH_TOKEN"),
  con = htan_con()
)

Arguments

files

The files to fetch: the result of htan_files() (lazy or collected), or a character vector of HTAN_Data_File_IDs.

dest

character(1) or NULL. A directory to copy the files to, as well as caching them. NULL (the default, unless option HTAN.download_dir is set) returns the paths in the cache.

overwrite

logical(1). Download again even if a verified copy is cached or present in dest.

max_bytes

numeric(1). Refuse to start if the files total more than this many bytes (Level 2 images are often several GB each).

token

character(1). Synapse personal access token, only needed for "open-synapse" files.

con

A connection from htan_con().

Value

A tibble, one row per file: HTAN_Data_File_ID, access, path (in the cache, or in dest if given) and status: "downloaded" (fetched now), "cached" (served from the cache) or "present" (already in dest).

Details

  • "open-gc": open Level 2 imaging in the CRDC General Commons, through anonymous GA4GH DRS (no account needed).

  • "open-synapse": Level 3/4, auxiliary and other open files on Synapse, through the Synapse REST API. Needs a free Synapse account and a personal access token in SYNAPSE_AUTH_TOKEN.

  • "controlled": Level 1/2 sequencing (dbGaP phs002371). Not supported; requesting one is an error.

Downloads are cached. Each file is kept in a BiocFileCache, keyed by its HTAN_Data_File_ID and md5, under the same relative path as HTAN's Filename (so 10x/Visium bundles stay together), and checked against the md5 in the metadata before it enters the cache. Asking for a file again is served from the cache after checking its md5 again; a cached copy that no longer matches is downloaded afresh. Nothing is written to the working directory.

The cache is the downloads directory inside the snapshot cache of htan_con() (option HTAN.cache, then tools::R_user_dir("HTAN", "cache")), or option HTAN.download_cache if set. It is a BiocFileCache of its own, so it can be moved or cleared without touching the snapshot, e.g. with BiocFileCache::removebfc().

With dest, each file is also copied to <dest>/<Filename>, and a copy already there with the right md5 is used as is.

Examples

fixture <- system.file("extdata", "snapshot", package = "HTAN")
con <- htan_con("r7", base_url = fixture)
## a 250 KB open IMC image, fetched anonymously from General Commons
tiff <- htan_files(drs_uri == "dg.4DFC/6231a7d4-7e68-11ee-b51d-e7db3b9207b8",
    con = con)
# \donttest{
dl <- htan_download(tiff, con = con)
dl$status
#> [1] "downloaded"
## a second call is served from the cache
htan_download(tiff, con = con)$status
#> [1] "cached"
# }