Find HTAN files

One row per HTAN data file, combining every file-level metadata table in the snapshot (the tables with HTAN_Data_File_ID, Filename, File_Format and Component columns) with the participants and biospecimens from id_provenance and the CRDC General Commons DRS URI from gc_drs_map. The result is lazy: filter it with dplyr::filter(), then pass it to htan_download().

Usage

htan_files(con = htan_con())

Arguments

con

A connection from htan_con().

Value

A lazy tbl_duckdb_connection, one row per file.

Details

Columns:

HTAN_Data_File_ID, Filename, File_Format, File_Size, md5, Component, entityId, HTAN_Center, Data_Release

From the file's metadata table. File_Format is coarse (hdf5 covers .h5ad and 10x .h5; RData covers .rds), so see also extension.

table

The snapshot table the row came from. A file listed in several tables appears once, from the most recently released one (in r7, 260 mass-spectrometry files are also in the legacy r2_v2 lipidomics_metadata and metabolomics_metadata tables).

assay, level

Component split into the assay and its level (1-4; NA for auxiliary and other files).

extension

Lower-case file extension from Filename, e.g. .h5ad, .rds, .csv, .tsv.gz.

HTAN_Participant_ID, HTAN_Assayed_Biospecimen_ID

From id_provenance. A file derived from several participants (about 1.6 percent of files in r7) lists them comma-separated, so == misses it: to select files by participant, join htan_file_participants() instead.

n_participants

Number of distinct participants.

drs_uri

CRDC General Commons DRS ID, if the file is there.

access

Where the file can be fetched: "open-gc" (open Level 2 imaging, anonymous DRS), "open-synapse" (Level 3/4, auxiliary and other open files, Synapse with a token), or "controlled" (Level 1/2 sequencing; dbGaP phs002371, not supported).

See also

htan_file_participants() to select files by participant.

Examples

fixture <- system.file("extdata", "snapshot", package = "HTAN")
con <- htan_con("r7", base_url = fixture)
htan_files(con)
#> # A query:  ?? x 18
#> # Database: DuckDB 1.5.6 [unknown@Linux 7.0.0-1019-azure:R 4.6.1/:memory:]
#>    table        HTAN_Data_File_ID Filename File_Format File_Size md5   Component
#>    <chr>        <chr>             <chr>    <chr>           <dbl> <chr> <chr>    
#>  1 bulkWES_lev… HTA12_15_1007     whole_e… fastq         2.90e 9 853c… BulkWESL…
#>  2 bulkRNAseq_… HTA12_15_1010     bulk_RN… fastq         2.75e 9 1587… BulkRNA-…
#>  3 bulkWES_lev… HTA12_15_1026     whole_e… fastq         1.17e 9 983c… BulkWESL…
#>  4 bulkRNAseq_… HTA12_15_1030     bulk_RN… fastq         8.07e 8 c5bf… BulkRNA-…
#>  5 scRNAseq_le… HTA12_15_2004     sc_RNAs… bam           2.53e10 3cf4… ScRNA-se…
#>  6 bulkRNAseq_… HTA12_251_1008    bulk_RN… gzip          3.47e 9 3d6b… BulkRNA-…
#>  7 10xvisium_s… HTA12_253_1010    V10X_le… gzip          5.12e 9 3478… 10xVisiu…
#>  8 10xvisium_s… HTA12_253_2010    V10X_le… bam           2.32e 9 ac5a… 10xVisiu…
#>  9 bulkWES_lev… HTA12_15_1025     whole_e… fastq         1.19e 9 9385… BulkWESL…
#> 10 imaging_lev… HTA12_15_1105     imc_lev… OME-TIFF      2.55e 5 a649… ImagingL…
#> # ℹ more rows
#> # ℹ 11 more variables: entityId <chr>, HTAN_Center <chr>, Data_Release <chr>,
#> #   assay <chr>, level <int>, extension <chr>, HTAN_Participant_ID <chr>,
#> #   HTAN_Assayed_Biospecimen_ID <chr>, n_participants <dbl>, drs_uri <chr>,
#> #   access <chr>
htan_files(con) |>
    dplyr::filter(level == 4, extension == ".h5ad")
#> # A query:  ?? x 18
#> # Database: DuckDB 1.5.6 [unknown@Linux 7.0.0-1019-azure:R 4.6.1/:memory:]
#>   table         HTAN_Data_File_ID Filename File_Format File_Size md5   Component
#>   <chr>         <chr>             <chr>    <chr>           <dbl> <chr> <chr>    
#> 1 scRNAseq_lev… HTA5_0000_1003297 HTAN DF… hdf5        132312198 7c55… ScRNA-se…
#> 2 scRNAseq_lev… HTA5_0000_1003255 HTAN DF… hdf5         11599590 ac4f… ScRNA-se…
#> 3 scRNAseq_lev… HTA5_0000_1003257 HTAN DF… hdf5         40003858 0834… ScRNA-se…
#> 4 scRNAseq_lev… HTA5_0000_1003276 HTAN DF… hdf5        109992026 483f… ScRNA-se…
#> # ℹ 11 more variables: entityId <chr>, HTAN_Center <chr>, Data_Release <chr>,
#> #   assay <chr>, level <int>, extension <chr>, HTAN_Participant_ID <chr>,
#> #   HTAN_Assayed_Biospecimen_ID <chr>, n_participants <dbl>, drs_uri <chr>,
#> #   access <chr>