diff --git a/README.md b/README.md index f20cff0..4804393 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ OME-Arrow uses [Open Microscopy Environment (OME)](https://github.com/ome) specifications through [Apache Arrow](https://arrow.apache.org/) for fast, queryable, and language agnostic bioimage data. +> 📐 Benchmark results that inform OME Arrow's design decisions are available in the [ome-arrow-benchmarks](https://github.com/WayScience/ome-arrow-benchmarks) repository. + __Images are often left behind from the data model, referenced but excluded from databases.__ @@ -266,7 +268,8 @@ Notes: ## Benchmarking lazy reads -Use the lightweight benchmark utility in `benchmarks/` to compare lazy tensor read paths (TIFF source-backed, Parquet planes, Parquet chunks): +Use the lightweight benchmark utility in `benchmarks/` to compare lazy tensor read paths (TIFF source-backed, Parquet planes, Parquet chunks). +For more detailed benchmark results and analysis, see the [ome-arrow-benchmarks](https://github.com/WayScience/ome-arrow-benchmarks) repository. ```bash uv run python benchmarks/benchmark_lazy_tensor.py --repeats 5 --warmup 1 @@ -342,4 +345,5 @@ OME Arrow is used or inspired by the following projects, check them out! - [`coSMicQC`](https://github.com/cytomining/coSMicQC): performs quality control on microscopy feature datasets, visualized using CytoDataFrames. - [`pycytominer`](https://github.com/cytomining/pycytominer): supports feature profiling, normalization, and downstream analysis workflows for image-based profiling datasets. - [`iceberg-bioimage`](https://github.com/WayScience/iceberg-bioimage): defines warehouse-oriented patterns for connecting bioimage formats and analytical tables at scale. +- [`ome-arrow-benchmarks`](https://github.com/WayScience/ome-arrow-benchmarks): contains benchmark results and analysis that inform the design choices in OME Arrow. - [`CytoTable`](https://github.com/cytomining/CytoTable): converts image-based profiling outputs into analysis-ready tabular formats such as Parquet. diff --git a/docs/src/presentations.md b/docs/src/presentations.md index f4e19fb..1ff6661 100644 --- a/docs/src/presentations.md +++ b/docs/src/presentations.md @@ -3,3 +3,5 @@ ## OME Community Meeting 2026 - Poster PDF: [OME-Arrow 2026 poster (WayScience/ome-arrow)](https://github.com/WayScience/ome-arrow/blob/main/docs/presentations/2026-OME-community-meeting/ome-arrow-2026-poster.pdf) + +To see benchmark results that help inform OME Arrow's design decisions, check out the [ome-arrow-benchmarks](https://github.com/WayScience/ome-arrow-benchmarks) repository. diff --git a/docs/src/why-ome-arrow.md b/docs/src/why-ome-arrow.md index 17e5d25..6e6984a 100644 --- a/docs/src/why-ome-arrow.md +++ b/docs/src/why-ome-arrow.md @@ -40,6 +40,7 @@ OME Arrow is not a universal replacement for OME-Zarr. ## Preliminary benchmark signal Preliminary results in [ome-arrow-benchmarks](https://github.com/WayScience/ome-arrow-benchmarks) show that outcomes are highly workload- and layout-dependent. +These benchmarks provide important insights into the performance tradeoffs of different OME Arrow layouts and access patterns. In the repository's synthetic wide-table plus image-column runs, Arrow-table-native backends can reduce full-table read time and storage size relative to some alternatives, while write performance varies by backend. In the OME-Arrow-only benchmark that compares against directory-per-image OME-Zarr and TIFF layouts, full write/read timings and random-read timings diverge in different directions depending on operation type. In that same OME-Arrow-only setup, Lance showed random-read timing similar to OME-Zarr (about 0.020 seconds vs about 0.019 seconds average), suggesting Lance can be a practical OME-Arrow-based option for large image repositories when table-native workflows are desired.