PMA.core 3.0.5

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Storage management =====

Storage is important in digital pathology. Image data can be subject to any of the [[https://en.wikipedia.org/wiki/Big_data|3]] ([[https://www.ibm.com/blogs/watson-health/the-5-vs-of-big-data/|or 5]]) [[https://en.wikipedia.org/wiki/Big_data|Vs of data science]]:

  * Variety - Imaging data in pathology is generated during biopsies (macroscopic observations on the sectioning station), brightfield microscopy (high-resolution), immuno observations (multiple channels), and z-stacking.
  * Volume - The recorded images are large: think 100k x 50k pixels in 16-bit RGB color resolution. An individual slide can be anywhere between 100 MB (a needle biopsy), or several GB in size (a solid tumor section sample scanned at 40X magnification)
  * Velocity - Data comes in rapidly, with 100s of slides being scanned on a daily basis. This poses challenges in terms of how much pre-treatment and time you can spent on any individual slide.

For these reasons it's important to have tile server solution that is flexible.

PMA.core supports the following [[rootdir_config#adding_mounting_points|storage media]]:

  * local hard disk (think conventional ''C:'' and ''D:'' drives and partitions)
  * network storage like SMB shares (must be accessible via UNC ''%%\\server\path\to\data%%'' routes)
  * [[rootdir_s3|S3-compliant cloud storage]] (Amazon AWS, Western Digital HGST, NetApp, Arvados, IBM...)
  * [[rootdir_azure|Microsoft Azure storage]] (including [[rootdir_azure#data_lake_gen2_storage|Data Lake Gen 2]])
  * [[https://filezilla-project.org/|FTP server]]

Our tile server introduces [[rootdir|root directories]]: virtual mounting points that can point to any of these types of storage where you have your slides and make them available to end users.

Most importantly, you can configure your root-directories in a hybrid fashion, with some storage pointing to traditional hard disks, and other (perhaps long term) storage pointing to cloud resources.

This hybrid configuration model also means you can scale easily over time: you can start with a setup whereby your slides are mostly placed on a (big) local hard disk. After a while, you can seamlessly switch over to your organization's network storage. At an even later stage, you can fluidly migrate to S3-compliant cloud storage. When you have an external collaborator who temporarily wants to share their slide collection with you, you can ask them to setup an FTP server and patch a root-directory through to it.

[[rootdir|Root-directory resources]] can have authentication and impersonation information attached to them. In addition, PMA.core has its own [[acl|access control lists]] to determine what [[user_groups]] and [[user_management|individual users]] can see and do (according to [[crud|the CRUD principle]]).

A comprehensive blog article on the subject of storage and image management is provided at [[https://realdata.pathomation.com|our blog]].
PMA.core 3.0.5

User Tools

Site Tools

Page Tools
