caching
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
|
caching [2022/02/09 16:38] yves created |
caching [2023/11/27 18:27] (current) chris |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ===== Caching ===== | ===== Caching ===== | ||
| - | For optimal performance, PMA.core support both tile cache and cloud cache. | + | For optimal performance, PMA.core supports both tile cache and cloud cache. |
| - | We have a strategy where cache is deleted after a retention period automatically and no new cache is created if there is not enough disk space. | + | Our policy is that caches are deleted automatically after a retention period and no new cache is created if there is not enough disk space. |
| + | |||
| + | There's a [[settings_image#cache_settings|separate configuration page for cache settings]] | ||
| ==== Tile caching ==== | ==== Tile caching ==== | ||
| - | PMA.core will not cache tiles or thumbnails if there is not enough disk space (as defined in the [[settings]]). | + | Extracted tiles from [[https://realdata.pathomation.com/what-whole-slide-images-wsis-are-made-of/|WSIs]] are stored in a cache repository. This is Just In Time (JIT) caching: the tiles are only stored in the cache after being generated for the first time. |
| + | |||
| + | When many people look at the same slide, they are typically interested in the same regions of interest and slide details. This means that as a system is used over time, almost all of its accessed content will be available through the tile cache. Tile caching in this regard also makes PMA.core a much more scalable system than it would be without this mechanism. | ||
| + | |||
| + | PMA.core will not cache tiles or thumbnails if there is not enough disk space (as defined in the [[settings_image#cache_settings|settings]]). | ||
| ==== Cloud caching ==== | ==== Cloud caching ==== | ||
| + | |||
| + | Slides that are retrieved through cloud storage like S3 or Azure, benefit from an additional layer of caching. | ||
| + | |||
| + | {{ :cache_structure.png?nolink&400 |}} | ||
| + | |||
| + | Based on requests received from clients, PMA.core will retrieve part of the original slide from cloud storage, and keep that in a local cache repository. From that //data chunk// eventual tiles are extracted. | ||
| ==== Troubleshooting ==== | ==== Troubleshooting ==== | ||
| Line 15: | Line 27: | ||
| It is impossible to load an image from cloud storage if there is no disk space. | It is impossible to load an image from cloud storage if there is no disk space. | ||
| - | Let's assume that the whole disk space is 100GB, currently consumed is 95GB and PMA.core is instructed not to cache if there is not at least 1GB of free space. Furthermore, let's assume the retention days is set to 30 and that all the cache has been created in the last 10 days. | + | Let's assume that the whole disk space is 100GB, 95GB is currently consumed and PMA.core is instructed not to cache when there is less than 1GB of free space. Furthermore, let's assume the retention days option is set to 30 and that all the cache was created in the last 10 days. |
| + | |||
| + | If 100 requests to load different images come in, PMA.core has to cache chunks of these images, and suddenly there is no space and there is nothing that can be deleted. | ||
| - | If now there are 10 or 100 requests to load different images, PMA.core has to cache chunks of these images and suddenly there is no space and there is nothing that can be deleted. | + | You can take this to a higher extreme and assume you have 100GB of free space on a 100GB disk and you receive 100s of requests for "Image Info". This will again cause PMA.core to store chunks of 100s of images to the disk simultaneously, and may end up in a full disk (without breaking any rule). |
| - | you can take this to a higher extreme and assume you have 100GB free space on a 100GB disk and you receive 100s of requests for "Image Info". This will again cause PMA.core to store chunks of 100s of images to the disk simultaneously, and may end up in a full disk (without breaking any rule). | + | After this point nothing works, because PMA.core doesn't know what to delete; all the cache is fresh. You now have to purge the cache folder(s) manually, after which the build process may re-commence (you may also provide more storage for your cache repositories this time around). |
| - | After this point nothing works, because PMA.core doesn't know what to delete; all the cache is fresh. | ||
caching.1644413939.txt.gz · Last modified: 2022/02/09 16:38 by yves
