How are files moved from cache to store?


this is somewhat related to the previous thread.

Files get uploaded to cache, and when I save successfully I can see in the logs that the promote job is performed asynchronously. However it seems that the file remains in the cache folder of the bucket, instead of being moved to the store folder - both cache and store use the same bucket.

How are file moved during promotion? I wonder if it’s a particular function not supported by Exoscale, which is the S3 compatible storage I am using.


Uhm… I am using derivation_url to display these images, and now that I checked I have specified a different folder (derivations) for the derivations instead of store. Is this the reason why the images don’t end up in the store folder?

Files are copied to the store, not moved. This is to keep the cached file available to potentially avoid any broken links during promotion, or not-found errors during processing.

The storage class determines this in the #upload method. The built-in S3 storage checks whether the file to upload is a cached file uploaded to S3, and in that case performs an S3 COPY operation instead of an actual upload.

As for the upload location of processed derivation results, the derivation_endpoint plugin determines the location here, so #generate_location is not called. You can override the location via the :upload_location plugin option, which you can find in the docs.