Use S3 copy when assigning to an Shrine::Attachment field

Hi,

I would first like to thank you for all the work done on Shrine ! Especially the refactor done with Shrine 3.

Having said that, I’m seeking your help :
Let’s say that I have 2 models : ModelA and ModelB. Each ones have a field/Shrine::Attachment named “file”.
Let’s also say that I have a Shrine::Storage::Filesystem as :cache, and a Shrine::Storage::S3 as :store .

In that situation, I would like to copy a file attached (and uploaded to S3) on a ModelA instance, to a ModelB instance, with both file being “independant” (destroying the instance of ModelA will only delete its file, not the one on the ModelB instance). Doing something like :

instanceA = ModelA.first
instanceB = ModelB.first

instanceB.file = instanceA.file
instanceB.save

Now, I know that Shrine::Storage::S3 can handle S3 copy directly (and not having to download and reupload) when Shrine::Storage::S3.upload is called on promotion.
The problem is that when I assign something to the Shrine::Attacher, there’s an “upload” to the cache storage, and this will download the file from S3. On promotion, the file is then upload again to S3.

So my question is : how can I assign another’s Shrine::UploadedFile (stored in S3) to an other Shrine::Attacher, and use the S3 copy, and NOT the download & upload logic ?

One way would be to force S3 as the cache storage (I think? I’m not sure), but I don’t find that solution acceptable in my case.

Does anybody have an idea?

1 Like

(Am by no means an expert on shrine, so if I’ve made a mistake etc please correct me thanks)

One idea would be to add another storage type: s3_cache, for example. Then, depending on whether the file already exists on an s3 bucket, you can change the filesystem storage option to an s3 one. That way nothing is downloaded to the filesystem. I’m assuming you still want up have the cache and promote strategy.

I will try this method. One thing that I need is for the Attacher instance to have a reference to the assigned file/UploadedFile : default_storage doc says that the block is run in the Attacher context, but I’ll have to try and see if that mean I can access the assigned file (to check if it’s a UploadedFile, and that this UploadedFile is on S3).

I will let you know how it turns out.

Add a virtual attribute to your model as a flag? There are probably cleaner ways to do this.

@Haerezis 'm whipped something up for you - I but I have not checked that it works 100%. It feels hacky so there are probably better ways. You could also set the default cache on your attacher directly.

Shrine.storages = {
  cache: Shrine::Storage::Memory.new,
  s3_cache: Shrine::Storage::Memory.new, # set this to an s3 and not as demoed here
  store: Shrine::Storage::Memory.new,
}

class MyUploader < Shrine
  # plugins and uploading logic
  plugin :default_storage, cache: -> {
    if record.is_a?(Post)
      if record.copy_to_s3_cache
        :s3_cache
      else
        :cache
      end
    else
      :cache
    end
  }
end

class Post < ActiveRecord::Base
  attr_accessor :copy_to_s3_cache

  include MyUploader::Attachment(:image)
end

class PostTest < Minitest::Test
  def test_s3_cache
    post1 = Post.create(image: File.open("./files/image.jpg"))
    post2 = Post.new
    post2.copy_to_s3_cache = true # should not use the default filesystem
    post2.image = post1.image
    post2.save 
  end
end

Thanks for the example. I think it works, and I should also be able to access the uploaded file in the attacher via the file attribute, and check if it’s an UploadedFile, and if it is, and the storage of the file is S3, then cache directly to S3.

But in the end, I was just being really dumb. Your proposition forced me to read the document with more attention.

I can simply use Attacher#attach instead of assigning via the operator =. Attaching will skip the caching step, forcing a direct store, thus using the S3 copy.

I think that what you’ve suggested should work, but it feels a bit clunky, and Attacher#attach seems more straightforward.

Thanks again for the help !

2 Likes