Not storing file in cache

Hi!

I have an uploader defined like this:

require "image_processing/mini_magick"

class FacebookImageUploader < Shrine
	plugin :download_endpoint, prefix: "files", host: "https://#{Settings.asset_host}"
	
	Attacher.validate do
	  validate_max_size 5.megabytes, message: "is too large (max is 5 MB)"
	  validate_extension %w[jpg jpeg png gif webp], message: "is not a valid image"
		validate_min_size 5.kilobytes, message: "is too small (min is 5 KB)"
		validate_mime_type %w[image/jpeg image/png image/webp image/tiff image/webp], message: "is not a valid image"
	end
end

And in the Blog model I have this:

include FacebookImageUploader::Attachment(:facebook_image)

In the Getting Started guide I can see that if I assign a file to an instance of the model, the file is first uploaded to cache even without saving. Is this correct?

If I try this:

irb(main):115:0> b = Blog.last
irb(main):116:0> b.facebook_image = File.open(Rails.root.join("app/javascript/images/anonymous.jpg"), "rb")
=> #<File:/app/app/javascript/images/anonymous.jpg (closed)>

I get nil when querying the facebook_image attribute:

irb(main):117:0> b.facebook_image
=> nil

In my shrine.rb I have this:

require "shrine"
require "shrine/storage/s3"
require "shrine/storage/memory"

s3_options = {
  bucket:            ENV.fetch("S3_BUCKET") { "" },
  access_key_id:     ENV.fetch("S3_ACCESS_KEY_ID") { "" },
  secret_access_key: ENV.fetch("S3_SECRET_ACCESS_KEY") { "" },
  region:            ENV.fetch("S3_REGION") { "" },
  endpoint:          ENV.fetch("S3_ENDPOINT") { "" },
  force_path_style:  ((ENV.fetch("S3_FORCE_PATH_STYLE") { "false" }) == "true"),
}

s3_mirror_options = {
  bucket:            ENV.fetch("S3_MIRROR_BUCKET") { "" },
  access_key_id:     ENV.fetch("S3_MIRROR_ACCESS_KEY_ID") { "" },
  secret_access_key: ENV.fetch("S3_MIRROR_SECRET_ACCESS_KEY") { "" },
  region:            ENV.fetch("S3_MIRROR_REGION") { "" },
  endpoint:          ENV.fetch("S3_MIRROR_ENDPOINT") { "" },
  force_path_style:  ((ENV.fetch("S3_FORCE_PATH_STYLE") { "false" }) == "true"),
}

begin
  if Rails.env.test?
    Shrine.storages = {
      cache: Shrine::Storage::Memory.new,
      store: Shrine::Storage::Memory.new,
      mirror: Shrine::Storage::Memory.new,
    }
  elsif s3_options[:bucket].present? && s3_options[:access_key_id].present? &&
    s3_options[:secret_access_key].present? && s3_options[:region].present? &&
    s3_options[:endpoint].present?

    Shrine.storages = {
      cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
      store: Shrine::Storage::S3.new(prefix: "store", **s3_options),
      mirror: Shrine::Storage::S3.new(prefix: "store", **s3_mirror_options),
      derivatives: Shrine::Storage::S3.new(prefix: "derivatives", **s3_options),
    }
  end

rescue Aws::Sigv4::Errors::MissingCredentialsError, ArgumentError

end

Shrine.logger = Rails.logger

Shrine.plugin :activerecord
Shrine.plugin :cached_attachment_data # for retaining the cached file across form redisplays
Shrine.plugin :restore_cached_data # re-extract metadata when attaching a cached file
Shrine.plugin :determine_mime_type, analyzer: :marcel
Shrine.plugin :derivatives, storage: :derivatives
Shrine.plugin :validation_helpers
Shrine.plugin :upload_endpoint, url: true
# Shrine.plugin :presign_endpoint
Shrine.plugin :store_dimensions
Shrine.plugin :remove_invalid
Shrine.plugin :remove_attachment
Shrine.plugin :backgrounding
Shrine.plugin :mirroring, mirror: { store: :mirror }

Shrine::Attacher.promote_block do
  PromoteJob.perform_later(self.class.name, record.class.name, record.id, name, file_data)
end
Shrine::Attacher.destroy_block do
  DestroyJob.perform_later(self.class.name, data)
end

Shrine.mirror_upload_block do |file|
  MirrorUploadJob.perform_async(file.shrine_class.name, file.data)
end

Shrine.mirror_delete_block do |file|
  MirrorDeleteJob.perform_async(file.shrine_class.name, file.data)
end

Shrine.plugin :url_options,
	cache: { host: "https://#{Settings.asset_host}", public: true },
	store: { host: "https://#{Settings.asset_host}", public: true }

Shrine.plugin :upload_endpoint, max_size: 5*1024*1024

Shrine.plugin :derivation_endpoint,
  secret_key: Rails.application.secret_key_base,
  prefix:     "derivations"

Shrine.derivation :resize_to_fit do |file, width, height|
  ImageProcessing::MiniMagick
    .source(file)
    .resize_to_fit!(width.to_i, height.to_i)
end

# Shrine.plugin :presign_endpoint, presign_options: -> (request) {
#   filename = request.params["filename"]
#   type     = request.params["type"]

#   {
#     content_disposition:    ContentDisposition.inline(filename), # set download filename
#     content_type:           type,                                # set content type (required if using DigitalOcean Spaces)
#     content_length_range:   0..(5*1024*1024),                    # limit upload size to 5 MB
#   }
# }

I am using several uploaders but having problem with the cache when forms are submitted with invalid data and files have been attached. As you can see I have the relevant plugins enabled.

What am I missing?

Thanks!

Ehm I was trying with a small file in the console so it failed validation… :smiley:

Next thing: I have extracted the code that updates a model, Blog, into a form object to improve some mess. When initialising the form object, I assign the values of the blog to the equivalent attributes of the form instance. Then when calling submit on the form, I do the opposite provided that the form validations passed.

If I submit a valid form, the attachment fields are correctly set and the files are uploaded just fine.

However if the form has some validation errors, the files I have already attached are lost and I need to attach them again. I have the relevant plugins enables as said earlier, but I am trying to adapt this to the form way of doing updates.

The main difference between what I see in the console when attaching a file directly to a model, and a form submission, is that the facebook_image param is set to an instance of ActionDispatch::Http::UploadedFile. I don’t see anything in the log indicating that the file is even being uploaded to cache. I can confirm this by looking at the s3 bucket in the cache folder.

What can I do to make the cache storage work with the form object? I could share the code of the form but it’s really just passing the values from the blog to the form on initialise, and from the form to the blog when saving.

Thanks in advance

I got it working… with a hack:

    IMAGE_ATTRS.each do |attr|
      blog.send("#{attr}=", params[attr])
      send("#{attr}=", blog.send(attr))
      send("#{attr}_data=", blog.send("#{attr}_data"))
    end

I found that when submitting even an invalid form, if I set the image field the file is uploaded even though the blog isn’t saved due to validation errors, and I can get the image_data value as well that I can use with the hidden field in the form. It seems to work… but I wonder if there is a better way?

Have you permitted the attachment parameter in your controller? That could explain why the file assignment was seemingly ignored.

Note that the official Rails example app has working caching which persists when correcting validation errors. Maybe it’s not easy for new users to understand how exactly is the cached file retained with the cached_attachment_data plugin, in which case a documentation could be improved.

I get nil when querying the facebook_image attribute

In the code above you didn’t actually persist the attachment assignment, so it’s expected that if you retrieve that record via a database query the attachment will remain unassigned.