Question about saving files from uppy and the order in which derivatives are created

My implementation for direct s3 uploads goes ever so slightly further than what is described in the official Shrine doc, but the gist of it is that I’m using Uppy’s Dashboard, S3, and Form modules to insert a json encoded string into a hidden input element. This data is then saved as a text field upon form submission, but I can only pass in the data for the original file. The question I have is about the order in which derivative files are created.

My uploader has plugin :derivatives, create_on_promote: true declared, which worked perfectly before. For some reason, rolling my MediaFile and Item creation actions into one made it stop working as intended.

My Item create action has this:

def create
 @item = current_user.owned_items.create(item_params)
 successful_results.each do |file|
  file_data_id = file["meta"]["key"]
  file_data_id.sub!("cache/", "store/")
  filename = file["name"]
  size = file["size"]
  mime_type = file["type"]
  MediaFile.create(fileable_type: "Item", fileable_id: @item.id, file_data: "{\"id\":\"#{file_data_id}\",\"storage\":\"store\",\"metadata\":{ \"filename\":\"#{filename}\",\"size\":\"#{size}\",\"mime_type\":\"#{mime_type}\" }}")
 end
end

They’re not created when the files are saved, and the entirety of the derivatives field is nil. Calling file_derivatives! after MediaFile.create() causes this error: Shrine::FileNotFound (file "store/b021c53720b2699c399695f11e98e573.jpg" not found on storage).

How do I make derivatives from S3 upload data after saving it to my app?

It’s a bit hard to tell the issue without additional context (e.g. where successful_results is coming from). But the exception seems legit and not related to derivatives; it’s saying that the source file from which you’re attempting to create derivatives cannot be found on the storage service (Shrine internally downloads the attached file and passes it to the Attacher.derivatives block when you call file_derivatives!).

1 Like

successful result is the array of successful json uploads. looks like:

def successful_results
 results = JSON.parse(params["uppyResult"])
 results[0]['successful']
end

the param comes from a hidden input element named “uppyResult”

after further investigation, you are very correct. the file is being uploaded to the cache, but is not propagating to store

sorry to keep bothering about this but

if the file is saving to cache how do i get it over to store? my create action and successful_results method are above, and this is my uploader:

class MediaFileUploader < Shrine
	require "image_processing/mini_magick"
	plugin :pretty_location
	plugin :derivatives, create_on_promote: true
	plugin :validation
	plugin :validation_helpers

	s3_dev = { 
     bucket: ENV['S3_BUCKET_NAME_DEV'],
	 region: ENV['AWS_REGION'],
	 access_key_id: ENV['AWS_ACCESS_KEY_ID'],
	 secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
	}
    self.storages = { 
	 cache: Shrine::Storage::S3.new(prefix: "cache", **s3_dev), 
	 store: Shrine::Storage::S3.new(public: true, upload_options: { acl: "public-read"}, **s3_dev),
	}

	Attacher.validate do
     validate_max_size 10.megabytes, message: 'must be smaller than 10MB'
     validate_mime_type %w[image/jpeg image/png]
     validate_extension %w[jpg jpeg png]
    end

    Attacher.derivatives do |original|
     magick = ImageProcessing::MiniMagick.source(original)
     {
      small: magick.resize_and_pad!(225, 220),
	  medium: magick.resize_to_limit!(1000, 1000)
     }
   end
end

update:

Oddly, saving the file data through AJAX with each successful upload in the uppy Javascript file works. The new file is uploaded to cache and persisted to the official storage and all derivatives are created.

a.on('upload-success', (file, response) => {
 const uploadedFileData = {
  id: file.meta['key'].match(/^cache\/(.+)/)[1], // object key without prefix
  storage: 'cache',
  metadata: {
   size: file.size,
   filename: file.name,
   mime_type: file.type,
  }
 }

 $.ajax({
  type: 'POST', 
  url: 'media_files',
  data: { 
   media_file: {
    fileable_type: 'Item',
    fileable_id: fileable_id,
    file: uploadedFileData
   }
  }
 })
})

I thought maybe this has something to do with the format for uploadedFileData but it does not. uploadedFileData is formatted identically to the way that it is formatted in my controller.