Use validations and location from Shrine class for presigned endpoints

Hi All,

I have successfully managed to upload a file using the presign_endpoint module described in the docs.

The issue for me is, I wanted to somehow presign this using the settings from a Shrine class, and ideally using the validate methodings as well for example:

class AttachmentUploader < Shrine
  plugin :default_url
  plugin :activerecord, callbacks: false
  plugin :determine_mime_type
  plugin :validation_helpers
  plugin :presign_endpoint

  Attacher.validate do
    validate_min_size 1, message: "must not be empty"

    validate_max_size Rails.configuration.attachment_max_mb_size.to_i.megabyte, message: "is too large (max is #{Rails.configuration.attachment_max_mb_size.to_i.megabyte} MB)"
    validate_mime_type_inclusion %w[application/pdf image/png image/jpeg text/csv text/plain], message: "We do not support that file type"
  end

  Attacher.default_url do |options|
    "profiles/attachments/#{name}/default.jpg"
  end

  def generate_location(io, context = {})
    profile = context[:record].profile
    "profiles/#{user_uuid}/#{Rails.configuration.attachment_dir}/#{super}"
  end
end

If this isn’t possible, is there a custom way of doing this, for instance defining the location of the file based on the request params?

Thanks in advance for any advice

This is the basic presign that I have working right now:

initializer:

Shrine.plugin :presign_endpoint, presign_options:  -> (request) {
  {
    content_length_range:   0..(10*1024*1024)                   # limit upload size to 10 MB
  }
}

controller endpoint:

def presign
    status, content, presigned_post = AttachmentUploader.presign_response(:store, request.env)
    render json: presigned_post.first , status: status
  end

Just an update, I have now removed the initializer as I realized wasn’t required. so the presign_endpiont plugin is now only loaded within the AttachmentUploader:

class AttachmentUploader < Shrine
plugin :presign_endpoint, presign_options: -> (request) do
	  filename     = request.params["filename"]
	  extension    = File.extname(filename)
	  content_type = Rack::Mime.mime_type(extension)

	  {
	    content_length_range: 0..(10*1024*1024),                     # limit filesize to 10MB
	    content_disposition: "attachment; filename=\"#{filename}\"", # download with original filename
	    content_type:        content_type,                           # set correct content type
	  }
	end
end

The location returned by aws still doesn’t obey the generate_location method though.

hi @abepetrillo

A script to reproduce this would be extremely helpful in debugging because it communicates precisely where the problem could/might lie.

Questions:

  1. Are you using the latest verison of Shrine from the master branch? If not perhaps try the latest version because there were some temporary fixed introduced.
  2. as a secondary question: is it failing when you are using uppy? are you using a custom uploader? How are you calling the presigned endpoint?

Your link to the activerecord script doesn’t include things like Rack or the aws gem to set all this up. I’m looking into this now but really the heart of the issue is how does the presign_endpoint plugin gather the context required for MyUploader.generate_location ?

require "active_record"
require "shrine"
require "shrine/storage/memory"
require "shrine/storage/s3"
require "shrine/plugins/presign_endpoint"
require "down"
require 'pry'
require 'rest-client'
Pry.config.input = STDIN
Pry.config.output = STDOUT

Shrine.storages = {
  cache: Shrine::Storage::Memory.new
}

Shrine.storages[:store] = Shrine::Storage::S3.new(
  bucket:            ENV['S3_BUCKET_PRIVATE'],
  access_key_id:     ENV['S3_KEY_PRIVATE'],
  secret_access_key: ENV['S3_SECRET_PRIVATE'],
  region:            "us-east-1"
)

Shrine.plugin :activerecord


class MyUploader < Shrine
  # plugins and uploading logic

  def generate_location(io, context = {})
    "custom/location/#{super}"
  end

  plugin :presign_endpoint, {
    presign_options: -> (request) do
  	  filename     = request.params["filename"]
  	  extension    = File.extname(filename)
  	  content_type = Rack::Mime.mime_type(extension)

  	  {
  	    content_length_range: 0..(10*1024*1024),                     # limit filesize to 10MB
  	    content_disposition: "attachment; filename=\"#{filename}\"", # download with original filename
  	    content_type:        content_type,                           # set correct content type
  	  }
    end
  }
end

ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Base.connection.create_table(:posts) { |t| t.text :image_data }

class Post < ActiveRecord::Base
  include MyUploader::Attachment(:image)
end

post = Post.create(image: Down.download("https://images.pexels.com/photos/2607443/pexels-photo-2607443.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260"))
puts "without s3 location:"
puts post.image_data
puts "with s3 location"
env = {
        "REQUEST_METHOD" => "GET",
        "SCRIPT_NAME"    => "",
        "PATH_INFO"      => "/s3/params",
        "QUERY_STRING"   => "filename=foo.txt",
        "rack.input"     => StringIO.new,
      }
status, content, presigned_endpoint = MyUploader.presign_response(:store, env)
endpoint = JSON.parse(presigned_endpoint.first)
fields = endpoint["fields"]
url = endpoint["url"]
puts "url: #{url}"
RestClient.post(url, fields.merge("file" => Down.download("https://images.pexels.com/photos/2607443/pexels-photo-2607443.jpeg?auto=compress&cs=tinysrgb&h=750&w=1260")))
# This is where I am stuck? Where is the post related to this file? What is the developer expected to do to generate a post?
# Also observe that the file is in the bucket, but not in "/custom/location"

This is all sorted via the Github issue?

I dont see how my question at the end of the script is answered in the issue.

Apologies, I missed the question you wrote at the end of the script.

By “post”, you mean a Post record in your example? When you’re doing direct uploads, uploads are decoupled from attachment or record creation. After direct upload you assign JSON data referencing the uploaded file when creating the record, e.g:

...
RestClient.post(url, fields.merge("file" => Down.download("..."))

Post.create(image: {
  id: fields["key"], # strip away storage prefix if there is any,
  storage: :cache,
  metadata: {
    ...
  }
})

The cached file will then automatically be copied to permanent storage, which will then use the location defined by Shrine#generate_location.

Thanks, that clears things up. In this case cache is an s3 storage instance correct?

Thanks, that clears things up. In this case cache is an s3 storage instance correct?

  • “cache” refers to the storage identifier you specified in shrine. it is the name of the s3 storage instance you have specified to shrine - which is then mapped to a bucket name.