[SOLVED] Problem validating a cached file before promoting to storage

This is for Shrine version 2.19.3.

Here’s what I’m trying to do:

  • User uploads a cached file directly to S3
  • They then post this cache ID to a controller, and I create a record in the db.
  • I want to validate that the file size is less than 100 MB before I promote the cached file to storage.

Shrine Config:


Shrine.plugin :activerecord
Shrine.plugin :instrumentation
Shrine.logger = Rails.logger
Shrine.plugin :cached_attachment_data
Shrine.plugin :backgrounding
Shrine.plugin :copy

# Handle parsed JSON for API endpoints
# See: https://github.com/shrinerb/shrine/issues/335
Shrine.plugin :parsed_json

Shrine.plugin :validation    # <= this is crashing

Controller:

      def create
        @file = MyCustomFile.new(
          file: {
            'id' => params[:cache_id],
            'storage' => 'cache',
          }
        )
        @file.save
        ...

Model:

class MyCustomFile < ApplicationRecord
  include CustomFileUploader.attachment(:file)
  
  ...

Uploader:

# frozen_string_literal: true

class CustomFileUploader < Shrine
  plugin :delete_raw
  plugin :determine_mime_type
  plugin :add_metadata
  plugin :upload_endpoint
  plugin :presign_endpoint

  # Cache file from presigned upload route has Content-Type: binary/octet-stream
  # We need to replace this metadata when promoting from cache to store.
  # See: https://github.com/janko-m/shrine/pull/105
  plugin :upload_options,
         store: lambda { |io, _context|
           if io.is_a?(Shrine::UploadedFile)
             { metadata_directive: 'REPLACE' }
           end
         }

  plugin :validation_helpers
  plugin :remove_invalid
  Attacher.validate do
    validate_max_size 100 * 1024 * 1024 # 100 MB
  end
end

Crash:

I’m getting a crash in my test suite, because the metadata is not being generated:

    Failure/Error: validate_max_size 100 * 1024 * 1024 # 100 MB

     NoMethodError:
       undefined method `<=' for nil:NilClass
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins/validation_helpers.rb:54:in `validate_max_size'
     # ./app/uploaders/custom_file_uploader.rb:23:in `block in <class:CustomFileUploader>'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/attacher.rb:92:in `validate'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins/remove_invalid.rb:11:in `validate'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/attacher.rb:86:in `set'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/attacher.rb:222:in `assign_cached'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/attacher.rb:72:in `assign'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins/parsed_json.rb:14:in `assign'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins/rack_file.rb:67:in `assign'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins/parsed_json.rb:12:in `assign'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins/rack_file.rb:67:in `assign'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/attachment.rb:49:in `block in define_attachment_methods!'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/attr_encrypted-3.1.0/lib/attr_encrypted/adapters/active_record.rb:28:in `perform_attribute_assignment'
     # /Users/myuser/.rvm/gems/ruby-2.5.5/gems/attr_encrypted-3.1.0/lib/attr_encrypted/adapters/active_record.rb:36:in `assign_attributes'
     # ./app/controllers/api/v1/custom_files_controller.rb:39:in `create'

I’m also looking at the docs for the validation plugin, and it says I should call Shrine.plugin :validation. So I tried that, in case this was the missing piece. But then I get this error when starting the application:

cannot load such file -- shrine/plugins/validation (LoadError)
	from /Users/myuser/.rvm/gems/ruby-2.5.5/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:291:in `block in require'
	from /Users/myuser/.rvm/gems/ruby-2.5.5/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:257:in `load_dependency'
	from /Users/myuser/.rvm/gems/ruby-2.5.5/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:291:in `require'
	from /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine/plugins.rb:15:in `load_plugin'
	from /Users/myuser/.rvm/gems/ruby-2.5.5/gems/shrine-2.19.3/lib/shrine.rb:85:in `plugin'
	from /Users/myuser/code/myapp/config/initializers/shrine.rb:150:in `<main>'

I’m also looking at the Validation Helpers docs, so I also have plugin :validation_helpers in each of the uploaders that requires validations.

Do I need this Shrine.plugin :validation line in my initializer, or do I just need plugin :validation_helpers?

Ahhhhh, I see the validation plugin wasn’t added for v2.19.3: https://github.com/shrinerb/shrine/tree/v2.19.3/lib/shrine/plugins

I always find the Shrine docs through a Google search, and it always takes me to the Markdown files on GitHub’s master branch, so that’s a bit annoying. It would be great if the docs on the master branch could be for the latest stable version, and maybe work on release candidates on a separate branch, to avoid confusion.

I’m also worried about updating to 3.0.0.rc, because I’m not sure what to do about the copy plugin. (See my other post.) But I’ll update and see if I can figure it out.

Yeah, sorry about the confusion with the docs. Lots of things have changed for 3.0, which is about to get released. Starting with the 3.0.0 release, we’ll also be publishing docs to a new website, which will always contain the latest released version (instead of master), so that should resolve the confusion.

There error inside the Attacher.validate block sounds like the size metadata is absent, which would be the case if it wasn’t sent from the frontend. You could try loading the restore_cached_data plugin.

1 Like

Thanks very much, the restore_cached_data plugin is what I was looking for! That fixed it, and now the size validations are working correctly.