Weird bug with the virtual attribute definition

Hello again,

I’ve got a very weird bug in my dev environment. Accessing the virtual attribute defined via the uploader randomly fails. If I move the line include DocumentUploader::Attachment(:pdf) one line down it starts working again, then fails again on later rails console restart.

I disabled spring during my testing. First by issuing ./bin/spring stop then by export DISABLE_SPRING=1 before starting the rails console again.

If not working, and I issue a reload! inside rails console, it starts working again (having disabled spring with the above commands).

####  Migration file ####
class AddPdfDataToMyModels < ActiveRecord::Migration
  def change
    add_column :my_models, :pdf_data, :jsonb
  end
end


####  Model stuff ####
class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true
end

class MyModel < ApplicationRecord
  include DocumentUploader::Attachment(:pdf) # generates a virtual pdf column
end

class DocumentUploader < Shrine
end

####  Rails console testing ####
# This randomly fails with:
#   "NoMethodError: undefined method `pdf' for #<MyModel:0x000055e10a5a2eb8>"
MyModel.take.pdf

# This always succeeds
MyModel.take.pdf_data

Ok so I really wondered if that was just a spring bug, but I can reproduce the problem as described above (having to reload!), while confirming the spring process is not running via ps aux | grep spring

Trying to work around this issue, I removed the spring gem completely, and tried to use the attacher directly by feeding it my model instance and:

irb> DocumentUploader::Attacher.from_model(my_model_instance, :pdf)                                                           
# =>  NoMethodError: undefined method `[]' for nil:NilClass
# =>  from /cache/gems/gems/shrine-3.2.1/lib/shrine/plugins/model.rb:91:in `initialize'  


90 module AttacherMethods
91   def initialize(model_cache: shrine_class.opts[:model][:cache], **options)                                 
92     super(**options)
93     @model_cache = model_cache
94     @model       = nil
95   end

I don’t understand what happens behind the options hash.

@janko, would you have an idea what could be causing this failure??

Here is what my config looks like:

# ./config/initializers/shrine.rb
require "shrine"
require "shrine/storage/s3"

s3_options = {
  bucket:            my_bucket,
  region:            region,
  access_key_id:     access_key,
  secret_access_key: secret_key,
  endpoint:          endpoint_url,
}

Shrine.storages = {
  cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
  store: Shrine::Storage::S3.new(**s3_options),
}

Shrine.plugin :activerecord
Shrine.plugin :cached_attachment_data
Shrine.plugin :restore_cached_data
Shrine.plugin :download_endpoint, prefix: "attachments"
Shrine.plugin :derivatives
Shrine.plugin :determine_mime_type

I stripped my model and uploader of any logic, and printed to STDERR to observe the loading of the code on rails console start:

loading my_model...                                                                                                   
SHRINE WARNING: loading document uploader...   

Update: I’ve banged my head against that bug for about 2 hours today. I think this may be a Shrine bug.

Here is what I’ve been able to gather. Here Email is an ActiveRecord class that looks like this:

class Email < ApplicationRecord
  include DocumentUploader::Attachment(:file) # generates a virtual "file" column
end

I’ve been using Shrine to attach files to other similar models in my app with no problems.

But as soon as I introduced Shrine into an active record class making use of an ActiveSupport concern, then the shrine generated virtual columns break (randomly) for every ActiveRecord class that was previously working fine.

class Whatever < ApplicationRecord # not the Email class
  include DocumentUploader::Attachment(:pdf) # generates a virtual pdf column
  include MyConcern
end
module MyConcern
 extend ActiveSupport::Concern
 
 included do
   validates :whatever*

   belongs_to :whatever2
 end
end

Random behavior below, the fact the the “bug” disappears if I reload! is very odd.


Called from /cache/gems/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
Loading development environment (Rails 4.2.11.1)
[1] pry(main)> Email.take.file
NoMethodError: undefined method `[]' for nil:NilClass
from /cache/gems/gems/shrine-3.2.1/lib/shrine/plugins/activerecord.rb:27:in `included'
[2] pry(main)> Spring
NameError: uninitialized constant Spring
from (pry):2:in `__pry__'
[3] pry(main)> reload!
Reloading...
=> true
[4] pry(main)> Email.take.file
  Email Load (2.8ms)  SELECT  "emails".* FROM "emails" LIMIT 1
=> #<DocumentUploader::UploadedFile storage=:store id="113eca6be2fea03e0132eb4668d7c30e.eml" metadata={"size"=>335676, "filename"=>"d275c9f1bdf8e7dc444a7ceaac4e2515d4ba77cc.eml", "mime_type"=>"message/rfc822"}>

I remove the problematic included uploader as such:

class Whatever < ApplicationRecord # nothing to do with Email
  #include DocumentUploader::Attachment(:pdf) # generates a virtual pdf column
  include MyConcern
end

Then the previously broken code in Email is “fixed”

Called from /cache/gems/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
Loading development environment (Rails 4.2.11.1)
[1] pry(main)> Spring
NameError: uninitialized constant Spring
from (pry):1:in `__pry__'
[2] pry(main)> Email.take.file                                                                                             
  Email Load (2.7ms)  SELECT  "emails".* FROM "emails" LIMIT 1
=> #<DocumentUploader::UploadedFile storage=:store id="113eca6be2fea03e0132eb4668d7c30e.eml" metadata={"size"=>335676, "filename"=>"d275c9f1bdf8e7dc444a7ceaac4e2515d4ba77cc.eml", "mime_type"=>"message/rfc822"}>

@janko, do you have an idea what could be happening here?

Hi Benjamin, apologies for the late reply, I was busy with some other open source conquests.

This sounds like a reloading bug, but I don’t know what could be causing it. What I know about Zeitwerk and recent versions of Rails is that, when a class is being reloaded and evaluating the file raises an exception, Rails will swallow it and give you back the amount of class that was loaded.

So for example, in this case:

class Foo
  attr_reader :foo
  raise "some error"
  attr_reader :bar
end

you would get a class Foo with #foo reader but not #bar. This was the issue that I raised. This could be fixed in the latest version of Rails, allowing you to actually see the exception.

What I’m suspecting is that an error is happening while DocumentUploader is subclassing Shrine (in the inherited hook), and the DocumentUploader still ends up being defined, but the opts hash wasn’t copied from the Shrine class, causing it to not have some exepcted keys like you were seeing in your case.

This is just my guess on what could have happened, I hope it helps.

Thanks @janko for your feedback.

I was pressed on time, so I added an extra model/class with a one to one relationship to work around the issue.

For the moment, I’m fine as long as I don’t introduce shrine within a class using the “concern” feature of rails.

I also suspected something might be happening and related code autoloading from rails, so I ensured that the shrine stuff was right at the top, but that didn’t help.

I’ll report here if I find something useful but that’ll be in a while.

Also, for info that bug triggered in production for me so I know that it’s not related to the way code reloads in dev env.