Shrine

File type is stuck on application/octet-stream when saving JSON to S3

Question
#1

Here is a method from my model that stores a JSON table of contents from a title.

  def create_toc_json
    filename = xml_identifier + '_toc.json'
    source = self.sources.where(file_name: filename).first_or_initialize
    toc = ApplicationController.new.render_to_string('titles/toc', formats: [:json], locals: { title: self })
    source.file_attacher.assign(StringIO.new(toc, 'rb'), metadata: { 'filename' => filename })
    source.save!
  end

When I run it in console, I get a successful save, but the file is stored on S3 with the content-type application/octet-stream. That is also reflected in my database save:

2.7.4 :011 > Title.first.send :create_toc_json
  Title Load (0.5ms)  SELECT `titles`.* FROM `titles` ORDER BY `titles`.`id` ASC LIMIT 1
  Source Load (0.4ms)  SELECT `sources`.* FROM `sources` WHERE `sources`.`original_id` = 1 AND `sources`.`original_type` = 'Title'
  Source Load (0.5ms)  SELECT `sources`.* FROM `sources` WHERE `sources`.`original_id` = 1 AND `sources`.`original_type` = 'Title' AND `sources`.`file_name` = 'Ricardo_0687-02_toc.json' ORDER BY `sources`.`id` ASC LIMIT 1
  Rendering titles/toc.json.jbuilder
  Heading Exists? (0.2ms)  SELECT 1 AS one FROM `headings` WHERE `headings`.`title_id` = 1 LIMIT 1
  Heading Load (0.6ms)  SELECT `headings`.* FROM `headings` WHERE `headings`.`title_id` = 1 AND `headings`.`visible` = TRUE
  Rendered titles/toc.json.jbuilder (Duration: 22.4ms | Allocations: 20026)
MIME Type (0ms) – {:io=>StringIO, :uploader=>FileUploader}
Metadata (0ms) – {:storage=>:cache, :io=>StringIO, :uploader=>FileUploader}
Upload (255ms) – {:storage=>:cache, :location=>"titles/61e6e8e192ba347e8c86b4bfd383b4b1.json", :io=>StringIO, :upload_options=>{}, :uploader=>FileUploader}
   (0.2ms)  BEGIN
  Source Create (4.3ms)  INSERT INTO `sources` (`file_name`, `file_size`, `file_data`, `created_at`, `updated_at`, `original_type`, `original_id`) VALUES ('Ricardo_0687-02_toc.json', 7320, '{\"id\":\"titles/61e6e8e192ba347e8c86b4bfd383b4b1.json\",\"storage\":\"cache\",\"metadata\":{\"filename\":\"Ricardo_0687-02_toc.json\",\"size\":7320,\"mime_type\":\"application/octet-stream\"}}', '2021-11-10 11:49:16.882978', '2021-11-10 11:49:16.882978', 'Title', 1)
  Source Update (1.3ms)  UPDATE `sources` SET `sources`.`content_format` = 'JSON' WHERE `sources`.`id` = 31274
   (1.5ms)  COMMIT
Upload (78ms) – {:storage=>:store, :location=>"titles/1/Ricardo_0687-02_toc.json", :io=>FileUploader::UploadedFile, :upload_options=>{}, :uploader=>FileUploader}
   (0.2ms)  BEGIN
  Source Update (0.4ms)  UPDATE `sources` SET `sources`.`file_data` = '{\"id\":\"titles/1/Ricardo_0687-02_toc.json\",\"storage\":\"store\",\"metadata\":{\"filename\":\"Ricardo_0687-02_toc.json\",\"size\":7320,\"mime_type\":\"application/octet-stream\"}}', `sources`.`updated_at` = '2021-11-10 11:49:16.971915' WHERE `sources`.`id` = 31274
  Source Update (0.2ms)  UPDATE `sources` SET `sources`.`content_format` = 'JSON' WHERE `sources`.`id` = 31274
   (0.5ms)  COMMIT
 => true 
2.7.4 :012 >

Rails 6, Shrine 3.4.0, Ruby 2.7.4.

My attacher loads the following plugins:

  plugin :remove_attachment
  plugin :metadata_attributes, filename: :name
  plugin :determine_mime_type

My Shrine initializer also loads

Shrine.plugin(:activerecord)
Shrine.plugin(:cached_attachment_data)

Where can I start picking at this to ensure that the files have the correct type on S3?

Thanks,

Walter

#2

Update to this post. I was looking at the wrong attacher for my plugins. The problem is the same, just updating my post to include the correct plugins:

  plugin :remove_attachment
  plugin :metadata_attributes, filename: :name, size: :size
  plugin :determine_mime_type, analyzer: :marcel
  plugin :refresh_metadata

This is the code that is currently producing the incorrect output mime.

Walter

#3

has this been solved?

#4

No, you’re the first person to reply. I’ve just been putting up with it. It hasn’t appeared to keep my clients from using it, but it is odd, and not what I’ve seen in the past with Shrine.

#5

Is it possible that Marcel is not correctly detecting JSON content? You should be able to override the mime_type on assignment by passing the mime_type metadata:

source.file_attacher.assign(StringIO.new(toc, 'rb'), metadata: {
  'filename' => filename,
  'mime_type' => 'application/json'
})
#6

Hey Walter, got held up with other issues. I’ll have a whip up a script to reproduce it. i don’t want to guarantee a time it will be done though