Lots of files in the buckets that should have been deleted

Hi, I just noticed that there are thousands of files in both cache and store in my bucket, although there should be less than 100 attachments. It’s like it has never deleted lots of files due to lots of testing of things over time…

I have these in the initializer:

Shrine::Attacher.promote_block do
  PromoteJob.perform_later(self.class.name, record.class.name, record.id, name, file_data)
end
Shrine::Attacher.destroy_block do
  DestroyJob.perform_later(self.class.name, data)
end

Shrine.mirror_upload_block do |file|
  MirrorUploadJob.perform_async(file.shrine_class.name, file.data)
end

Shrine.mirror_delete_block do |file|
  MirrorDeleteJob.perform_async(file.shrine_class.name, file.data)
end

What could explain the missing deletion of those files? Also, is there a way to find out which files should have been deleted, and clean them up in one shot? Of course I want to fix the issue first. Thanks in advance

Have you created a background worker and verified its running the PromoteJob, DestroyJob, MirrorUploadJob, and MirrorDeleteJob successfully? Is it showing any errors when trying to delete the files?

Hi! It’s super weird because I checked and all the background jobs seem to work fine, including deletion and mirroring. I have no idea of what happened and when. How can I clean up the storage by removing the orphans? Thanks!

here’s one idea: you could mark the legitimate files with some type of meta data, and then simply iterate over the whole collection and delete those orphans which do not have the meta data?

I’ll see if I can come up with something like that. Thanks for the suggestion!