Update Framework to Rails 7.1

[awilfox]

This is The Big One™, updating Framework to Rails 7.1. Change details are below. In summary:

• Good Job 4.x requires us to move to an intermediate 3.x release first.
• Rails 7.2 (or 8.0) requires us to move to 7.1 first (for cache and db migration layer changes).

Unfortunately that means this is not the last "big one". It will however be the most invasive as far as I can tell.

I've brought in a number of reviewers because I'm not sure who will have the most time to review these changes. I think two or three will be enough, but the more the merrier.

---

Dockerfile:

• Replace libvips42 with libyaml-dev

  We have never used VIPS in Framework; this was mistakenly copied from UCBEARS. We need libyaml-dev for the psych gem that Rails 7.1's required irb pulls in.

• Update Bundler to 2.7.2

  Otherwise, we hit a lot of duplicate constant errors when bundling.

Gemfile:

• Remove the Gems that Rails explicitly requires now

  We only had drb and mutex_m to avoid Ruby 3.3 deprecation warnings.

• Update Good Job to at least 3.25

  3.25 is the minimum for Rails 7.1 compatibility. We pin to lower than 3.99.0 because we aren't ready to migrate to 4.x yet.

• Update Rails to 7.1.6

• Change mingw mswin x64_mingw to windows

  This is part of the Bundler 2.7 update.

• Update database_cleaner to at least 2.1

  This is the minimum version for Rails 7.1 compatibility.

• Update RSpec to 3.13

  This is required for the receive(…).with(…) syntax to work correctly on Rails 7.1.

• Update RSpec-Rails to 6.1

  This update is necessary for Rails 7.1 compatibility and removes multiple deprecation warnings.

TindDownloadController:

• Add _options param to the cache fetch block

  This is a new addition in Rails 7.1.

jobs:

• Don't require 'request_mailer'

  Rails 7.1 no long adds autoloaded paths to $LOAD_PATH. The documentation clearly states you should not be requiring files autoloaded anyway, so stop doing that.

config:

• Updates for Rails 7.1 compatibility

  Note that after this is live, rolling back to Rails 7.0 will require invalidation of cache as the cache format on-disk has changed with 7.1 defaults.

db:

• Update Good Job schema

  This is required to update past 3.15.3.

spec:

• Completely rewrite async_job_context

  Good Job has had a lot of internal rework, and the old way we were doing this wasn't working. The new way uses a lot fewer "private" APIs, and hopefully this new code will serve us well as we continue to update.

Closes: AP-561
Ref: AP-270

[awilfox]

v2: Update simplecov-rcov as well to fix CI run error.

[steve-sullivan]

Updating Good Job alone looked tedious, then throw in everything else. Brilliant job. I see a few files that were added where all lines are commented out (I'm assuming those are some new standard template files for Rails....?)

[awilfox]

Updating Good Job alone looked tedious, then throw in everything else. Brilliant job. I see a few files that were added where all lines are commented out (I'm assuming those are some new standard template files for Rails....?)

Thank you! 😺

Yes, those are new default initialisers that Rails includes now. rails app:update adds them automatically.

[anarchivist]

👏 r+!

[yzhoubk]

Nice work, Anna! This is not an easy task. I pulled the branch and tried a Bibliographic test job locally, I saw this error: " ArgumentError: SMTP to address may not be blank: [].". Actually, this should be ok locally, I just comment it out. It works just fine.

[awilfox]

@yzhoubk I think we should add a validation on the Bibliographics::HostBibTask model that ensures email is set properly. That would give us better info to debug that failure, and also is a good idea in general, since that job will always fail without a valid email.

I will file a ticket and add that validation separate from this change, since I don't think anything in Rails 7.1 has changed how we handle these jobs.