Adds support for AWS OpenSearch Serverless (AOSS)

[JPrevost]

Why are these changes being introduced:

• We are planning a migration to AOSS
• We need to maintain our existing AWS OpenSearch Service (ES) integration while migrating to AOSS

Relevant ticket(s):

• https://mitlibraries.atlassian.net/browse/USE-423

How does this address that need:

• Added support for AWS OpenSearch Serverless (AOSS) using either expiring credentials by passing a session token or by assuming a role.

• Configured the application to support AWS OpenSearch Serverless (AOSS) in addition to the existing AWS OpenSearch Service (ES).

• Added logic to choose the appropriate client based on environment variables.

• Implemented AWS SigV4 signing for AOSS authentication.

Document any side effects to this change:

• Updated lambda configuration to support session tokens. It does not have assume role configuration at this time, but we needed to support temporary credentials in the lambda to support them locally in OpenSearch Serverless (AOSS) so I included that in this change.

Developer

• All new ENV is documented in README
• All new ENV has been added to Heroku Pipeline, Staging and Prod
• ANDI or Wave has been run in accordance to
  our guide and
  all issues introduced by these changes have been resolved or opened as new
  issues (link to those issues in the Pull Request details above)
• Stakeholder approval has been confirmed (or is not needed)

Code Reviewer

• The commit message is clear and follows our guidelines
  (not just this pull request message)
• There are appropriate tests covering any new functionality
• The documentation has been updated or is unnecessary
• The changes have been verified
• New dependencies are appropriate or there were no changes

Requires database migrations?

NO

Includes new or updated dependencies?

YES

[Copilot]

Pull request overview

This PR adds dual support for AWS OpenSearch Service (ES) and AWS OpenSearch Serverless (AOSS) so the app can authenticate appropriately during a staged migration.

Changes:

• Add AOSS support via opensearch-aws-sigv4 and STS assume-role / session-token based credentials.
• Select the OpenSearch client implementation at boot based on AWS_AOSS / AWS_OPENSEARCH.
• Update Lambda client configuration to optionally include AWS_SESSION_TOKEN.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 4 comments.

File	Description
Gemfile	Adds dependencies needed for AOSS SigV4 + STS assume role.
Gemfile.lock	Locks added AWS/OS SigV4 + STS dependencies.
config/initializers/opensearch.rb	Adds AOSS client + credentials selection logic; updates AWS ES client config.
config/initializers/lambda.rb	Adds session token support for Lambda client credentials.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

aws_aoss_client passes OPENSEARCH_URL directly as the host: option. In this repo OPENSEARCH_URL is documented as a full URL (scheme + host + optional port), while host: typically expects just the hostname (and separate scheme/port). This will likely break AOSS connections when OPENSEARCH_URL includes https://.... Consider parsing OPENSEARCH_URL (e.g., via URI) and passing host/scheme/port separately (or using the option that accepts a full URL if supported).

[JPrevost]

This is incorrect. It works with the full URL and the examples show a full URL:
https://github.com/awsdocs/amazon-opensearch-service-developer-guide/blob/master/doc_source/serverless-clients.md#ruby

I do agree that host is a poor name if it actually means url, but we are indeed using as documented.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[matt-bernhardt]

I've been able to see these modes running locally after cleaning up my local ENV, and a read through of the code seems like it makes sense to me. I've got one non-blocking comment about the approach to testing here, but feel free to ignore that if it isn't something you agree with.

In general, I can confirm this works, and I think the re-arranged readme should be easier for future-us. Thanks!

[matt-bernhardt]

Comment:

This is an intriguing idea - thanks for proposing it. I'm intrigued by how this will feel moving forward, but I could imagine that this way of ordering might make it easy to onboard someone to using the tool, or ensuring that we're catching all the related env vars if something needs to change.

[matt-bernhardt]

Suggestion - not a required change:

I see this and all the prior tests as exercising the various conditions that we think validate_aws_aoss_config would be confronted with, and it seems correct. My question is whether there's a risk of keeping all these tests in sync if we end up needing to add new parameters? Do these tests feel more maintainable if we define a helper method in the test suite that defined everything needed, which we then proceed to break in different ways for individual tests?

I'm thinking of something like

def setup_aoss_correctly
  ClimateControl.modify(
    OPENSEARCH_URL: 'good value',
    AWS_REGION: 'good value',
    ...
  )
end

test 'happy path' do
  setup_aoss_correctly
  
  assert_nil OpensearchConfigValidator.validate_aws_aoss_config
end

test 'break one thing' do
  setup_aoss_correctly

  ClimateControl.modify(
    OPENSEARCH_URL: nil
  )

  error = assert_raises(RuntimeError) do
    OpensearchConfigValidator.validate_aws_aoss_config
  end

  assert_match(/OPENSEARCH_URL/, error.message)
end

... etc

This might be too clever, and it just repeats what you've written for the moment, but it feels easier to keep all the parameters in sync this way as conditions change (if they're likely to - maybe this is all stable and its fine)