AO3-7351 Deduplicate fandom tags for autocomplete

[tommilligan]

Pull Request Checklist

• Have you read "How to write the perfect pull request"?
• Have you read the contributing guidelines?
• Have you added tests for any changed functionality?
• Have you added the Jira issue number
  as the first thing in your pull request title (e.g. AO3-1234 Fix thing)
• Do you have fewer than 5 pull requests already open? If not, please wait
  until they are reviewed and merged before creating new pull requests.

Issue

https://otwarchive.atlassian.net/browse/AO3-7351

Purpose

When searching for tags in multiple fandoms, the backend currently retrieves cached results from redis for each fandom, adds them together, and returns them.

If a tag is present in multiple fandoms, this means the tag is returned multiple times, and the user sees duplicate values in the autocomplete UI.

This PR deduplicates tags in the backend when searching across multiple fandoms.

Testing Instructions

There are already testing instructions in JIRA.

Note that I was not able to test this via the browser locally (I'm looking into why), but I have only confirmed the behaviour is as expected via unit tests so far (the unit test I added fails against master)

If desired, I can add end to end cucumber tests.

References

Not aware of any other relevant issues.

Credit

What name and pronouns should we use to credit you in the Archive of Our Own's Release Notes?

Tom Milligan (they/them)

[tommilligan]

It was suggested in the ticket that deduplication be done on the frontend. I think it makes sense to do here and I don't see this having a massive perf impact, but wanted to call it out in case there are edge cases/concerns I'm not aware of

[Bilka2]

Not a full review, just responding here because I wrote that part of the issue: A change like this is what was meant! The autocomplete controller calls this method, so it is indeed de-duplicating just before responding in the autocomplete controller. Another option could have been to try to change the data that's in redis, which is what we didn't want

[tommilligan]

Ah, that makes much more sense - thanks for clarifying

Glad I got to the right place even if I couldn't read the original description 🥲