In order for resource mailboxes / room mailboxes to appear in Outlooks Room Finder they need to be added to a hidden distribution group as noted in this article.
Configure rooms and workspaces for Room Finder - Outlook | Microsoft Learn
Each group is formatted in the style "Birmingham Rooms" with a UPN of [email protected]
To add a room, create it in Exchange admin, then use the command:
Add-DistributionGroupMember -Identity [email protected] -Member [email protected]
You can view the members in a distribution group with the command: