Yes… build a solid mass of blocks however far underwater you want and then hollow it out from the bottom (leaving the walls and roof. When removing blocks they leave an air pocket… so if you protect it from being flooded you can make whatever you want… granted it can’t be open to the water from the top/sides… that would be bad hehe.
I would recommend using ether something easy to brake like glass (to speed things up) or something that has a chance for extra resources like leaves for starberries / logs for sap etc
Hope this helps =)