It has been quite a while since I went there. I was a day-tripper there mostly, so I don't know where to stay. You might check something like VRBO for non-hotel rentals as May is not high season for rentals and there might be lots available.
The most important thing is to get into all the info from CMBO (Cape May Bird Observatory)/NJ Audubon - cmbo.org. They have maps, site guides, activity schedules, links on where to stay, etc. I highly recommend the guided walks done by CMBO, and the boat trips in the salt marshes are great. CMBO also used to have one of the best birding and optics stores around - I assume they still do.
