When Galen Miller wants to visit the South Carolina cemetery where much of his ancestry is buried, he must get permission to enter Hilton Head Plantation, a private, gated community on the north end of Hilton Head Island, where the wealthy jostle for tee times between trips to the beach and sips of gin and tonics.

On a recent July morning, a guard waved Mr. Miller through the gate. He meandered past multimillion-dollar homes and lush green fairways to the Talbird Cemetery, now squeezed between a scenic creek and a luxury condominium.

“To be able to come here and visit my family from time to time, it really touches your heart,” Mr. Miller said. “It also saddens you to see what’s happening.”

Mr. Miller is Gullah Geechee, a community descended from enslaved West Africans who were forced to work the cotton and rice plantations of the Carolina Lowcountry. The Gullah Geechee are now struggling to preserve sacred traditions as wealthy northerners swallow up valuable waterfront property.