The Alapaha River goes underground because the underlying karst limestone rises in what’s called the Cody Scarp, which runs across north Florida. Other rivers that go underground there include the Little Alapaha River and the Santa Fe River. The Withlacoochee River does not go underground, but it does sprout Madison Blue Spring.
Here’s a cutaway diagram of how all that works underground:
This map shows dozens of springs in the area of the Cody Scarp, and how the Floridan Aquifer is semiconfined just above and unconfined below the Cody Scarp, including in southern Brooks and Lowndes Counties, Georgia, as well as in Madison, Hamilton, and Suwannee Counties, Florida.
This map shows the Cody Scarp as the brown line with the rivers named, plus the geologic age of the underlying rocks.
The Cody Scarp continues to the west outside the Suwannee River Basin, passing through the Aucilla, Wacissa, St. Marks, Ochlockonee, and Apalachicola river watersheds below the Red Hills north of Tallahassee.
For much more detail about the Cody Scarp, see Geomorphic Influence of Scarps in the Suwannee River Basin, Southeastern Geological Society, Field Trip Guidebook 44, 7 May 2005.