Fort Taber Park

There is a lot of history here, including an outline of the original wooden structure (circa 1797), earthwork forts from the Civil War era (finished 1861), the large granite fort (construction stalled out in 1863 but for the most part finished later), military museum and tank, and so on. 

The main hallway through Battery Milliken and the hills themselves are all traversable. The small artillery hill mounts and bunkers, which have rooms to explore, are safe and the city cleans them up every so often. Then of course there is the largest granite fort, Fort Taber, which looks like a proper fortress, closed to the public except for events, such as pirate attack or Revolutionary reenactments.