Guesthouses may provide organised tours by car for USD20 per day. Taxis are also common and not different from the ones in Hanoi or Saigon, although a little cheaper.
A motorbike is the best way to view and the surrounding area. A motorbike can be rented for about 100,000 dong per day, a motorcycle driver/guide to take you around the sites for about USD10-20 a day.
The roads in general are quite well maintained and not overly busy, but the junctions, just like everywhere else in Vietnam, can be dangerous. Renting your own motorbike is possible but not without risk.
Cycling is a good option for visiting nearby sights. Guesthouses hire out bicycles for about 40,000 dong/day. There is a country road (mountain bike is recommended) that connects Hoa Lu and Tam Coc.
The town of Ninh Binh is quite small and walkable. However, most of the interesting sights are outside the city.