I originally had an oil furnace in my basement. The chimney always dripped, the pipe kept rusting, and I had quite a bit of rusty concrete. A few years ago I had a gas furnace installed. I tried capping if off in the basement after installation of the new furnace and had a large cover put on the top of the chimney to try to keep out rain. Last summer when I had 2 men on the roof repairing shingle loss, I also hired them to take out the chimney. Did I do something wrong. Was that connected in any way to my furnace to let in fresh air or was it, as I thought, an air escape thing? My gas furnace is vented out the back of my house.
it sounds like you are ok with removing your chimney (if that was the only use for the furnace) and your new furnace sounds like a "direct vent" and gets its own air from out side, but check with your heating contractor. your water problem should be solved?