Am i suppost to boil water for baby formula?

well, we don't got any of that good water, that comes in those big jugs, or bottled water, and I'm babysittinq my little nephew and he's still pretty small. he was born 2 months ago . but yeauh. i heard tap water can make them sick or something? so..am i suppost to boil the water for a few minutes??