I have heard that homeschooling is expensive. Is this true?

No.  Your greatest expense out of pocket is for your curriculum.  With all of the options now for used curriculum and learning units on educational websites, you can tailor your curriculum to meet the needs of each of the children you are teaching.  You may spend as much or as little as you feel you need to in order to fulfill the learning needs.