I have been reading with interest the issues concerning young children riding in The Instructor. My niece is 4 years old, and small for her age, but loves horses and riding. We would sit her on a horse from the time she was able to sit up, being held by an adult for a short lead-line ride. When she was two we got her a toddler helmet and adjusted it to fit her head well, and someone would lead her around while I taught her how to hold her reins, correct leg position, and steering. At three she began competing in shows, assisted, of course, did well and won some ribbons. My question is this: Was doing this unsafe, and should I stop? I hate to make her stop riding now that she is addicted; nor do I want to do risky things.
I think what you are doing is just fine. While I cannot go so far as to say there is no risk or that it is a safe thing to do, you are mitigating the risk as much as possible and therefore doing it as safely as it can be done. You are knowledgeable and experienced and have looked at every avenue you can control and made the best decisions on horse, equipment and support.
Unfortunately not everyone has this much judgment and experience, so you can see why we can't come right out and say, "sure, it's fine for your toddler to ride, you go right ahead." There are many great instructors that specialize in riding and horseplay with young children. The most critical factor is that it is at least one-on-one and for children under five, there should be at least a two-to-one ratio of adults to child. At its best, riding is risky and for toddlers and children under six, it is even riskier because of their size and vulnerability and lack of coordination and judgment.
We take many risks daily around horses. What is important is that first, the risk is a worthwhile risk that we are benefiting from it in some way; and second, that we mitigate the risk as much as possible. For instance, while riding without your hands on the reins may be a worthwhile risk when certain safety considerations are met; it would be a pointless risk to not have a hand on the reins when you are standing around waiting for something. If there is not a direct benefit, don't take the risk. And if you take the risk, mitigate it by riding a safe horse, securing the reins, wearing a helmet, riding in a confined area, etc. Riding without reins certainly increases the risk to a rider but if we address the potential risks first, the fact that it will make a better rider of you makes the risk worth taking. Having enough experience to have the judgment to determine what the potential problems are in the first place is the part most people are lacking.
I must confess that my son started as a toddler, well, as a baby really; but like you, I looked at every possible risk. Except there was that one time when he was about three and we carefully planned out his first "trail" ride. Two people tacking one horse is never a good idea.... Sure enough, whichever one of us was supposed to tighten the girth before we hoisted Hunter up, didn't (and we routinely left the cinches loose when we first saddled, and waited until we were ready to mount the rider to tighten). With my trusted protégé ponying Hunter on his very nice older Welsh-Shetland cross, I was riding directly behind Hunter on my trusted mare, with the eyes of an eagle on its prey (or should I say, on its offspring).
Hunter squealed with delight on the ride and chattered and sang the whole way (one of the great joys of life is to watch a young child on their first ride bubbling over with joy). Approaching our first little hill, I ran my eyes over the pony to check the gear and to my horror, there was about two inches of daylight between the cinch and the horse's girth. Luckily a steady, smooth gaited and balanced pony kept the saddle from slipping and I rectified the situation post haste. It was a stupid mistake that could of caused a wreck but we got away with it. Anyway, the short answer to your question is, yes, you are doing it right. Good job! Keep it safe.
Post-script: Literally five minutes after finishing this article, I received a phone call from a woman who was looking for a horse for her daughter and was told I might be able to help. When questioned further, I found out that the family had never owned or really ridden horses other than at a trail barn and that they were looking for a horse for their three-year-old daughter to start barrel racing on. No kidding. You can see why CHA has to stick with the statement that toddlers and small children should not ride!