Perhaps you are using light weights. Generally people will just add on massive amounts of weight, which is the problem
I have two sets of weights...5lbs for each ankle. When the 10lb set began getting too light, I attempted to double up (the 10lb per ankle sets I found were miserable bulking/uncomfortable, I love the fabric on the ones I have). When I doubled up, they dug into my ankles so badly that I had to take 2 days off to heal. So, now, I just walk with 5lbs an ankle.
Ya if you do use it, do not go heavy, just treat it as a light workout. Standing enough is a lot of workout for your legs.