How dumb can you get

So, before this years muster, I install a DC/DC charger for my AGM's which run my fridge that sits in the back of the ute. I have 3 AGM's so when one goes down I switch over to another one, that was before the DC/DC charger install.
With the DC/DC charger I thought while I'm driving around, it would put charge into the battery and it will, but here's the problem.
The kit I bought has an Anderson plug between the crank battery and the charger.
Now I have to say, the fridge ran ok for a week on one battery (it was plugged into the big one) before I had to switch over but it had me buggered why the next battery wasn't taking a charge.
So today I check the wiring and there's no problem except for one LITTLE/BIG thing. I had disconnected the Anderson plug at the crank battery. I figured if I wasn't charging another battery, then why have it connected.
I've just learnt a valuable lesson

Clifford