You are right, the sides do suck in when the temperature drops and this will reabsorb some air; but not much. A 20 litre vessel will suck back what? half a litre? 1 litre being really generous? so 5% (this is an estimation but please let me know if you disagree, let me know how much suckback you get). If the full 5% volume of CO2 was reabsorbed into 20 litres that would increase the volumes of CO2 from (lets say) 1.5 to 1.575 an increase of just 0.075 volumes.
At 4C there are around 1.48 Volumes
At 20C there are around 0.87 Volumes
I used beersmith's priming calculator for these numbers but I presume most calculators will be similar. If you put 20 litres and 4C or 20C then the target volume, the amount of sugar necessary to get to these volumes will be almost zero. Maybe a gram or so descrepancy, feel free to use your own priming calculator.
Which means the increase in reabsorbtion (of 0.075 volumes) would increase the volumes of CO2 from 0.87 (20 litres at 20C) to 0.945.
So, you are inputting 4C into the priming calculator and the priming calculator thinks there are 1.48 volumes when you actually have 0.945. This is why it needs to be set at 20C (highest temperature reached after the generation of CO2). Otherwise the calculator starts assuming you have much less CO2 in absorbtion than you have.
In addition it won't just be CO2 as CO2 doesn't completely stratify, although there would be more CO2 in the FV than in the air, it is still always going to be mixed with its other components.
Suck back also occurs as water is at it's most dense at 4C, therefore there is some shrinkage of the actual liquid which causes suckback/the sides caving in a little as well.
Bearing all this in mind, I don't think the amount of CO2 that will be re-absorbed into the liquid is really going to contribute very much and therefore, as I mentioned, wouldn't be significant. Whereas the temperature is.
I might be wrong, and if so please let me know as these discussions help everyone learn.