Small for dates

I had my 19 week scan and bubs is measuring within norms but only just and so is small for dates, had another scan at 21 weeks and he is growing but still small so they are refering me to an obstetrician.
My LMC doesn't seem too concerned but just wondered if anyone else has gone through or is going through this as I am finding it hard not to worry.
I have been reading about mothers malnutrition being a cause, but I find it hard to believe that that could apply to me.