Maternal caffeine consumption is associated with adverse gestational outcomes. The aim of this study was to assess the intake of caffeine and factors associated with the non-adherence to caffeine intake recommendations in a cohort of 463 women before (T0) and in each trimester of gestation (T1, T2, and T3), by using validated questionnaires. Caffeine intake (median (mg/day), IQR) was 100.0 (181.1) at T0, 9.42 (66.2) at T1, 12.5 (65.6) at T2, and 14.0 (61.1) at T3 (p < 0.001). Non-compliance prevalence (intake > 200 mg/day) was 6.2% at T1, 4.2% at T2, and 2.7% at T3. Not being an active smoker at T1 (OR = 0.17; 95% CI 0.05–0.59) and T2 (OR = 0.22; 95% CI 0.09–0.52), adherence to the Mediterranean Diet at T1 (OR = 0.50; 95% CI 0.28–0.88) and T2 (OR = 0.39; 95% CI 0.15–1.02), and moderate physical activity at T1 (OR = 0.50; 95% CI 0.28–0.88) were inversely associated with caffeine consumption. Although caffeine intake may be considered low, intake prevalence increases throughout pregnancy. Although the main source of caffeine during pregnancy is coffee, attention must be also paid to the increasingly intake of chocolate, of which the effect during pregnancy is controversial. Smoking, non-adherence to a good quality diet, and light physical activity are associated with a higher caffeine intake and a lower compliance with caffeine intake recommendations. Perinatal dietary and lifestyle educational policies are needed.