There is emerging evidence for the positive effects or benefits of coffee in patients with liver disease. We conducted a retrospective cross-sectional study on patients with non-alcoholic fatty liver disease (NAFLD), hepatitis C virus (HCV), and hepatitis B virus (HBV) infection to determine the effects of coffee intake on a non-invasive marker of liver fibrosis: liver stiffness assessed by transient elastography (TE). We assessed coffee and tea intake and measured TE in 1018 patients with NAFLD, HCV, and HBV (155 with NAFLD, 378 with HCV and 485 with HBV). Univariate and multivariate regression models were performed taking into account potential confounders. Liver stiffness was higher in males compared to females (p < 0.05). Patients with HBV had lower liver stiffness than those with HCV and NAFLD. After adjustment for age, gender, smoking, alcohol consumption, M or XL probe, and disease state (NAFLD, HCV, and HBV status), those who drank 2 or more cups of coffee per day had a lower liver stiffness (p = 0.044). Tea consumption had no effect (p = 0.9). Coffee consumption decreases liver stiffness, which may indicate less fibrosis and inflammation, independent of disease state. This study adds further evidence to the notion of coffee maybe beneficial in patients with liver disease.