Background: Previous studies have suggested that the pandemic impact on mental health could vary according to gender. We aimed to evaluate the gender influence in the prevalence of depressive and anxiety symptoms in Latin American and the Caribbean (LAC) countries in the first stage of the COVID-19 pandemic. Methods: We conducted a secondary analysis employing the Facebook–COVID-19 Symptom Survey developed by the University of Maryland. We categorized gender as men, women, and non-binary. The outcomes were the presence of anxiety or depressive symptoms, measured with two adapted questions extracted from the Kessler Psychological Distress Scale (K10). We used generalized linear models from the Poisson family, considering the survey's complex sampling. We calculated crude and adjusted prevalence ratios (PR) with their 95% confidence intervals (95% CI) and explored interactions with gender using the adjusted Wald test. Results: We included 1,338,320 adults from LAC countries; 48.0, 50.6, and 1.4% were men, women, and non-binary participants, respectively. The overall prevalence of anxiety or depressive symptoms was 44.8 and 46.6%, respectively. We found interactions between gender and the rest of the independent variables. In the non-binary group, the association between age and anxiety symptomatology was lost after an age of 55 years. Furthermore, whereas living in a town was associated with a lower prevalence of anxiety and depression symptomatology in men and women, this did not happen among non-binary individuals. Compliance with physical distancing was associated with a lower prevalence of anxiety and depression symptomatology among women (anxiety: PRa = 0.98; 95% CI = 0.97–0.99; p < 0.001, depression: PRa = 0.96; 95% CI = 0.95–0.97; p < 0.001) and only anxiety in non-binary participants (anxiety: PRa = 0.92; 95% CI = 0.88–0.98; p = 0.005). This was not evidenced among men participants (anxiety: PRa = 0.99; 95% CI = 0.96–1.01; p = 0.199, depression: PRa = 0.98; 95% CI = 0.96–1.00; p = 0.084). In addition, compliance with handwashing was associated with a higher prevalence of anxiety symptomatology among men (PRa = 1.06; 95% CI = 1.05–1.11; p < 0.001) and women participants (PRa = 1.03; 95% CI = 1.01–1.05; p = 0.016). Conclusion: Approximately 4 out of 10 participants had anxiety or depressive symptoms. Women and non-binary gender people had more symptoms of anxiety or depression. The factors associated with these symptoms varied according to gender. It is essential to evaluate gender-related strategies to improve mental health during the COVID-19 pandemic.