Previous investigations have found that smokers with schizophrenia demonstrate reduced performance on cognitive tasks compared to non-smokers. However previous studies have not taken into account other environmental factors associated with cognitive functioning such as exposure to Herpes Simplex Virus type 1 (HSV-1). We examined these factors in a sample consisting of individuals with schizophrenia (n = 773), bipolar disorder (n = 493), or controls without a psychiatric disorders (n = 548). Participants were assessed on a cognitive battery, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), and had a blood sample drawn to measure seropositivity to HSV-1. Within each group linear regression models were constructed to determine whether cigarette smoking and HSV-1 seropositivity were jointly associated with cognitive functioning after adjusting for relevant covariates. Within the schizophrenia group, the effect size of lower total cognitive score was −0.279 (p < 0.0001) for individuals who were both smokers and HSV-1 seropositive and a significant effect was found in all cognitive domains. The odds of being in the highest quartile of RBANS Total score were significantly lower for smokers (OR = 0.58, 95% CI 0.41, 0.82, p = 0.002). Smoking was not as consistently associated with levels of cognitive functioning in the bipolar disorder or the non-psychiatric control group. While experimental studies show that nicotine transiently improves functioning on sensory gating and attention tasks known to be deficient in schizophrenia, long-term nicotine exposure via smoking appears to have an adverse effect on cognitive functioning.