P300 is an event-related potential evoked as a response to external stimuli. The P300-speller is a widely used BCI that has proven to be a reliable method in enabling people who cannot communicate via normal methods. Improving single-trial P300 classification helps

increase communication bandwidth as it reduces the averaging process necessary to reduce noise. Deep learning approaches in this area have become more popular recently [2][3]. Certain architectural choices result in a better classifier. In this paper, we performed a study of EEGNet [2] as it performed well in our experiments. We report some improvements to the model. We also confirm that these suggestions improve other CNN based models by showing improved results with CNN-1 proposed in [4] on BCI Competition III Dataset II. We perform knowledge distillation of the resultant linear model into a logistic regression model and show that the model learns information similar but superior to LDA. We show that the logistic regression model obtained this way outperforms more complex models and is easy to interpret.