Recent advances in unmanned aerial vehicles and camera technology have proven useful for the detection of smoke that emerges above the trees during a forest fire. Automatic detection of smoke in videos is of great interest amongst the Fire Department. To date, in most parts of the world, the fire is not detected in its early stage and turns catastrophic. This thesis introduces a novel technique that integrates temporal and spatial features in a deep learning framework using semi-supervised spatio-temporal video object segmentation and dense optical flow. However, detecting this smoke in the presence of haze and without the labeled data is difficult. Considering the visibility of haze in the sky, a dark channel pre-processing method is used that reduces the amount of haze in frames and consequently improves the results. Online training is performed on a video at the time of testing that reduces the need for ground-truth data. Tests using public datasets and videos show that the proposed algorithms outperform previous work and are robust across different wildfire-threatened locations.