Power tower inspection images captured by unmanned aerial vehicles(UAVs) have complex background, and the normal and abnormal samples are extremely unbalanced, which significantly affect the accuracy of anomaly detection for power towers. In order to improve the anomaly detection accuracy in the complex environment, this paper proposes an anomaly detection method for UAV power tower inspection images based on Squeeze-and-Excitation improved fast unsupervised anomaly detection with generative adversarial network (SE-f-AnoGAN). Firstly, the Squeeze-and-Excitation Network (SENet) is added to the encoder in f-AnoGAN to extract the saliency information of images. Next, a combination of unsupervised learning by generative adversarial network and supervised learning by binary classifier is proposed, which consolidates the advantages of feature extractor and discriminator respectively. On this basis, the optimization training strategy based on transfer learning is used to further effectively improve the generalization of the model on the large datasets. The experimental results show that the accuracy rate of overall samples is 95.74% and the recall rates of positive and negative samples reach 96.05% and 95.36% respectively, which demonstrates the effectiveness of SE-f-AnoGAN in anomaly detection.