Laboratory for
Climatology and Remote Sensing

Abstract:

There are many uncertainties in radiation fog forecast. Continuous effort is being made to improve the forecast. A supplementary and increasingly popular approach to numerical weather forecast is the forecast with machine learning (ML) algorithms. While numerical weather forecast is based on mathematical models with partial differential equations, ML algorithms take a more heuristic approach. The latter strategy calls for three steps. Precise data preprocessing is the initial step. This implies that after preprocessing, the dataset must contain the forecast-relevant information in a way that the algorithm can learn from it. This is not a trivial step because it necessitates a thorough understanding of the fundamental principles underlying radiation fog. Even when the relevant information is contained in the data, it is not always evident, especially in severely unbalanced fog datasets. The best strategy to achieve a pleasing result may therefore not be to simply feed the algorithm all the data and variables that are available. So that the appropriate dynamics may be detected by the algorithm, the data and information should be adjusted accordingly. The second step is the data splitting into training, validation and test datasets. The ability to predict fog is driven by the temporally linked process that describes the ongoing change in atmospheric state but in order to guarantee constant independence between the training, validation and test dataset, the data splitting method must consider this temporally linked information between the individual datapoints. Otherwise, the algorithm’s forecast accuracy can be based on the temporally correlated information content of the individual data points. The third step is the interpretation of the model scores. When looking at the forecast score alone, it is a very abstract number that does not directly allow a statement about the forecast performance of the model. In order to evaluate the model performance, two baselines are of relevance: algorithm complexity and dataset complexity. A baseline for algorithm complexity justifies the chosen algorithm and also classifies the model performance. A baseline for dataset complexity also classifies the model performance and enables a better comparability of different datasets. Following these principles, our current objective is to improve the ML based fog forecast with XGBoost for a forecasting period up to four hours for the station in Linden-Leihgestern (Germany). The training and evaluation are based on the Expanding Window Approach (Vorndran et al. 2022) that considers the autocorrelation of a fog time series and maintains the temporal order during both training and evaluation. The evaluation is based on a score for each of the following categories: Overall performance, fog formation, and fog dissipation. The results are set in relation to different baselines to evaluate the performance and the dataset complexity. Building on this scheme, newly preprocessed data led to an improvement in the prediction of radiation fog for the station in Linden-Leihgestern. We will present the most recent findings from our research.

Abstract:

The current goal of our research is to improve the machine learning (ML) based fog forecast for a forecasting period up to four hours for the station in Linden-Leihgestern. The prediction of radiation fog is still subject to large uncertainties. In particular, the precise prediction of fog start and dissipation, i.e. the transitions, is very difficult. The high-frequency fluctuations of the variables in the formation and dissipation phases pose a particular challenge to ML models. These strong fluctuations make it difficult to extract the necessary information about the past, namely increasing or decreasing trend. However, the temporal evolution in the past is determining for the development of radiation fog. Thus, these dynamics must be prepared in such a way that they can be learned during model training. Therefore, different smoothing levels were tested using a Gaussian moving average filter. Furthermore, additional trend variables for model training were generated that carry information about the temporal evolution of previous data points. Training and evaluation have been carried out with the Expanding Window Approach (Vorndran et al. 2022) that has recently been accepted as a training and validation method for radiation fog prediction. Building on this scheme with the tree-based algorithm XGBoost, the newly preprocessed data led to an improvement in the prediction of radiation fog for the station in Linden-Leihgestern. The results from this research will be presented in the poster session. The study is funded by the DFG research project “FOG-ML FOrecasting radiation foG by combining station and satellite data using Machine Learning”.

Abstract:

Large inaccuracies still exist in accurately predicting fog formation, dissipation, and duration. To improve these deficiencies, machine learning (ML) algorithms are increasingly used in nowcasting in addition to numerical fog forecasts because of their computational speed and their ability to learn the nonlinear interactions between the variables. Although a powerful tool, ML models require precise training and thoroughly evaluation to prevent misinterpretation of the scores. In addition, a fog dataset’s temporal order and the autocorrelation of the variables must be considered. Therefore, classification-based ML related pitfalls in fog forecasting will be demonstrated in this study by using an XGBoost fog forecasting model. By also using two baseline models that simulate guessing and persistence behavior, we have established two independent evaluation thresholds allowing for a more assessable grading of the ML model’s performance. It will be shown that, despite high validation scores, the model could still fail in operational application. If persistence behavior is simulated, commonly used scores are insufficient to measure the performance. That will be demonstrated through a separate analysis of fog formation and dissipation, because these are crucial for a good fog forecast. We also show that commonly used blockwise and leave-many-out cross-validation methods might inflate the validation scores and are therefore less suitable than a temporally ordered expanding window split. The presented approach provides an evaluation score that closely mimics not only the performance on the training and test dataset but also the operational model’s fog forecasting abilities.

Abstract:

A method is presented for fog and low stratus detection from daytime satellite imagery based on Meteosat 8 SEVIRI (Spinning-Enhanced Visible and Infra-Red Imager) data. With its excellent spatial, spectral and temporal resolutions, this imagery is an ideal basis for operational fog monitoring. The scheme utilizes a range of pixel-based and novel object-oriented techniques to separate fog and low stratus clouds from other cloud types. Fog and low stratus are identified by a number of tests which explicitly and implicitly address fog/low stratus spectral, spatial and microphysical properties. The scheme's performance is evaluated using ground-based measurements of cloud height over Europe. The algorithm is found to detect low clouds very accurately, with probabilities of detection (POD) ranging from 0.632 to 0.834 (for different inter-comparison approaches), and false alarm ratios (FAR) between 0.059 and 0.021. The retrieval of sub-pixel and temporal effects remain issues for further investigation.