Climate-related phenomena are increasingly affecting regions worldwide, manifesting as floods, water scarcity, and heat waves, which can significantly impair companies' assets and productivity. We develop a framework, based on the Merton model for credit risk, that introduces downward jumps due to climate phenomenon in a company asset's dynamics. These negative shocks aim to mirror the negative effect of extreme climate events. We explore various possibilities for the distribution of jumps and provide a comprehensive characterization of the model and its calibration process. The calibration relies on hierarchical clustering of firms and countries on asset intensity and geographical exposure. Finally, we utilize this new Merton model framework with jumps to understand the influence of climate related extreme events on different portfolios expected and unexpected losses. The results suggest introducing additional safe capital to offset the extra expected loss resulting from physical climate risks.