We characterize optimal taxation of foreign capital and optimal sovereign debt policy in a small open economy where the government cannot commit to policy, seeks to insure a risk-averse domestic constituency, and is more impatient than the market. Optimal policy generates long-run cycles in both sovereign debt and foreign direct investment in an environment in which the first best capital stock is a constant. The expected tax on capital endogenously varies with the state of the economy, and investment is distorted by more in recessions than in booms, amplifying the effect of shocks. The government's lack of commitment induces a negative correlation between investment and the stock of government debt, a "debt overhang" effect. Debt relief is never Pareto improving and cannot affect the long-run level of investment. Furthermore, restricting the government to a balanced budget can eliminate the cyclical distortion of investment.