In this paper, we present a Hybridizable Discontinuous Galerkin (HDG) method for the solution of the compressible Euler and Navier-Stokes equations. The method is devised by using the discontinuous Galerkin approximation with a special choice of the numerical fluxes and weakly imposing the continuity of the normal component of the numerical fluxes across the element interfaces. This allows the approximate conserved variables defining the discontinuous Galerkin solution to be locally condensed, thereby resulting in a reduced system which involves only the degrees of freedom of the approximate traces of the solution. The HDG method inherits the geometric flexibility and arbitrary high order accuracy of Discontinuous Galerkin methods, and offers a significant reduction in the computational cost as well as improved accuracy and convergence properties. In particular, we show that HDG produces optimal converges rates for both the conserved quantities as well as the viscous stresses and the heat fluxes. We present some numerical results to demonstrate the accuracy and convergence properties of the method.