The performance of popular Hartree-Fock-based effective core potentials in Hartree-Fock and density functional calculations of 3d transition metals has been evaluated by basis-set convergence studies for ten cases: the equilibrium bond dissociation energy (D e) for dissociation of ground-state Ti 2 to ground and excited atoms, the ground-state dissociation energies of FeO, Cu 2, ScH, TiH, Sc 2, Fe 2, and TiV +, and the first excitation energy (E x) of Ti atom. Each case is studied with 11 or 13 density functionals. For comparison, the accuracy of the all-electron def2-TZVP basis set is tested with both relativistic and nonrelativistic treatments. Convergence and accuracy are assessed by comparing to relativistic all-electron calculations with a nearly complete relativistic basis set (NCBS-DK, which denotes the cc-pV5Z-DK basis set for 3d metals and hydrogen and the ma-cc-pV5Z-DK basis set for oxygen) and to nonrelativistic all-electron calculations with a nearly complete nonrelativistic basis set (NCBS-NR, which denotes the cc-pV5Z basis set for 3d metals and hydrogen and the ma-cc-pV5Z basis set for oxygen). As compared to NCBS-DK results, all ECP calculations perform worse than def2-TZVP all-electron relativistic calculations when averaged over all 130 data (13 functionals and ten test cases). The compact effective potential (CEP) relativistic effective core potential (RECP) combined with a valence basis set developed for the many-electron Dirac-Fock (MDF10) RECP performs best in effective core potential calculations and has an average basis-set incompleteness error of 3.7 kcal/mol, which is much larger than that (0.9 kcal/mol) of def2-TZVP relativistic all-electron results. Hence, the def2-TZVP relativistic all-electron calculations are recommended for accurate DFT calculations on 3d transition metals. In addition to our general findings, we observed that all kinds of density functionals do not show the same trends. For example, when ECPs are used with hybrid functionals, which sometimes are not recommended for calculations of transition metal systems, they are found to perform better at achieving the basis-set limit than when used with local functionals and meta-GGA functionals. The most successful combination of RECP and basis set has a basis-set incompleteness error of 1.7-2.4 kcal/mol for hybrid generalized gradient approximations, which is smaller than that of nonrelativistic NCBS calculations (whose average basis-set incompleteness error for hybrid functionals is 2.7-2.9 kcal/mol). The average basis-set incompleteness error in Hartree-Fock calculations is 1.0-4.4 kcal/mol for five of the ECP basis sets but is 5.8-10.8 kcal/mol for six others.