A Li or Li-ion or Na or Na-ion battery cell is provided that comprises anode and cathode electrodes, a separator, and a solid electrolyte. The separator electrically separates the anode and the cathode. The solid electrolyte ionically couples the anode and the cathode. The solid electrolyte also comprises a melt-infiltration solid electrolyte composition that is disposed at least partially in at least one of the electrodes or in the separator.