A charged particle beam device includes a charged particle source which emits a charged particle beam radiated on a sample; a condenser lens system which has at least one condenser lens focusing the charged particle beam at a predetermined demagnification; a deflector which is positioned between a condenser lens of a most downstream side and a charged particle source in the condenser lens system, and moves a virtual position of the charged particle source; and a control unit which controls the deflector and the condenser lens system. The control unit controls the deflector to move the virtual position of the charged particle source to a position of suppressing a deviation, which is caused by a change of the demagnification of the condenser lens system, of a center trajectory of the charged particle beam downstream of the condenser lens system.