A second band-like mask material layer having a first band-like mask material layer of a same planar shape on its top is formed on a mask material layer on a semiconductor layer. Then, fourth band-like mask material layers having third band-like mask material layers of same planar shape on their top are formed on both side surfaces of the first and second band-like mask material layers. Sixth band-like mask material layers having fifth band-like mask material layers of same planar shape on their top are formed on the outside thereof. Then, an orthogonal band-like mask material layer is formed on the first band-like mask material layer, in a direction orthogonal to a direction in which the first band-like mask material layer extends. Semiconductor pillars are formed on overlapping areas of this orthogonal band-like mask material layer and the second and sixth band-like mask material layers by etching the semiconductor layer. Then, a pillar-shaped semiconductor device is formed with these semiconductor pillars being channels.