Ultrasound B-mode images are reconstructed directly from transducer channel signals using a convolutional neural network (CNN). The CNN is trained with a dataset including, as inputs, simulated transducer array channel signals containing simulated speckle and, as outputs, corresponding simulated speckle-free B-mode ground truth images. After training, measured real-time RF signals taken directly from an ultrasound transducer array elements prior to summation are input to the CNN and processed by the CNN to generate as output an estimated real-time B-mode image with reduced speckle.