Embodiments of the present disclosure are directed toward techniques and apparatus comprising at least one layer of an ONN that includes an optical matrix multiplier provided in a semiconductor substrate to receive a plurality of optical signal inputs and to linearly transform the plurality of optical signal inputs into a plurality of optical signal outputs. The optical matrix multiplier comprises one or more 2×2 unitary optical matrices optically interconnected to implement a singular value decomposition (SVD) of a matrix, and a nonlinear optical device coupled with the optical matrix multiplier in the semiconductor substrate, to receive the optical signal outputs and to provide an optical output that is generated in a nonlinear manner in response to the optical signal outputs of the optical matrix multiplier reaching saturation or attenuation. Additional embodiments may be described and claimed.