In this paper, we describe the design and analysis of a scalable architecture suitable for large-scale DSMs (Distributed Shared Memory) systems. The approach is based on an interconnect technology which combines optical components and a novel architecture design. In DSM systems, as the network size increases, network contention results in increasing the critical remote memory access latency, which significantly penalizes the performance of DSM systems. In our proposed architecture called RAPID (Reconfigurable and scalable All-Photonic Interconnect for Distributed-shared memory), we provide high connectivity by maximizing the channel availability for remote communication to reduce the remote memory access latency. RAPID also provides fast and efficient unicast, multicast and broadcast capabilities using a combination of aggressively designed wavelength, time and space-division multiplexing techniques. We evaluated RAPID based on network characteristics, power budget criteria and simulation using synthetic traffic workloads and compared it against other scalable electrical networks. We found that RAPID, not only outperforms other networks, but also, satisfies most of the requirements of shared memory multiprocessor design such as low latency, high bandwidth, high connectivity, and easy scalability.