System identification is a key enabling component for the implementation of new quantum technologies, including quantum control. In this paper we consider a large class of input-output systems, namely linear passive quantum systems, and study the following identifiability question: if the system's Hamiltonian and coupling matrices are unknown, which of these dynamical parameters can be estimated by preparing appropriate input states and performing measurements on the output? The input-output mapping is explicitly given by the transfer function, which contains the maximum information about the system.We show that two minimal systems are indistinguishable (have the same transfer function) if and only if their Hamiltonians and the coupling to the input fields are related by a unitary transformation. Furthermore, we provide a canonical parametrization of the equivalence classes of indistinguishable systems. For models depending on (possibly lower dimensional) unknown parameters, we give a practical identifiability condition which is illustrated on several examples. In particular, we show that systems satisfying a certain Hamiltonian connectivity condition called "infecting", are completely identifiable.