This paper presents a method to improve the robustness of the position control of a small permanent magnet within a living organism, such as the human body in micro-surgery. So far, position control has been achieved up to 5 Degrees of Freedom with robustness against model uncertainties. In order to achieve robust control against non predicted disturbances, this paper uses a disturbance observer (DOB) which adapts to the non-linearity of the system. Disturbance observers require fast and accurate position sensing in order to estimate and compensate the disturbance accurately. The proposed method depends even more on the quality of the position sensing. To ensure good performances, robust stability conditions are derived regarding position feedback, and the proposed DOB is validated by simulations and experiments.