Ultrafine-grained metals whose grain size is less than one micron have attracted interest as high strength materials. Whereas nanostructured metals produced by severe plastic deformation express remarkably peculiar behavior in both material and mechanical aspects, its mechanism has been clarified by neither experimental nor computational approaches. In this study, we develop a multiscale crystal plasticity model considering an effect of grain boundary. In order to express release of dislocation from grain boundaries, information of misorientation is introduced into a hardening law of crystal plasticity. In addition, carrying out FE simulation for FCC polycrystal, the stress-strain responses such as increase of yield stress due to existence of grain boundary are discussed. We investigate comprehensively the effect of dislocation behavior on the material property of nanostructured metal.