To investigate the effect mechanism of elastic joint on the natural frequencies and modal shapes of flexible manipulators, an elastic restraint model of the flexible manipulator was established. Based on the elastic restraint model, frequency equation and modal shape function of the flexible manipulator with elastic joint (FMEJ) were subsequently derived. Numerical method was conducted to investigate the first three natural frequencies and modal shapes of the FMEJ. The results demonstrate that the elasticity of the joint has considerable effect on the natural frequencies and modal shapes, and assuming the joint as an ideal fixed restraint a noticeable error will present. According to the sensitivity analysis, the linear restraints have a more considerable effect on the natural frequencies than torsional restraints, especially for the higher frequencies. The effect of torsional restraints on the modal shapes is more considerable, with the torsional restraint increases, the modal shapes change which is obvious for the higher modal shapes. Experiment was conducted to verify the elastic restraint model and the results indicate that the constructed model can characterize the elastic restraints of the joint, and approach to the actual situation which has a smaller error.