The modal peak in frequency domain was interfered strongly under the strong noise, causing the inaccurate modal parameters, as well as producing modal frequency miscalculation. According to this situation, this paper adopted spectrum clustering algorithms to cluster the vibration spectrum, and come up with a new method for spectrum segmentation. First, according to the concept of wave spectrum, the vibration signal spectrum was divided into pack of waves. For each wave as a sample of clustering, this paper used spectrum clustering to cluster these waves. The wave similarity function, Laplace matrix and clustering algorithm were constructed, and this algorithm was used to cluster automatically. Finally, the clustering result is that a big peak in macroscopic. By the experimental results of simulation signals, this spectrum clustering algorithms can accord with the macroscopic observation of modal separation efficiency,and it is stronger to the noise resistance than that of the existing k-means algorithms. According to the test of a cable, the stability and accuracy of this algorithm is demonstrated again.