Most nubibranchs are specialized eaters. If you have found some near or on this coral, then most likely they are feeding on your colt.
I would recommend a coral dip in iodine or one of the commercially available coral dips. After the dip all the nudibranchs should fall off. You may need to repeat this procedure every week for a month or two to get rid of them all.
P.S. Identification will be hard because of the vast numbers of nudibranch species, but not impossible.