I'm assuming you want something you can use over and over. I like the idea.
I have never done it for frag plugs but have drilled lots of dry/live rock for supports etc.
Just use a masonry bit the size of the frag plug stem or a little larger (probably 1/4" or 3/8").
Go slow and use water. Don't put a lot of pressure on the bit. Let it do the work. To much pressure will sometimes fracture the rock..