You can drill the smaller through-hole first, then use a counterbore tool. The end of the counterbore will use the through-hole as a guide and will cut a flat-bottomed hole of a larger size exactly centered on the smaller hole.

CobaltDrill bits

If the workpiece is too thin, try clamping a scrap piece below the planned hole to give the pilot tip somewhere to indicate onto.

With a smaller drill press, you cannot just keep the workpiece clamped in place because to remove the smaller bit and replace it with a larger bit (or vice-verse) will end up breaking the alignment and it's nearly impossible to get it back again (within a few thousandths of an inch) by eye.

CarbideDrill bits

Professionaldrill bits to drill hardened steel

That way there will still be some wood there to hold the smaller drill bit, and the pointy pilot tip in the middle will act as a center.

Additionally, as Graphus pointed out in his comments on the OP accuracy within a few thousandths is highly unlikely with home- or even commercial-grade woodworking drill presses. You'd likely need an expensive CNC drill press to achieve that accuracy and a much harder, or at least much more consistent material (plastic or metal) to achieve that kind of precision.

If you use brad point drill bits drill the larger one first then carefully center the smaller one over the point (left in the hole) of the larger one then drill through. OR Drill a very small hole as a center point through the board use this as a center for the 1/4" and the 3/8" hole.

Forstner bits have a small point on the end which leaves a slight indentation in the wood in the center of the hole. Use this to align your drill bits after changing them. But, to do this, you'll have to drill them the opposite way you're thinking - drill the biggest (but shallowest) hole first, followed by the next smallest / deepest. This keeps that slight indentation from the point on the drill bit present in the wood, allowing you to align the next drill bit.

Carbidedrill bitsforhardened steel

If you use, say a 1/16" bit for your pilot hole, your 1/2", 3/8", and 1/4" bits of any sort should line themselves up quite nicely on that 1/16" hole.

Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Often times I need to drill maybe a 1/4" hole, all the way through the piece, and then a 3/8" or 1/2" hole half way through the piece in the same spot. It is often imperative that these holes are exactly concentric.