Fifty Plus (50+) - .001mm

My son cut some machine tool pallet shims that were accurate to .001mm. I told him that when I was his age we had to get stuff like that exactly right. How do you measure something to .001mm accuracy?

If this is "Production 101," I apologize, but this is what I think.

Technically nothing is ever formed exactly right except by chance. The job for a factory, machine shop, or craftsman is to get it so close to right (tight enough tolerance) that the remaining error does not compromise performance, fit, function, feel, durability, or appearance. If you were to look at a "smooth" surface under an electron microscope, you'd see something that looks like the Rocky Mountains.

Mirrors do not have specs that say "perfect." They say things like "allowable tolerance 0.000002 inches" or some such number.

I'd start by looking in McMaster-Carr for precision measuring tools, then perhaps (walking up the price ladder) to Edmund Optics and Scientific for how optical surfaces are evaluated. At some point looking at tools priced from $300 to $30,000 a pop you'll think "how good does it really need to be, and why is what we normally do not good enough this time?" and then you've arrived at the right question.

I assume in his shop there are some digital mikes or surface gauges stored in fitted wooden boxes in a locked cabinet along with some surface plates, that are used for tasks like this. I can't imagine a good shop foreman accepting a job that he couldn't accomplish with reasonable margin. Over-working the smoothness can eat up your profits.

My son cut some machine tool pallet shims that were accurate to .001mm. I told him that when I was his age we had to get stuff like that exactly right. How do you measure something to .001mm accuracy?

With a micrometer or gauge calibrated to an accuracy level/tolerance of .001mm?

I do gunsmithing on occasion and have some high accuracy measuring tools. Not to that level but I know that they exist. I know that turbine blades in the "hot" section of a jet motor have to be machined to incredible accuracy.

I've got a Mitutoyo digital caliper that goes 2 decimal places and a older manual Starrett micrometer that measures directly to .004 and interpolated to closer. They are fairly expensive devices but not in the ballpark as what a top end machinist would have.

I just took a tour of the UA Mirror Lab in Tucson and they achieve nanometer accuracy with lasers. An analogy the docent gave was that a 2" bump in an area the size of the CONUS would be just tolerable.

I used to grind automotive cranks for a living. Never had to be that accurate. .0001" (.00254 mm) was as 'close' as I ever had to be.

You can use the following:

http://www.mitutoyo.com/TerminalMerchandisingGroup.aspx?group=1816

I imagine it's pretty salty.

Just stack 1000 of them together and see if they measure 1MM

1 micron/micrometer precision is an extremely exacting standard and all but impossible without high end professional tooling.

And there's no such thing as "exactly right". It's just a question of the measurable precision.

I used to Build racing 2 stroke engines and part of that involved trueing the crank.Most other other builders used to retrue so that it was within 1/1,000" and that was good enough for them. I wanted truer than that and I got it so the dial gauge just showed a shudder on the crank. Then one day a customer came in for a big end change and I put the crank back in the Lathe to check and it was perfect straight away. The customer was surprised at how I had done it so easily and I was shocked--Normally took 3 or 4 attempts to get it that perfect.

But going on perfection--I am very particulalr about how I set up my bike and have even taken to marking the seat post and bar angle so I get it right if it changes. 1/4" out on saddle height and I know it. Wheels slightly out of true and I can feel it. But I work to within 1/4"--Others I know measure in 1/2 mm's. Thats the difference of putting a pair of tights on for a ride.

.001 mm is 1 micron, the wavelength of NIR light, measurable with optical matchplates (for flatness) a laser surface profilometer (for surface finish) or a set of gage heads heads, an LVDT and an amplifier for absolute thickness.
An interferometer would also work as well.
Not reliably and repeatably measurable, mechanically to my knowledge.

I worked in a machine shop many years ago.This shop produced several Space Shuttle components that were mostly full-floating 3-way air or hydrophilic piston type valves iirc.

These valves valves required extreme tolerance control which, in that shop and at that time, could only be obtained by the use of an "optical comparator" device and a NASA approved and supplied "standard" with which to compare them against.

My only involvement with the production process was "culling the good from the bad" by the careful use of this comparator device. I was, at ~ age 22, one of only ~ 2 other employees who had eyesight good enough to use the device with an acceptable level of repetitive accuracy. You couldn't use this device for more than about an hour before you eyes "gave out" on you.

These valves were tested by at least two of us before they were sent out (and more than a few out of every batch were returned for being "out of specs"). I vaguely remember that only ~ 1 out of 10 or so of these valves passed the shop's "final" two-stage inspection.

If the part being measured is steel and more than a couple of inches thick, the thermal expansion of steel (0.0000116mm/mm/deg C) will exceed a micron with low temp change.

If the part being measured is steel and more than a couple of inches thick, the thermal expansion of steel (0.0000116mm/mm/deg C) will exceed a micron with low temp change.

Very true, but the piece is said to be a shim, which could be a fraction of a mm thick. Even so, a tolerance of 1 micron is tough.

The Hubble space telescope missed on the Mirror polishing..

rumor has it they got the mirror right for the Spy satellite that is over your head right now ,
taking your licence plate Number. Priorities..

The Hubble space telescope missed on the Mirror polishing..

rumor has it they got the mirror right for the Spy satellite that is over your head right now ,
taking your licence plate Number. Priorities..

There's a lot more than one of them. Not so long ago the NSA rang up NASA and asked if they wanted two surplus Hubble-equivalent satellites they had sitting in storage. All NASA had to do was to pay for the launch and adapt them so they look outwards instead of down.

Thats kind of a knats hind leg sand papered on both sides!!!!:)

Just stack 1000 of them together and see if they measure 1MM

But that might mean 900 of them are under size by 0.0002 and 100 of them are over size by 0.0018 :rolleyes:

One of the machine I worked on had an adjustment that was listed as a NEGATIVE 5/10,000, basically a controlled bind.

I can't imagine a good shop foreman accepting a job that he couldn't accomplish with reasonable margin. Over-working the smoothness can eat up your profits.

This isn't a production job. The shims are for the pallets that hold the production parts while they are being machined. Paul maintains the machine tools, does some set up, and approves cutting tool purchases for a fairly large machine shop. They do a lot of aero space work for Boeing. A recent contract is for boom skins for the Air Force's new tanker. It's a good job to have because the booms have to be replaced periodically so it's a long term contract.

I don't know how thick the shims actually are, but they came from Japan oversized. The Makino tech rep said a tolerence of .002 to .005 would be adequate but Paul said he was having fun and decided to see how precise he could get them. He said the grinder doesn't make any sparks or noise as you sneak up on it because you're not taking off very much metal.

I do gunsmithing on occasion and have some high accuracy measuring tools. Not to that level but I know that they exist. I know that turbine blades in the "hot" section of a jet motor have to be machined to incredible accuracy.

I've got a Mitutoyo digital caliper that goes 2 decimal places and a older manual Starrett micrometer that measures directly to .004 and interpolated to closer. They are fairly expensive devices but not in the ballpark as what a top end machinist would have.

I have an antique 1" Lufkin mike with a secondary vernier that indicates in 0.0001 (ten-thousandths) inch, but it hasn't been maintained, i.e. zeroed, lubricated, play-adjusted, and checked for tracking accuracy. A thousandth of a millimeter corresponds to about 0.4/10,000 inch, about half of a ten-thousandth of an inch. So when calibrated my mike would probably not indicate a difference of .001 mm between two measurements, but it's close - its resolution ONLY needs to be doubled to be able to measure that. One way to do that is to simply remake the mike with larger barrels so that a tighter vernier scale could be read, or add a magnifying attachment of some sort. This assumes the screw is calibrated will enough to perform to this level.

This isn't a production job. The shims are for the pallets that hold the production parts while they are being machined. Paul maintains the machine tools, does some set up, and approves cutting tool purchases for a fairly large machine shop. They do a lot of aero space work for Boeing. A recent contract is for boom skins for the Air Force's new tanker. It's a good job to have because the booms have to be replaced periodically so it's a long term contract.

I don't know how thick the shims actually are, but they came from Japan oversized. The Makino tech rep said a tolerence of .002 to .005 would be adequate but Paul said he was having fun and decided to see how precise he could get them. He said the grinder doesn't make any sparks or noise as you sneak up on it because you're not taking off very much metal.

Interesting projects! I worked on some NASA and Air Force satellites whose purposes were classified. I was in the electrical engineering team responsible for the electrical power system design and validation and I couldn't know! Some were supposed to be optical sensors, at least our optics PhD was on the project.

Any more and they'd have to kill me, and of course, all of you.

Most people wouldn't have the skill set to measure that accurately.
You pretty much have to be in a temperature controlled environment, using an insulated glove.
What happens to a micrometer when you pick it up and raise it to body temperature from room temperature??? It EXPANDS!

But that might mean 900 of them are under size by 0.0002 and 100 of them are over size by 0.0018 :rolleyes:

And if you can't tell by feel that that 100 are 9X thicker than the others (which are 1/2 as thick as they should be) you shouldn't be leaving your house.

My son cut some machine tool pallet shims that were accurate to .001mm. I told him that when I was his age we had to get stuff like that exactly right. How do you measure something to .001mm accuracy?

How does he measure them?????

294650

Most people wouldn't have the skill set to measure that accurately.
You pretty much have to be in a temperature controlled environment, using an insulated glove.
What happens to a micrometer when you pick it up and raise it to body temperature from room temperature??? It EXPANDS!

I agree with this. In the techy companies I've worked for (Lockheed Martin, Ford, Visteon, Autoliv, Toyota), there are precision measurement labs with the abilities to perform difficult measurements of this sort, but there were very skilled people around. But complex test procedures and setups were de rigeure.

I agree with this. In the techy companies I've worked for (Lockheed Martin, Ford, Visteon, Autoliv, Toyota), there are precision measurement labs with the abilities to perform difficult measurements of this sort, but there were very skilled people around. But complex test procedures and setups were de rigeure.

Given enough time and money, anything can be done, the issue is that you need to determine with a project like this, whether it's worth the time or money. If your putting up a steel frame for a building, you could be an inch out, and nobody would really be concerned, lots of precision parts have these kinds of tolerances though. I would be shocked to find a brifter doesn't have several of them.....

I was mainly agreeing with Bill Kapaun and his comment about necessary skills. Those installations, procedures, and training cost a lot of money, and to execute those measurements took a lot of time, so yes, time and money are involved.

The superintendent I am teamed with for NAVFAC contracts was a machinist for Westinghouse NCD plant here in Pensacola. The nuclear power industry and some of the new combustion power generator systems routinely make measurements that precise and sometimes more precise for reactor components and such. Lasers come onto play as I understand from talking with him about various project aspects he worked on over his 32 year career with them.

My Sis-I-L worked with him as a nuclear engineer/team engineer for production and finally in R&D, QC and development laboratory work until the plant closed, she has told me about the design and building that they did for measurement devices. Tube guide plates that require extreme precision on their production and installation orientation to allow tubes of Inconel alloy to be inserted without any wrinkling what so ever as they are inserted have to be measured down below the precision RG said his son used. Tubes were inspected after insertion for wrinkling and had to be correct or the assembly would be rejected. that is a large sum of money in time and materials down the drain. Careful work, organization and good QC make it possible to be done in a efficient and effective manner. As a Quality Engineer I can back up what they have been able to tell me within security requirements they worked under.

Bill