Sunday, November 14, 2010

The Wild, Wild West of 3D Metrology

Optical metrology system suppliers are often present at
manufacturing trade shows. Casual discussions with the personnel at
the booths, though, may reveal an astonishing discovery: the stated
accuracies of the optical metrology systems seem to sit in the wild, wild
west. No clarification on how numbers are achieved, or under which
circumstances, or on which types of surfaces, or even at which sigma values the
numbers may be quoted. Metrology systems providers
that do provide thorough information on accuracy often reference
“ideal” test artifacts such as matte white surfaces in dark environments.
Even at a trade show, where people come to learn about measurement systems, this
is not fair to customers. But back in the real world, in your
environment, where it really matters, be especially sure to cross
your t’s and dot your i’s. Assuming the wrong numbers, even a
little bit, is very, very bad because measurement systems have to be a small
fraction of actual manufacturing blueprint tolerances…so any unclear
factors have severe consequences for applicability to a process.

Why is this noteworthy? Well, for one thing, the cameras and lensing
determine the noise/accuracy floor of the system. Once this floor is
determined, software factors such as image processing algorithms, data
extraction, etc. set the operating parameters. But here’s where things get
interesting (and challenging): environmental factors such as vibration,
lighting, and heat, and application factors such as the parts being measured,
the camera’s field of view relative to part size, and the method used to
collect/finalize the data work to compromise this theoretical floor.

The challenge for optical metrology systems providers is that they have
to work with customers’ requirements/environments, whereas the challenge for
companies that purchase optical measurement systems is that their requirements
can change.

What, then, is the solution?

The solution is for metrology systems suppliers to GUARANTEE that their
measurement systems will work in the customer environments, as installed, for
the intended purpose. Whereas this may leave some ambiguity later if the
customer’s requirements change, at very least the system is guaranteed to work
at least once. If it doen’t work, no payment. This means that the
company using the measurement system should verify the results, and if the
system does not meet specification, request a refund. This
should be part of the discussion from the beginning. No performance, no

Some metrology systems suppliers reference established NIST, ISO, ANSI,
VDI/VDE, etc. standards when quoting accuracy. In many/most cases,
these standards are useless for an application. VDI/VDE 2634, for
example, allows measuring matte white tooling balls that are large relative
to the camera’s field of view. This is irrelevant for many
applications. Who manufactures matte white tooling balls? Probably
not your company. Your company should ask the metrology system provider to
verify accuracy on your component. These standards are fine for
traceability but say nothing about applicability. For a
measurement system to be effective, you need both.

If anything, these “idealized” test procedures will show the
metrology system at its best - but in reality, we want to know how the metrology
system will perform at its worst - when things go wrong in a production
environment. This indicator of robustness is what separates production
metrology equipment from a laboratory experiment. If you are a lab, then
you can carefully set up each experiment/test/analysis. But, if you are
producing parts, and rely on your measurement system to provide an accurate
description of your process, the laboratory is a far world away.

One quick way to confirm a system’s performance is to perform a repeatability
study. Simply measure the exact same part 10 times in the exact same
way. Compare runs 2 through 10 to run 1. The largest deviation
should be less than the stated value at 2 sigma. Then, perform an accuracy
study. Simply measure the exact same part 10 times, but change as many
things as possible between each measurement (the part’s location in the work
area, the vibration in the workspace, lighting brightness, lighting changes,
camera angles, calibration routines, etc.). Again, compare runs 2 through
10 to run 1. And, again, the largest deviation should be less than the
stated value at 2 sigma.

For further information on accuracy studies, read our blog entry on
measuring surface plates, and also return to this blog - we’ll be continuing to
discuss this topic in the future. The important thing to take
away is not to idealize the experiment, but to push the metrology system to
its limits to better understand it. Then, spec the equipment at a
reasonable indication of its peformance. This may mean specifying it
differently than what the manufacturer states.

If applied with perspective, a metrology system won’t be a wildcard in your
production environment, and won’t force your environment into the wild,
wild west.

No comments:

Post a Comment