Uniformity, Repeatability, Stability, and Accuracy

Sensor manufacturers often provide information regarding uniformity, repeatability, stability, and accuracy on their specification sheets. But what do they really mean to you, the person making the measurement?

{Image from http://www.freedigitalphotos.net}

Uniformity is a type of systematic error that refers to the consistency of a measured value, either in a spatial distribution or compared to another sensor taking the same measurement. For example, a scientist seeking to know the uniformity of a large set of thermistors (same type and model) could take measurements of a circulating water bath held at a “constant” temperature. The manner in which the thermistors vary in comparison to one another would determine their uniformity.

Repeatability is similar to uniformity except that it deals with how consistent a particular sensor is against itself. It can be used to describe the ability of a sensor to provide the same result, under the same circumstances, over and over again. For instance, infrared radiometers (IRRs) manufactured by Apogee Instruments have a repeatability of 0.05°C (when within calibrated target and sensor body temperature ranges), denoting that measurements made under the same conditions should be within 0.05°C of the mean of those measurements. Repeatability is typically due to random noise, which is often outside of our control.

Stability deals with the degree to which sensor characteristics remain constant over time. Changes in stability, also known as drift, can be due to components aging, decrease in sensitivity of components, and/or a change in the signal to noise ratio, etc. Amplified pyranometers from Apogee Instruments illustrate this concept well with a stability of less than 3% per year. In other words, each year, under normal conditions, one can expect the sensor’s readings to change by less than 3%.

Accuracy, analogous to uncertainty relative to a reference, is in its simplest terms the difference between measured and “true” values. During calibration, measurements are compared to a reference, ISO or NIST traceable where available. Accuracy is determined by performing a general expanded uncertainty analysis, where all of the systematic and random uncertainties are combined using ISO uncertainty standards (which can be a discussion for a different day).

For each of the examples above, we must assume that a large sample size was used, and that the sensors used were chosen at random. Sensor specifications listed for uniformity, repeatability, stability, and accuracy from Apogee Instruments are based on a 95% level of confidence. This means that specifications listed for each sensor are guaranteed for at least 95% of all measurements taken and/or sensors used. Specification sheets for all of our sensors can be found in each of our product sections online. If you have any questions about this blog post, feel free to comment below.

Extra Resources:

Experimentation, Validation, and Uncertainty 
Analysis for Engineers
Third Edition
Coleman, H.W. and Steele, W.G.
John Wiley & Sons, Inc., 2009

Meteorological Measurement Systems 
Brock, F.V. and Richardson, S.J.
Oxford University Press, 2001

 

Adam Del Toro
Mechanical Engineer

New Kid on the Blog-k

Apogee Instruments continually strives to develop new and innovative instrumentation, improve existing products, advance sensor applications, streamline current manufacturing processes, and provide immediate and effective customer support, all in an effort to help our customers make better measurements. The lifeblood of this effort, or of any company, is the people doing the work. To expand our capabilities, we are pleased to announce the addition of a new engineer, Ryan Lindsley. We wanted to give Ryan the opportunity to introduce himself. 

I graduated a few weeks ago from Utah State University with a degree in mechanical engineering and a minor in business. While my scholastic educational experience may be at a close, I’m beginning a whole new learning experience as a new mechanical engineer for Apogee Instruments (maybe you’ve heard of them?). While I previously spent most of my time on campus, Apogee is my new home away from home. I’m still learning about this company, but as its newest addition, allow me to share a little about myself.

College was a good experience, but as any engineering student could tell you, it was no picnic. There were plenty of projects and homework assignments to fill my time, but all that work helped me to become more efficient and allowed me to expand my abilities. My last year in school was my favorite. Not only did I have rock climbing in my schedule, but I was also able to take the senior design class. I had always looked forward to that class; an opportunity to get hands-on experience in applying what I had learned. My team built a transformable wheelchair. I did all the modeling and supervised the building phase of the project, both of which were great experiences. I also created a poster for the chair, which summaries the project and explains why such a wheelchair is needed.

While my degree is in mechanical engineering, I don’t view myself as a typical engineer. I don’t particularly enjoy math, but I like solving problems. I took business classes to broaden my knowledge, and to open more doors. I like to work with others and I know the value of teamwork. My inquisitive nature and desire to create solutions led me into engineering, but it is also leading me to learn as much as I can from all areas of study.

Here are some more semi-applicable facts about me, in case you’re curious:

• I’ve been married for almost 3.5 years (feels like 2 max).
• I want to get an MBA. However, because of the high costs associated with attending, those plans have been put on hold.
• I’ve always wanted to own a business. It doesn’t matter if it turns into a big corporation, or if it’s just a small side project.
• A life goal of mine is to invent something. Something awesome.
• I’m published! I was able to write a guest article for SAE Momentum magazine about my thoughts during an internship experience. ("Outside the Classroom." SAE Momentum (2011): 2. Print.)
• I read a lot of tech news to stay up to date on the latest gadgets and inventions.
• My dad (David Lindsley) is an artist. You might even recognize some of his work.

I feel privileged to be able to work with such high caliber people as those here at Apogee. We make some amazing products and while I’ve only been here a short time, I have been able to see the drive and determination in everyone to provide the very best to our customers. I’m eager to learn all I can, and contribute everything I know to this company. Whether it’s modeling parts using Solidworks or crunching some numbers with Matlab, I’m excited to spend my time helping to research and create products that will enable our customers to ‘Make Better Measurements’ of the world around us.

 



Ryan Lindsley
Mechanical Engineer

Featured Projects - Solar 4R Schools

Our Featured Project series is a new blog series where we highlight what some of our customers are doing with our products. If you have a project you would like to have included please email us at admin@apogeeinstruments.com. 

{Photo from www.solar4rschools.org}

About a year ago, we were contacted by one of our customers to answer some technical questions about our pyranometer sensors. We talked to Craig Collins from Bonneville Environmental Foundation (http://www.b-e-f.org/) about a project they had developed called Solar 4R Schools (http://www.solar4rschools.org). The program works to put renewable energy technology in school buildings throughout the United States. The technology gives students a chance to learn about renewable energy as well as providing the schools with electricity.

Bonneville Environmental Foundation was founded in 1998, with a mission of “bridging the gap between different ideologies, the opposing views of political parties, environmental and business leaders to define common ground and solutions to our energy and water issues”. They work to “find innovative, high-impact solutions to some of the nations most pressing energy, carbon and freshwater challenges” (http://www.b-e-f.org/meet).

{Photo from www.solar4rschools.org}

Solar 4R Schools was founded in 2002 and has installed more than 100 photovoltaic systems for schools and community centers. It has also introduced more than 40,000 students to renewable solar energy. Through their outreach, they plan to educate more than just students. They hope to show parents, teachers, school administrators, city officials and even the installing contractors that solar energy is safe and reliable and can be a viable energy source for more than just schools.


Solar 4R Schools uses our SP-110 pyranometer and our free tool the ClearSky Calculator to monitor the solar panels they install in schools. The “About” page on Solar4RSchools.org says, “Solar 4R Schools educates students, teachers and community members about the science and benefits of renewable energy technology. The program provides hands-on activity guides, science kits and demonstration solar electric systems at no cost to schools, by working with local funding partners who want to show their commitment to renewable energy education. To receive a solar-electric system, schools must agree to own and maintain the system after installation. In turn, the school receives an exciting learning tool and all of the clean, renewable electricity it produces.“

Liberty High School in Renton, Washington features a solar panel with a dual-axis tracker and an Apogee SP-110 pyranometer. Solar 4R Schools is working on a new website feature called the Data Exploration Center where guests and students can see and share project data.

 At Apogee our ongoing goals include a commitment to creating innovative instrumentation for measuring climate change, improving sustainable food production and developing renewable energy. We are proud to have our sensors included in this great program. For more information about Solar 4R Schools, or to apply for a project at your school, please visit www.solar4rschools.org.







Whitney Mortensen
Marketing & Graphic Design

Energy Benefits of Highly Reflective Rooftops

Ten years ago I began building my own home in Nibley, Utah. When it came time to pick the color for the asphalt shingles I talked my wife into a light gray color, compromising aesthetics in lieu of performance. Although I didn’t have any data to back it up, I knew that a lighter roof would heat up less from solar radiation, reducing the need to cool the house in the summer. I rationalized that in Cache Valley, the roof is covered with snow most of the winter and so the benefit of a dark roof helping to heat the house would be minimized in that regard. I never measured the temperature of my roof, but I knew the light color made a difference because our house stayed cool while running our swamp cooler less often than our neighbors. Now that I am at Apogee, my understanding of albedo far surpasses what I knew then, and I could measure roof temperature with an infrared sensor to confirm the benefit of a light-colored roof.

My interest in the subject of high-albedo roofs was piqued when one of our customers forwarded me a copy of a paper (Gaffin et al., 2012 – see full citation below) they recently published. To partially mitigate the urban heat island effect, roofs with a high albedo membrane, such as EPDM (ethylene—propylene—diene monomer) and TPO (thermoplastic polyolefin), are used in new construction. Much of the study compared the difference in surface temperatures between black and white surfaces composed of the various roofing materials previously mentioned. The customer used an Apogee infrared radiometer to measure rooftop membrane temperature in New York City. For the TPO test case, a black geotextile cloth sample was used to compare against the white membrane. Peak temperature differences reached as high as 23.7 oC with average daily temperature differences equaling 6.6 oC. All surfaces had comparable initial albedos, with the white acrylic paint experiencing a considerable decline in albedo over time. Location is another significant factor. The acrylic paint site was possibly affected by a large traffic artery adjacent to the test site building, as well as an above ground subway line. Another site was affected by leaf litter and vegetation debris from surrounding trees.

When our current building for Apogee Instruments was constructed, a 60 mil TPO membrane was used. Historically, urban roofs have been asphaltic to allow for varying roof geometries, as well as the need for numerous roof structures such as heating and cooling units, ducts and vents. These asphaltic membrane roofs are typically topped with sand, gravel or rock and have low albedos. Gaffin et al. (2012) also looked at a white elastomeric acrylic paint that has been developed to apply to asphaltic membranes and is being used by New York City’s government for an albedo enhancement program. The white acrylic paint does significantly increase the albedo of the existing roof materials but the decrease over time indicates that all new construction should use EPDM or TPO membranes. Additionally, low emissivity may be a benefit in colder climates and this parameter should be considered in future performance standards.

I found the article very interesting and was surprised at the peak temperature differences demonstrated with different colors of the same material. It was validation for me of the choice to use a light gray shingle color when building my house. I was also pleased to see that when Apogee built our building, the materials selected help minimize heat gain, and in turn make the building more environmentally friendly. I appreciate working for a company that has a commitment to environmental responsibility and that builds sensors to promote research into renewable energy and green building practices.

Gaffin, S.R., M. Imhoff, C. Rosenzweig, R. Khanbilvardi, A. Pasqualini, A.Y.Y. Kong, D. Grillo, A. Freed, D. Hillel, and E. Hartung, 2012. Bright is the new black – multi-year performance of high-albedo roofs in an urban climate. Environmental Research Letters 7:014029 doi: 10.1088/1748-9326/7/1/014029.

 

Devin Overly
General Manager

Revisiting the Clear Sky Calculator

I usually spend every other Saturday in my hometown, working as a butcher in my Dad and Uncle’s grocery store. Last time I was there, during an afternoon lull, I noticed a calibration verification sticker on the scale used to weigh meat (not unlike the verification stickers found on gas pumps). Accurate calibration of the meat scale is required to ensure that the customer gets what they are paying for, and to ensure my Dad and Uncle are making a reasonable profit on goods being sold. I’ve never been around to witness the USDA inspector verify the calibration of the scale, but I can imagine he or she carries a small set of reference weights that are placed on the scale to determine the scale accuracy and need for recalibration.

Customers often contact Apogee Instruments to inquire about how often pyranometers and quantum sensors should be recalibrated. While it is safe practice to follow the general recommendation to recalibrate radiation sensors every two years, it may not be necessary to recalibrate on a fixed schedule if a sensor consistently matches a reference. The challenge for many pyranometer and quantum sensor users is they don’t have a handy reference, something analogous to weights that can be placed on a meat scale.

In response to customer inquiries regarding recalibration requirements, in 2009 Apogee Instruments developed the Clear Sky Calculator (www.clearskycalculator.com), an online tool that can be used to estimate the intensity of solar radiation (either total global shortwave radiation, measured by pyranometers, or global photosynthetic photon flux density, measured by quantum sensors) incident on a horizontal surface at any time of the day, at any location in the world. The equations used to estimate clear sky solar radiation with the Clear Sky Calculator come from the clear sky solar radiation model used to calculate net radiation in the ASCE Standardized Reference Evapotranspiration Equation (http://www.kimberly.uidaho.edu/water/asceewri/index.html). The only input requirements to the calculator are site elevation, latitude, longitude, reference longitude, and air temperature and relative humidity measurements or estimates. These data are typically easy to obtain, making the Clear Sky Calculator a simple solar radiation reference that can be used to estimate pyranometer and quantum sensor accuracy and determine the need for recalibration.

When used near solar noon over multiple clear, unpolluted days during spring and summer months, accuracy of the Clear Sky Calculator is estimated to be ± 4 % in all climates and locations around the world. As an example, modeled incoming shortwave radiation (SWi) from the Clear Sky Calculator closely tracked measured SWi (data from a heated and ventilated Kipp & Zonen CM21 pyranometer) for a clear day (April 21, 2012) in Logan, Utah (Figure 1). The ratio of measured SWi to modeled SWi was between 1.00 and 1.05 (0 % and 5 %) from 9 AM to 6 PM (solar zenith angles less than 65°) (Figure 2). The average ratio from two hours before solar noon to two hours after solar noon was 1.02 ± 0.01 (2 ± 1 %). A more detailed discussion of Clear Sky Calculator accuracy is given on the webpage (http://clearskycalculator.com/model_accuracy.htm), where the necessary accuracy of the required inputs is discussed.

Apogee strongly encourages our customers to use the Clear Sky Calculator as an effective way to monitor pyranometer and quantum sensor performance and determine the need for sensor recalibration. If a sensor is consistently different from the Clear Sky Calculator by more than a few percent, please contact us about recalibration.

Figure 1: Comparison of measured incoming shortwave radiation (SWi) (red line) and modeled SWi on April 21, 2012 in Logan, Utah. Measured SWi is from a heated and ventilated blackbody pyranometer (Kipp & Zonen model CM21). Modeled SWi is from the Clear Sky Calculator.

Figure 2: Ratio of measured shortwave radiation (SWi) to modeled SWi over the course of a clear day (April 21, 2012) in Logan, Utah. Mean ratio = 1.02 ± 0.01 (2 ± 1 %) for measurements averaged from two hours before solar noon to two hours after solar noon (solar noon occurred at approximately 13.5). The dip in the morning near 7 is due to mountains on the east side of the valley where Logan is located.

 

Mark Blonquist
Chief Science Officer