The EU has announced that it
will investigate multiple reports that Samsung’s LCD panels cheat on
energy efficiency by detecting when standardized tests are run and
adjusting dynamic brightness levels in order to obfuscate their actual
panel use. Samsung has denied any wrongdoing — while the company
acknowledges that power use can vary significantly by content, it claims
that the observed differences are caused by general energy efficiency
measures rather than any attempt to obfuscate a test. The EU is unlikely
to accept any company’s word on such topics in the near-future, however
— not after the recent VW scandal that’s rocked the auto industry.
Samsung’s hardware was tested by the EU’s ComplianTV group, which found that Samsung’s “motion lighting” feature reduced power consumption under internationally accepted test conditions, but not when television sets were deployed in normal use. A second Swedish group found evidence of additional cheating. “The Swedish Energy Agency’s Testlab has come across televisions that clearly recognize the standard film (IEC) used for testing… These displays immediately lower their energy use by adjusting the brightness of the display when the standard film is being run. This is a way of avoiding the market surveillance authorities and should be addressed by the commission.”
Unfortunately, there’s a confounding variable in all of this. Samsung may or may not be cheating on these standardized tests, but power consumption in an LCD panel can absolutely vary depending on what you’re doing with it. In his recent LCD vs. OLED shootout, Dr. Soneira posted power efficiency figures from multiple types of content between Samsung and LG displays:
What’s tricky about this situation is that it’s possible for all
sets of claims to be true. Samsung claims that its motion lighting
feature is meant to reduce power consumption in certain types of
content, the EU is investigating claims that Samsung cheats, and it’s
obvious that different types of content produce very different levels of
energy consumption. What this will come down to is whether Samsung and
other TV manufacturers are claiming to hit efficiency targets in
specific types of content that they don’t actually deliver, and whether
the TVs operate in the same mode when displaying standard content as
with test content.
To some extent, this illustrates the problem with trying to create formal metrics for measuring performance. On the one hand, you want to have a unified set of tests that any organization can deploy to test common hardware. Designing custom benchmarks or sequences is extremely time-consuming, and there’s always the chance that you may miss some type of content that ought to have been measured, but wasn’t. As TVs become more sophisticated, however, they have gained the ability to detect and obfuscate certain kinds of tests.
These results also highlight the dangers of allowing companies to self-test and report the efficiency of their own hardware. Allowing the fox to guard the henhouse may save dollars in the short run, but it’s an inevitable recipe for long-term trouble
Samsung’s hardware was tested by the EU’s ComplianTV group, which found that Samsung’s “motion lighting” feature reduced power consumption under internationally accepted test conditions, but not when television sets were deployed in normal use. A second Swedish group found evidence of additional cheating. “The Swedish Energy Agency’s Testlab has come across televisions that clearly recognize the standard film (IEC) used for testing… These displays immediately lower their energy use by adjusting the brightness of the display when the standard film is being run. This is a way of avoiding the market surveillance authorities and should be addressed by the commission.”
Unfortunately, there’s a confounding variable in all of this. Samsung may or may not be cheating on these standardized tests, but power consumption in an LCD panel can absolutely vary depending on what you’re doing with it. In his recent LCD vs. OLED shootout, Dr. Soneira posted power efficiency figures from multiple types of content between Samsung and LG displays:
Click to enlarge
To some extent, this illustrates the problem with trying to create formal metrics for measuring performance. On the one hand, you want to have a unified set of tests that any organization can deploy to test common hardware. Designing custom benchmarks or sequences is extremely time-consuming, and there’s always the chance that you may miss some type of content that ought to have been measured, but wasn’t. As TVs become more sophisticated, however, they have gained the ability to detect and obfuscate certain kinds of tests.
These results also highlight the dangers of allowing companies to self-test and report the efficiency of their own hardware. Allowing the fox to guard the henhouse may save dollars in the short run, but it’s an inevitable recipe for long-term trouble
No comments:
Post a Comment