There were two parts of testing – the less interesting but necessary restriction/flow testing, and the more interesting and hard thermal testing.  If you want to skip on to the thermal result analysis then please do – but if you’re interested in the technical details (and you should be) then read on.

Restriction/Flow Rate Testing

The goal of this test is to measure the pressure across a component as flow is varied through it.  This produces a plot like this one which enables the user to interpret how well a component may work in their loop with respect to pump power:

If you don’t know and would like to learn what this plot means, check out our guide.

The setup for this test involves a larger reservoir, a variety of pumps that get switched in and out including the mammothly beasty pump shown here (the Iwaki RD-30), a flow meter (to measure flow), a differential manometer (to measure pressure drop across the radiator) and a faucet (or tap) to vary the flow through the system.  The setup looks like this:
sr1_5The differential manometer is the black box with the yellow screen which is a Dwyer 490-3 wet-wet manometer.  The tall thing in the middle is a King Instruments Rotameter which measures flow.  Both of these were chosen to have sufficient accuracy for the testing involved.

Thermal Test Setup

Push/Pull vs Push vs Pull Fans

For testing of the first group of 15 – we initially ran Push/Pull only fan setups in order to maximize performance.  After feedback on this initial publication of data we added Push data in addition and went back to remeasure the Group A radiators also.  In order to save time on the push data we tested only at 1 GPM flow rate.

Radiators are by far the hardest water cooling component to test accurately and repeatably.  Note the “and” in that sentence.  Some reviewers can repeat their measurements, but it doesn’t mean that their performance numbers are accurate, just that they are doing the same thing consistently.  Let’s delve into the problems.

Signal to Noise Ratio

In electrical engineering we will often talk about signal to noise ratio, i.e. how strong is the signal that we want compared to all the crap that we want to ignore.  In other words how clean is it.  This idea carries through into testing.  I.E.  How large is your measurement compared to the sources of error.  The larger the ratio of measurement to error the easier it is to be repeatable and accurate.  When we judge radiator performance we are looking at how well the radiator cools coolant.  To measure this we are therefore fundamentally measuring air and coolant temperatures while we vary other factors (such as radiator, air flow and coolant flow). All of the following factors affect just how well we can measure air and coolant temperatures of the system:

Air currents

A large radiator has a whole lot of air being fed into it.  Some of that air will be of slightly different temperature to the rest of it due to localized air pockets in the room as well as air currents in the room due to wind, doors opening or movement about the room.  The only way to avoid such things is to use a chamber that the radiator is mounted in.  However unless that chamber is temperature controlled, or large enough it can not be sealed and so is still subjective to air currents around the room.  Ducting the inlet to such a chamber and baffling it therefore helps to shield the chamber from local air currents.  Note that the inlet should not be too restrictive otherwise the airflow will also be restricted.  This can be checked by measuring fan RPM – the fan should be running at the same speed it does when outside of the chamber.

Thermal Rig- Lr B&W-2

This is the test chamber that we built for this setup.  Note a plexiglass window is installed in the front to seal the chamber.

RPM vs CFM

So now that you’re forcing consistent amounts of air through a radiator are you keeping the fan speed constant or are you keeping air flow constant?  Then if you keep fan speed constant do you measure the air flow and do you plot performance vs air flow or vs fan speed?  In an ideal world we would strip things down to the basic ideas – i.e. what really matters is the airflow through the radiator.  Then you can simply test a radiator and correlate whatever airflow setting your setup has and measure off the performance.  However for this to be possible we would need a database of many fans and their airflows through different radiators under different situations.  However without such a huge database (and you can not trust a manufacturer’s fan specifications at all) then the data again becomes an approximation and at that point you’re back to correlating with one fan on one radiator.  By testing vs RPM we are at least testing with one type of fan across each radiator.  It’s also easier to understand at a glance and therefore for Noob’s to understand.

So yes – CFM is better in an ideal world, but sadly it’s not an ideal world that we live in.  Therefore all our plots are vs RPM of the excellent Gentle Typhoon AP15.  While this fan is essentially discontinued, the EK Vardar has very similar performance.  For all other fans with less static pressure you may need to adjust the RPMs a bit in your mind (i.e. an 1850rpm AP15 might be equal to a 2000rpm lesser fan for example) for the equivalent performance.

Temperature Sensors

Now that you have a reasonably shielded air flow input that is unrestricted you need to measure the air temperatures coming in.  This is not as easy as it sounds.  The air temperature will still vary some amount across the ducts.  We found that 8 temperature sensors were needed to get “acceptable” data for the air in temperature which was horribly noisey.  As you are primarily concerned with air temperature in, then you can skimp on the air temperature out temperature sensors (which are more uniform anyway). We used two sensors for air out for example which is only used to generate the “Air Efficiency” metric that is of little interest.

Thermal Rig Bits- Lr B_W-5

Now bear in mind that not all temperature sensors are equal. Our first iteration of the test rig used two Aquaeros and bunch of standard temperature sensors. These sensors even after calibration were found to be too inaccurate. Under a 300W load with fans at maximum speed we would see a 7-8 degree temperature delta. In order to acheive a +/- 1% error therefore we needed to have sensors with +/- 0.07 degree accuracy. The Aquaero setup was at best giving us 0.5-1.0C degree accuracy which is well beyond what was needed. We upgraded to a Crystalfontz setup with Maxim/Dallas “one-wire digital sensors”. Each sensor has an accuracy of 0.5C over a range of -10 to 85C. As we are operating in the middle of that range (30-40C) then we can expect greater accuracy, particularly after averaging many sensors:

error

Note that an offset on the mean doesn’t matter because the delta is a relative measurement and all probes used are the same. The sigma is really what is key here. The 3 sigma number is about 0.25 deg C. Because of this we fitted as many water sensors as we could (3 to inlet and 3 to outlet) in order to reduce the spread further. These coolant sensors had to be custom made where the sensor was glued into a fitting housing.

Ambient Drift

Assuming your chamber is not sealed then the air being fed into the chamber will have a certain temperature that will vary over time. This may be a slow drift through out the day or it may be a quick change as a door or window is opened, or an AC unit kicks on. Some reviewers make a big deal about having a temperature controlled environment by which they often mean that they have an AC unit programmed to a specific temperature but the temperature spike from an AC unit can absolutely destroy a set of results because it takes time for the system to equalize and settle out. Controlling ambients helps take out any common mode drift of your sensors, however if your sensors are good and generally drift together then a delta type measurement doesn’t get affected by it much. Controlling temperature spikes on the other hand is critical. Even in the case of a more advanced setup where a sealed temperature chamber is used, the chamber may still have heating and cooling devices that will kick on and off and cause such temperature spikes. The most advanced setups will use PID control to avoid spikes while maintaining a controlled ambient. Such systems however are very expensive, particularly in a large enough size to fit the test system as a whole inside. As we did not have such a system we had to make do with merely controlling the temperature spikes and suffering from some ambient drift. However ambient drift was minimized by only testing during the night. Daytime, and particularly sunrise/sunset caused too many problems in accuracy and repeatability so we went the extra mile and stayed up late for months on end to make sure we got the best data possible. How many reviewers would do that?

Thermal Rig Bits- Lr B_W-4

Data Logging

All these temperatures of course need to be recorded. There are two factors here:

1. The total amount of time you log for – i.e. the testing window. Generally the longer the window the more any temperature spikes get averaged out. However the longer period also means you are more likely to get a temperature spike. We decided to choose a 15 minute interval. Longer than that did give better data, but the downside was that it was hard to find a longer section of time that was clear of spikes. The other factor is that when testing a lot of radiators over a lot of different settings then the total time adds up. A review that never gets finished is a waste of time and at some point you have to say “it’s good enough”. We really do try and make things as scientific as possible, but even we sometimes have limits due to time and finances.

Thermal Rig Bits- Lr B_W-2

2. Logging frequency – i.e. how often is the data sampled. We did some experiments on this. Logging at either every 1 second or every 10 seconds showed little difference. However logging every 60 seconds showed an introduction of increased error. As 1 second produced tolerably sized data logs we went with that option. Do not ever manually log data by hand, for a start you can not do it fast enough. In addition humans are always the worst introduction of error, and the more you can avoid human error the better. Plus it means less work for the reviewer.

Thermal Rig Bits- Lr B_W-1-2

Heat Source

To test the cooling ability of the radiator we first have to heat it up. Some reviewers use a computer because they already have one and it gives the feel of “system setup” and how the radiator is used in real life. However this is a mistake in a similar way to a noob doing a ‘cpu block review’ by only ever looking at absolute cpu temperatures at idle load. The computer itself also cools coolant as well as heats coolant. You also have to wonder about consistent heat load. Running Valley for example on GPUs IMO is not good enough (which is why we test GPU blocks with Furmark instead). This is not said idley. Our initial setup tried to use this and the results were horrible. Our final setup used an inline wet aquarium heater located in a reservoir with a Variac to dial in the power load exactly. It also had to be modified to stay “always on”. So why is using a computer so bad?

Thermal Rig Bits- Lr B_W-7

Insulation

A computer as a heat source is bad because of insulation, or rather a lack of it. For a start you do not know the heatload going into the coolant. You can measure input power to the PC, but you have no idea what percentage of that power is transferred to the coolant. Once the coolant is heated, how much of that heat is lost to the surroundings before it gets cooled by the radiator? How does that amount of cooling vary depending on ambients or localized air flow changes. While we mentioned 7-8C deltas for the high air flow cases when running low air flow cases then the deltas can be significantly higher (20C). With a high temperature delta it’s actually quite impressive how much heat can be dissipated simply passively to the air by blocks, tube and fittings. To be accurate you have to insulate the crap out of everything *but* the radiator, which let’s be honest you probably don’t want to do with your PC. It is also worth using norprene tubing in order to provide some insulation on the tubes too.

Thermal Rig Bits- Lr B_W-6

The advantage of an aquarium heater sitting in the water is that 100% of the power turns into heat that transfers to the coolant because there is nowhere else for it to go. This gives the advantage that we can then calculate the coolant rise for a certain amount of power, or indeed the flipside of that. That is, the amount of power for a fixed coolant temperature rise.

Flow rate

What flow rate are you going to test at? Is it fixed or are you fixing pump power? In a test setup that uses a computer I generally favor fixing pump power because it means a more restrictive component will get lower flow just as it would in a real system. However as we are not using a “real” system to test then the system may be much more sensitive to restriction changes and so we decided to go with fixed flow rates instead. This will give a slight benefit to radiators that are more restrictive. Luckily the majority of radiators are not very restrictive. Sadly there is no right answer on this one, luckily it doesn’t matter much as long as you’re consistent.

The other factor is how do you achieve the flow? Obviously a pump is required but if you are fixing flow then how do you make it the same given different radiator restrictions.  Do you use a fixed pump power and then vary flow with a tap/faucet or vary the flow of the pump by PWM or voltage control? We are using the heater power input into the loop as a factor in the calculation so we either need to know or limit any pump power dump to the loop. By measuring the pump power we could try and choose some percentage to add as a fudge factor. However it’s hard to know what the fudge factor should be. I believe Martin (of Martin’s Liquid Lab) used 6W out of 15W measured for example. However instead we took a different approach. We tried to reduce power dump to the loop by using a DDC pump with PWM. DDC’s generally do not dump much heat to the coolant anyway. In addition PWM generally causes the pump to heat up less than a voltage controlled version. The DDC then had a heatsink attached with a fan blowing air on it so as to avoid any pump heat being dumped in the loop. At the end of the day this is a source of error and we hope that the heatsink only dumps heat from the pump, and does not cool the coolant. We also hope that the pump itself does not dump any heat to the loop. We also avoided using a faucet to control flow as they can sometimes cause turbulence that can affect results.

Thermal Rig Bits- Lr B_W-10

Repeat, Repeat, Repeat…

How do you know how good your results are? Well do they make sense, are they inline with expectations? Do the measurements and calculations still work when the heater is turned off. Are they repeatable? Before we embarked on retesting all our rads for the 2nd time in the “improved” setup we did tons of correlation runs to prove out repeatability. Then after every five runs we repeated a radiator in order to make sure nothing had drifted over time. With all this repetition we never saw more than 2% between results, indicating a possible +/-1% test error. In addition we took a minimum of 2 data runs and averaged for increased accuracy. Later tested radiators would often have 4-10 runs in order to increase accuracy further.  The push/pull numbers that average across all three flow rates are expected to be even more reliable.

Retrospect

In retrospect it might have been better to increase and decrease heater power so that approximate deltas always remain the same. Running a 15C delta at all times for example would really help the high air flow cases were errors will be larger due to smaller deltas in the current setup.

So in summary..

1. Build a test chamber
2. Duct the inlets
3. Use really accurate temperature sensors
4. Use lots of these temperature sensors
5. Control ambient temperatures as much as possible both static and dynamically
6. Log over a long enough time period
7. Log frequently enough
8. Use a modified always on aquarium heater that lives in the coolant
9. Insulate everything. Use Norprene tubing. Insulate some more.
10. Fix flow rate rather than pump power
11. Either isolate pump heat from the loop, or measure the pump power, insulate and correlate the heat dump to the loop, then add to your calculations
12. Repeat tests every so often to check for drift over time and that accuracy is meeting expectations

If you’re comparing our numbers to a review that hasn’t gone through similar steps then the results are simply not going to be as accurate.  Now if you’ve made it through all of this text – you’ll be hungry for some real data, and you’ll be aware of just how bad some other test data is that is out there. We are not trying to belittle the work of others but instead we hope to set standards for increased testing accuracy that help to reward good design!  Inaccurate testing rewards poor products which leads readers to buy poor products.

Thermal Rig- Lr B&W-3

And now… finally… results are up next!

RELATED ARTICLESMORE FROM AUTHOR

76 COMMENTS

  1. Great Work … a new standard for radiator testing.
    The Koolance was a surprisingly good result. Push pull makes a bigger difference than many think.
    I’m happy my EK PE rads did OK, mostly keeping up with the XT45 rads they replaced.

    • The extra 5mm the PE rads allowed made the difference between push only and push pull for an overall better result

    • Yeah the huge shrouds on the PE mean that it’s really a slim radiator core tucked into the body of a medium radiator wrapper. I do wonder if reducing the shrouds would have made no real change in performance but would have meant it could have won a slim award!

  6. Weighing in a bit late, but I just found this article (and the site). This was a great read! Thank you for doing this type of thing. There are many of us out there who want to upgrade, add to, or change our rigs completely. In depth articles like this are the life-bread for us regular folk to gather REAL information.
    I do have a question and of course am hoping you find this comment: How about testing the XSPC AX series radiators? I don’t know how you get your samples, but it would be interesting to see how they perform against the rads in this test.
    More fan tests would be a wonderful thing as well. Enermax has had the UCTP12P High Static Pressure PWM 120mm Fan out for some time with no virtually no information anywhere as to their validity for use a radiator fan. I noticed you folks have a pre-production sample Vardar from EK. I look forward to seeing how that plays out in production.

  13. Hey fast nate

    i was wondering how close you are to finishing the radiator roundup with just the push fan setup? thanks man

  16. Well, I think that the reviewers always get a mint rad. I already had to send back a Phobya G-ghanger 360 V2, a Phobya G-canger 420 V2, a Alphacool UT 60 420 , a Coolgate CG360CG2 and a Coolgate CG480CG2 , this because of heavy paint faults, scuff marks on the side panels , matt grey spots where paint is of, and the phobya 420 had a piece loose inside the rad. Al bad rads and these al got delivered and send back the last 2 months. The brands are asking good money, but deliver bad quality.

  21. I just posted this same comment on the other review for the Coolgate CG-360G2. I want to thank you for this extensive and extremely detailed review; an updated version to SkinneeLabs.com’s review and Martinliquidlab’s review on radiators (found here: https://martinsliquidlab.wordpress.com/category/radiators/). I was debating (as I’m a reseller for Koolance), weather or not to purchase the Koolance HX-360XC product or the Coolgate CG-360G2. As a result of your thorough testing I’m going to go with the Coolgate CG-360G2. I know you mentioned that it’s hard enough to put just one of these bad boys in your CaseLabs PC Case, but I’m going to put TWO of them in. My config will be set up like this:

    25mm Fan/65mm Rad/25mm Fan/65mm Rad/25mm Fan = 205 (sandwiched together side by side)

    Which will just cover the bottom of the width of my Origin Genesis PC Case. I’ll post the results later in a couple of months once I’m finished with my (hopeful) new Hackintosh build. We’ll see what happens. Again, thank you for your help. Greatly appreciated !!!

  30. Looks like the review is complete! Kudos!

    Yeah, I can guess how much it took to go through this painstaking review of 27 (!) radiators, and 486 (OMG!!!!) tests – and that so accurately and reliably. My greatest respect to you fast_fate, and all those who helped you on it.

    Needless to say, we are all missing the XS-PC AX360. Could you reach out to XS-PC and ask for a sample maybe? Or get a retail one? Anyway, we are all looking forward to that.

    • The Round Up will never be complete….
      Thanks for following along through the journey.
      It is a massive task to update, too much work to keep updating the individual reviews……
      which is why we had to make the decision to not keep updating all the screen-shots included in each indivdual radiator review.
      As each radiator test results is linked to every other radiator’s results for the relative scoring system, when a new set of results are added – all the old published data became redundant.
      So from now on, each new rad will still be included in the Round-Up, but will have a slightly different format in the individual reviews, so all currently published results do not appear incorrect.

      The AX360 has always been a noticeable ommission from the list.
      XSPC were kind enough to sponsor the RX360 V3 for inclusion, but the AX was never sent.
      If one becomes available it will be an inclusion for sure – but expect performance results to be very very similar to the EK PE360.

  32. […] mal ein umfangreicher Radiatortest, auch mit push/pull: Radiator Review Round Up 2015 – ExtremeRigs.net Verkaufe Plextor M6e 512GB, i3 & Pentium CPUs, Alphacool NexXxos ST30 360 Radiator. […]

  33. I’d really love to see similar testing on 140mm class radiators too. 140×2 (60.8 sq in) would be awesome since they’re so close to 360 rads (66.9 sq in)

  39. The links for the EK SE/PE reviews have become combined and lead to the SE review.

  44. About the awards, if the XTX360 were available along with the XE360, where would it place? Would it replace the SR2 360 as runner up or share a “co-spot” with it or another radiator?

    • Most likely it would be a runnner up alongside the SR2 MP. It has similar thermal performance, less optional ports than the SR2 MP, but it gains a drain port. Still a great radiator. I have to say though if you’re buying EK, the only reason to get the XTX over the XE is price, aesthetic or because you really want that drain port.

  45. […] to take good data requires a lot of attention to detail and data logging time.  Just look at our radiator round up for how hard it is to accurately measure air temperature into a radiator.  The reality is that the mainstream review sites simply don’t have time to […]

  47. Im kinda of noob on this, dont understand to much the graphs.

    According to this review, what rads should i use for using Scythe GT ran @ idle =400rpms / load = 800rpms ? HL SR2 or EK XE or another?

  49. Was wondering why you didn’t have any Swfitech rads. Was it you could not get any?
    I used Swiftech, EK, XSPC and Alphacool
    I end up most times going with Swiftech for most of my builds.

    • Swiftech would not sponsor, it would be helpful to let Swiftech know that their buyers like our reviews and would like them to participate 🙂

  50. Great job guys! Keep it up.
    After Martin has abandoned testing, you are the first who uses such high quality engineering standards.
    I think that you should think about more SEO for you site, found it accidently 🙂

  51. Your perspective is all wrong because your all young men and look to buy a single radiator that has the best specs but the price to performance ratio is really what matters because you could take the side of the case off and turn it into a wall of radiators so really being compact and having the highest spec radiator is simply a waste of money unless of course you used all the available nearby space(really this is impossible) and thus cooling power per dollar is king(as most people are not rich) and buying multiple magicools is truly the best solution(rad stacking doesnt work each one needs its own fresh supply of cool air you will have to think about your design. Really for the average person running your pc with the radio exhausting push pull and feeding a tube of air conditioned air off your air conditioner into the case sealing it and having all the rads exhausting is the cheapest easiest setup for cooling during gaming sessions. Then when your done gaming turn your ac off(you also have to set the temp very low so it runs continuous which also leads to the evaporator coils icing over after many hours(or days if temps above 70f into the 80s of course every ac is different) or outdoor temps below 70f). But this is hardly very efficient also if you have many magicool rads you dont really even have to run fans you can cool the pc passively while its idle and only turn the fans and ac on when you game that way you would spend the absolute least amount on electricity. Of course to be completely efficient with power would require allot more money and design skills in the future we should have much better condenser units outside with far more efficient assymetrical blade fans to limit noise and daisy chain from a very large outdoor compressor providing the first stage to smaller ones inside the house possibly even coupled with very large water cooling loops to second and third stages tied directly to the main processors. That way you get phase cooling to all your crucial chips and all heat is exhausted outside direct connections to the cooling loops would be the most efficient. We also need systems to switch from active phase for gaming and another maybe water cooling loop passive or with light fans for idle or home office web browsing usage.

  52. Great job and good review.

    In the last my worklogs alwats I installed 1×360(60mm) and 1×240(60mm).

    And now I would like if is this necessary for 5930k and 980 Ti Classified with a modered OC or is valid with a 2×240 and 30mm thick.

    Thanks for your job

    Regards!!!

  53. Hi, maybe that is a noob question, but are you only testing push configurations because push is better than pull? Heard some people say they have better results with pull.

    • Originally we did only test / publish Push Only results after our comparison tests showed Push Only performed better than Pull Only.
      Then we added Push/Pull to go along with the Push Only data.
      To do Pull only would add a significant amount of time to the testing process, so we chose the best of the two (+ Push/Pull)
      Some fan manufactures say Pull Only is what the fans are designed for and will possibly even last longer when run in that configuration.
      Without some data to back those statements up we are kind of skeptical. We’ve had fans fail prematurely in both scenarios and likewise have fans which have run seemingly forever.

      • Thanks for the detailed answer! Did your comparison tests show that push only also works better at low fan speeds (below 1000rpm)? I read that people having slightly better results with pull only at low speeds.

  54. Has anyone actually tried fitting HWL nemesis GTX 420 radiator ontop of a define r5 or define S ?
    I cant make up my mind if I should go for a rad that certainly fits like EK-CE 420 or Alphacool UT60 but hwl nemesis GTX is the best radiator but im not sure if it fits or not, it clears the memory wich is the highest part out of my mobo.

  55. Awesome review! I’m wondering if any of the brands you tested had radiators that had fill ports that were on both sides? I’m wondering if anyone knows of a radiator that, like many Thermaltake Pacific radiators, has ports on both sides of the radiators. I’m planning a setup in which several radiators are stacked to do an experimental convection driven counter current loop (the coolest air cools the coolest radiators, the warmest radiators cooled by the warmest air, and all the hot air pulled out the top). That means that it would be most efficient to have the “in” ports on one surface of radiator, and the out on the other surface. Unfortunately the Thermaltakes are made of aluminum, which I don’t really trust…

    • Just off the top of my head – Alphacool have a bunch such as the UT60, XT45. HardwareLabs SR2 MP version does. Yes I wouldn’t trust those TT rads. BTW HardwareLabs does have a front to back type flow on the Nemesis GTX if thats what you’re looking for. It helps at lower flow rates although they advise the coldest air to get the hottest coolant, but once flow rate is high enough then the coolant is already ‘fully’ cooled before it leaves the rad anyway.

Comments are closed.