|
|
|
|
markanite
Dec 9, 2008, 8:32 PM
Post #1 of 54
(17947 views)
Shortcut
Registered: Jun 30, 2003
Posts: 282
|
So last weekend I purchased the top 12 belay devices on the market and measured the friction of each device while locking-off and feeding. Each device was measured using 8,10 and 11mm ropes (10 tests per diameter and then the results were averaged; 720 tests total).
The full results are available here:
http://www.spadout.com/...evice-friction-test/
I'd especially encourage RC.com readers to look at the raw data near the bottom because the numbers on the top are calculated for beginners (who need a high friction lock-off and don't need a fast feed rate).
I'd love to hear your guys feedback on this (both positive and negative). Are there any additional tests you would be interested in (for belay devices)? Any ideas how we can incorporate dual friction modes (currently the high friction mode was used if the device was dual friction) or auto-blocking features into the test without making it subjective?
Thanks,
Mark
(This post was edited by markanite on Dec 9, 2008, 8:33 PM)
|
|
|
|
|
majid_sabet
Dec 9, 2008, 8:55 PM
Post #2 of 54
(17899 views)
Shortcut
Registered: Dec 13, 2002
Posts: 8390
|
interesting,but why not measuring the load at the belay device by dropping it from a fix line while keep the rope @ 45
|
|
|
|
|
roy_hinkley_jr
Dec 9, 2008, 9:12 PM
Post #4 of 54
(17861 views)
Shortcut
Registered: Sep 8, 2005
Posts: 652
|
I suspect you'd see substantial differences by greatly increasing the load. 31 lbs. isn't enough to deform the rope. Doubtful many gyms use 11mm anymore, probably 10.5 is the standard now. The 8mm rope needs to be wet for alpine simulation. No belay test is complete without a Munter in both orientations.
|
|
|
|
|
markanite
Dec 9, 2008, 11:35 PM
Post #5 of 54
(17754 views)
Shortcut
Registered: Jun 30, 2003
Posts: 282
|
Roy - I agree. Munter Hitch is a great idea. So I re-tested the setup with the munter hitch.
11mm Lock-Off: 3.31 lbs
11mm Feed: 3.56 lbs
10mm Lock-Off: 2.5 lbs
10mm Feed: 6.9 lbs
8mm Lock-Off: 3.8 lbs
8mm Feed: 6.6 lbs
Now this test isn't an exact comparison. Reason being that munter hitch increases friction when the rope is raised (when being belayed from your harness to a leader; obviously belay devices are vice versa). Therefore (because I didn't drill new bolt positions) the lock off was at 45 and the feed at 55. Still interesting data though for a quick comparison . I could do it correctly again when I do the next round of tests.
To no surprise the range is smaller. The interesting part (imo) is how well the 8mm lock-off did. If you think about this though it isn't extremely surprising (because of the small diameter and the looseness of the 8mm rope it makes a very sharp bend). The downside with the Munter is that feeding it is tough.
Majid - We did tests at various weights. We started at 10 lbs and did not see proportional results until we got about 30 lbs (hanging). 30 lbs,40lbs and 50lbs gave proportional results. We also pulled slack back toward the "hand" side of the device and dropped the weight 6 inches (allowing it to feed naturally). This was critical because the way the device sits is a big factor in the overall force (primarily due to leverage; for example if I set a belay device at 45 degrees towards the lock off direction and slowly lowered the weight allowing it to manually set at that position, the reading would be extremely high). At all of these weights the rope was squished and pressed against the device. I'm presuming that's why we didn't see consistent results until we got above 30 lbs. Ropes are not particularly stiff (obviously stiffness of the rope is another potential factor) though so the amount of weight to get to a reasonably consistent deformation isn't very high.
With that being said you are correct extended falls will generate (a lot) more force. The real challenge here is building an appropriate setup that can accurately monitor the maximum force and even better track force through out the fall + catch. This is currently out of our budget but we'd love to try it again in the future.
Acorneau - any device in particular? I agree narrowing the rope range would be interesting. We settled on three diameters for now to narrow the results to (only) 720 tests. Sorry, I try to be tough but lifting that weight 720 times in one 11 hour testing session left me sore. Do it 3000+ would have killed me. Once again though a more automated system would fix this and when the budget allows that is possible as well.
(This post was edited by markanite on Dec 9, 2008, 11:36 PM)
|
|
|
|
|
majid_sabet
Dec 9, 2008, 11:44 PM
Post #6 of 54
(17742 views)
Shortcut
Registered: Dec 13, 2002
Posts: 8390
|
Mark
Not everyone uses the same type of biner and I bit that you could get a variety of results with other type of biners as well.
|
|
|
|
|
patto
Dec 10, 2008, 3:43 AM
Post #7 of 54
(17683 views)
Shortcut
Registered: Nov 15, 2005
Posts: 1453
|
Nice work there. Its good to see some one put the work in to get some data.
One thing that I do question is why not the -90degree lock off? That is what Petzl recommend and certainly what I do. You mention you don't recommend locking off between you legs, why not?
(I'd more call it locking off in front of your legs)
|
|
|
|
|
JimTitt
Dec 10, 2008, 9:54 AM
Post #8 of 54
(17609 views)
Shortcut
Registered: Aug 7, 2008
Posts: 1002
|
A lot of work on the braking force of a number of belay devices was done by P Randelzhofer in 1996 in Munich as his doctorate. He established a lot of the parameters such as the "simulated hand" (along with the work of K & K Mauthner in 1994) which have been generally accepted as giving useful results.
More research was carried out by the DAV at TUV Bayern (the UIAA and CE testing laboratory) which included some other more recent devices (Grigri, Sirius TRE etc) as well as Munter and Fig.8and also used live belayers as a "real world" comparison. These tests included rope slippage distance and runner loading as well as braking force.
All these are of course in German, and in the case of Randelzhofer a long-winded dissertation as well, but I could translate some of the relevant parts if it would be useful.
|
|
|
|
|
USnavy
Dec 10, 2008, 10:27 AM
Post #9 of 54
(17605 views)
Shortcut
Registered: Nov 6, 2007
Posts: 2667
|
When you tested the Trango Pyramid did you test it in the low or high friction mode?
|
|
|
|
|
JimTitt
Dec 10, 2008, 1:06 PM
Post #10 of 54
(17577 views)
Shortcut
Registered: Aug 7, 2008
Posts: 1002
|
Why ask me?
Anyway, read the report carefully and you will see the answer, it´s all in the details.
|
|
|
|
|
USnavy
Dec 10, 2008, 1:40 PM
Post #11 of 54
(17561 views)
Shortcut
Registered: Nov 6, 2007
Posts: 2667
|
oops sorry I wasent asking you, I ment to ask the OP.
|
|
|
|
|
jt512
Dec 10, 2008, 5:08 PM
Post #12 of 54
(17511 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
I object to your combining three disparate parameters into a single performance index. First of all, any weights you assign to the parameters are completely arbitrary and unlikely to reflect how any one climber would weigh the same factors himself. There is really no sensible way to combine braking force, feeding friction, and device weight, so don't try. Simply present the results for each variable and let the end user decide their relative performance.
Secondly, combining disparate performance factors into a single index results in the index being uninterpretable. For instance, why does the Trango Jaws receive such a high rating with 11-mm ropes? Is it because it has tremendous holding power, or because it feeds so smoothly. There's no way to tell from the rating.
Furthermore, your combining algorithm has resulted in bizarre, implausible U-shaped relations between performance and rope diameter. For instance, you rate the Omega Pacific SBGII 8.9, 8.0, 9.3, for 11-mm, 10-mm, and 8-mm ropes, respectively. It is unlikely that a typical belay device would perform better with very high- and low-diameter ropes than with average-diameter ropes.
Also, it is silly to present the weight of the belay device in your main results table in concocted units. What's wrong with grams?
On another matter, were the belay devices tested sequentially, or did you mix up the order of testing. If you did all the reps with one belay device, then did all the reps with the next belay device, etc, it is at least logically possible that the observed results could have been affected by changes in the elasticity of the rope with time and use. Did you control for this in some way?
Finally, you give no indication of magnitude of the random error in your measurements. Are the results between the devices really meaningful, or just due to random error? There is no way to tell from your presentation.
Jay
|
|
|
|
|
markanite
Dec 10, 2008, 6:41 PM
Post #13 of 54
(17489 views)
Shortcut
Registered: Jun 30, 2003
Posts: 282
|
Great feedback!
Majid – Regarding the biner being static. Ya I agree. The real problem is creating good solid tests that can be done in a realistic time period / budget. This is why we limited it down to two variables (rope diameter and belay device). Each combination needs to be done 10 times so we can make sure the data is consistent.
So let's say if we decide to add 20 more ropes, 10 weight amounts and 10 other carabiners. Now you're doing (12 belay devices * 10 tests * 23 ropes * 10 carabiners * 10 weights) 276,000 tests. That's why we make lots of the variables static (this test took 11 hours so you can imagine how long 276k tests would take).
I 100% agree. We haven't tested every combination and more than any other sport I've ever done, experience keeps you alive in climbing.
I think one realistic solution is to create independent tests which keep other elements static. For example you could keep the belay device, weight as well as the carabiner static and use different ropes with the same diameter. This would allow you to fairly reasonably determine how similar belay devices handle (or lack thereof) ropes of the same diameter. Therefore, we could use a series of micro tests (50-150 tests a piece) to pin point the variables which actually affect the situation.
Patto – Though yes it's possible -90 degress isn't commonly used. As you can see in their intro video:
http://en.petzl.com/petzl/frontoffice/Sport/static/services/REVERSO/reverso.htm
the rope is being locked off on the climber's side not in front of the climber. This is mainly done to keep the climber's hand far away from the belay device to avoid the common “pinching” problem.
Also as an interesting side note, almost all the belay devices could hold the entire weight if we just allowed the 3 ounce scale to hang at -90 degrees (when the 10 or 11mm rope was used).
JimTitt – Very interesting. I can read German (lived in Hamburg for 5 years) so I'll check it out.
Usnavy - If the device had multiple friction modes, the high friction mode was used. You could argue that for dual friction devices we should have measured the lock-off mode in high friction mode and feed mode in low friction mode. My counter argument is that this setup is impossible during one belay / rappel making it an unrealistic setup.
Jt512 – Merging / simplification of data always brings controversy. I was discussing this a few months ago with a friend who works for consumer report's computer testing department. They give all the computers one “overall value”. The reason why they do it (and why we have chosen to do it as well) is lots of readers prefer simple, straight forward answers. If you had only been climbing for 2 or 3 weeks (when most climber's buy their first belay device) would the raw data at the bottom really be useful? I would argue no. You could counter argue and say they should get a PHD in climbing before touching the rock but this isn't what happens. Our goal is to help climbers at all levels make educated decisions.
Regarding the Trango Jaws performing so well - Our equation gives the lock off force lots of weight, the feed force very little and the device's weight almost nothing. These numbers are optimal for a new climber because feeding the rope extremely fast isn't necessary and an easy lock-off is critical. Experienced climbers who are confident catching falls may be interested in lower friction when feeding rope (so they can aggressively feed and take in slack) and therefore may be willing to trade some lock-off friction for a lower feed friction.
Regarding U-shaped relations: Actually this was the biggest surprise for us as well. The presumption that a belay device will have a linear relationship to rope diameter is actually wrong. This is because different rope diameters “sit” at different spots on the belay device some of which are more ideal than others. Also thicker ropes are more likely to be “pinned” between the biner and the device (at the base) providing lots of additional friction.
Sequential Test argument: We were concerned about this. Therefore, we re-tested the belay device that was tested first after ten other belay devices and received consistent results. The devices were tested in alphabetical order (brand + name).
Random Error: Average range of data between data sets was 0.63 ounces (data sets were ten tests per diameter).
|
|
|
|
|
JimTitt
Dec 10, 2008, 6:48 PM
Post #14 of 54
(17485 views)
Shortcut
Registered: Aug 7, 2008
Posts: 1002
|
That´s o.k. I guessed as much.
It´s all in the details was a play on words- the section "Details" contained what you wanted to know- British humour!!!!!!!!!!!!
|
|
|
|
|
jt512
Dec 10, 2008, 7:10 PM
Post #15 of 54
(17464 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
markanite wrote: Jt512 – Merging / simplification of data always brings controversy. I was discussing this a few months ago with a friend who works for consumer report's computer testing department. They give all the computers one “overall value”. The reason why they do it (and why we have chosen to do it as well) is lots of readers prefer simple, straight forward answers. If you had only been climbing for 2 or 3 weeks (when most climber's buy their first belay device) would the raw data at the bottom really be useful? I would argue no. You could counter argue and say they should get a PHD in climbing before touching the rock but this isn't what happens. Our goal is to help climbers at all levels make educated decisions.
But your combined ratings are meaningless, as I tried to explain. You might as well have skipped the testing entirely, and assigned ratings randomly.
In reply to: Regarding U-shaped relations: Actually this was the biggest surprise for us as well. The presumption that a belay device will have a linear relationship to rope diameter is actually wrong. This is because different rope diameters “sit” at different spots on the belay device some of which are more ideal than others. Also thicker ropes are more likely to be “pinned” between the biner and the device (at the base) providing lots of additional friction.
I don't believe that you can draw those conclusions from your data. I think the U-shaped relation is an artifact of the equation you used to calculate the index.
In reply to: Random Error: Average range of data between data sets was 0.63 ounces (data sets were ten tests per diameter).
That is not a measure of random error.
I will not pursue this further with you, because I believe that you are more interested in computing meaningless indeces than producing meaningful, useful information.
Jay
|
|
|
|
|
shoo
Dec 10, 2008, 7:23 PM
Post #16 of 54
(17443 views)
Shortcut
Registered: Dec 22, 2006
Posts: 1501
|
For the record, I agree with jt512 on this one.
|
|
|
|
|
roy_hinkley_jr
Dec 10, 2008, 7:37 PM
Post #17 of 54
(17433 views)
Shortcut
Registered: Sep 8, 2005
Posts: 652
|
jt512 wrote: But your combined ratings are meaningless, as I tried to explain. You might as well have skipped the testing entirely, and assigned ratings randomly.
That's how all the magazines do it these days. CR is worthless when it gets out of its realm (washing machines, etc).
|
|
|
|
|
ptlong
Dec 10, 2008, 8:14 PM
Post #18 of 54
(17412 views)
Shortcut
Registered: Oct 4, 2007
Posts: 418
|
What exactly is the "weight value" and how was it determined?
Seems like just listing the weight would provide far more "value" to the reader.
|
|
|
|
|
markanite
Dec 11, 2008, 6:07 PM
Post #19 of 54
(17291 views)
Shortcut
Registered: Jun 30, 2003
Posts: 282
|
I see your point ptlong. The logic was to provide everything in a relevant fashion so the user would never need to determine if X grams was relatively light. All it is (11-[weight]/[minimum weight]). That being said I think you are on the right page and we'll modify it to show the original data soon.
Roy,You can argue that all climbing gear is too complex to test and therefore no one should try. That's what consumer reports is told about computers and they get similar feedback to what you said about them often. I'm sure the manufactures would prefer if they just took all the computer stats that the manufactures report as fact. Millions of consumers (similar to new climbers) don't have years of background in computers and rely on complex tests with easy to read results to allow them to make educated purchases.
I'm trying to standardize a way of testing common climbing equipment. I do understand, even if I built a multi-million dollar testing facility, there will always be those who say it's junk and it's impossible. That's fine. The majority of new climbers come into the sport though without any guidance and they are actively look for it.
I wouldn't chat on RC.com if I wasn't willing to get shot at and I'd expect nothing less. We all know RC.com is a contact sport. I truly do greatly appreciate the feedback. One thing I can promise is we're ridiculously motivated. Creating a system that could scan every outdoor retailer (500 retailers, over 12 million price checks per day) on the web with no action from the retailers and help fellow climber's find the best deals with a total budget (at the start) of $10k was an impossible task as well. Now that system is a 6 person company and has been my only job for over 4 years. Point is, I'm very used to impossible tasks.
Jay, no problem. I respect your decision to not pursue my projects any further. I know you probably don't see it this way but the constructive, straight-forward, no “mr. nice guy” feedback like yours is what inspires the next generation of tests. Feedback that says “awesome.” though nice to read really doesn't get us anywhere.
Mark
(This post was edited by markanite on Dec 11, 2008, 6:10 PM)
|
|
|
|
|
jt512
Dec 11, 2008, 6:42 PM
Post #20 of 54
(17276 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
markanite wrote: I know you probably don't see it this way but the constructive, straight-forward, no “mr. nice guy” feedback like yours is what inspires the next generation of tests. Feedback that says “awesome.” though nice to read really doesn't get us anywhere.
Actually, that's exactly how I see it. I admire the fact that you do, too.
Jay
|
|
|
|
|
acorneau
Dec 11, 2008, 7:00 PM
Post #21 of 54
(17263 views)
Shortcut
Registered: Feb 6, 2008
Posts: 2889
|
markanite wrote: Acorneau - any device in particular? I agree narrowing the rope range would be interesting.
Not off the top of my head, but some of the more esoteric devices would be interesting.
In reply to: We settled on three diameters for now to narrow the results to (only) 720 tests.
My suggestion would be to pick some more common rope diameters like a 10.5mm, 9.6mm, and 8.8mm.
In reply to: Sorry, I try to be tough but lifting that weight 720 times in one 11 hour testing session left me sore. Do it 3000+ would have killed me.
You said each result within a set of 10 yielded approximately the same result. Why not reduce the number of tests-per-device to 5 each? It would allow you to test twice as many devices/modes in the same amount of work!
|
|
|
|
|
JimTitt
Dec 11, 2008, 8:11 PM
Post #22 of 54
(17242 views)
Shortcut
Registered: Aug 7, 2008
Posts: 1002
|
Well, I read the test a number of times and was still a bit confused so I have read it some more.
I would disagree with nearly all the test methods used and it is quite apparent they are giving completely false results.
For example the ATC.
Lock-off with a load of 31lbs on an 8mm rope is 6.59lbs.
This gives a friction multiplier of 4.70 which tallies well with that from Black Diamond (5 on an 8.1mm rope).
But for an 11mm rope you give 1.74lbs which is a friction multiplier of 17.8, Black Diamond give ca7.5 for a 10.8 rope.
Of course Black Diamond could have got it wrong so we could look at some other tests. How about this one, performed at the worlds leading climbing equipment laboratory. Using a 9,5mm rope they got friction multiplier of 6.8 for an ATC.
The results for the Reverso³ are even more impressive, 0.6lbs gives a friction multiplier of 51, 5 times higher then anyone has previously reported.
Or have I misintepreted something?
|
|
|
|
|
markanite
Dec 11, 2008, 11:08 PM
Post #23 of 54
(17195 views)
Shortcut
Registered: Jun 30, 2003
Posts: 282
|
Acorneau – Haha. I would love to decrease the number of “reps” but the scientist in me is already upset that I am only doing ten. As far as I am aware we are only missing a few devices including the buggette, a couple spring plates, simond toucan and a couple kong models. The reason why we test "popular" gear (5+ stores) is because the most annoying thing we can do is come out and say "this product is the best but you can't buy it!". But at the same time... I'm with you. The more experimental stuff would be interesting.
JimTitt – Who are you referring to by the “worlds leading equipment laboratory”?
(This post was edited by markanite on Dec 11, 2008, 11:10 PM)
|
|
|
|
|
jt512
Dec 11, 2008, 11:26 PM
Post #24 of 54
(17182 views)
Shortcut
Registered: Apr 12, 2001
Posts: 21904
|
markanite wrote: Acorneau – Haha. I would love to decrease the number of “reps” but the scientist in me is already upset that I am only doing ten.
How do you know whether 5 reps or 10 are necessary? You apparently have not measured the random error in your data.
Jay
|
|
|
|
|
adatesman
Dec 12, 2008, 12:23 AM
Post #25 of 54
(17153 views)
Shortcut
Registered: Jul 13, 2005
Posts: 3479
|
|
|
|
|
|
|
1
