Android OS Forum banner
1 - 3 of 3 Posts

·
Average Android
Joined
·
192 Posts
Discussion Starter · #1 ·
So, I wanted to compare benchmarking tools - I'm running (and loving) imnuts TW4 themed EP1W. I used two kernels - the imnuts ext4 kernel included with the ROM and imoseyon's 2.0.7 ext4 voodoo kernel. This test was not about the kernels. I love both, both perform excellently. This comparison is just about how reliable/meaningful the benchmark numbers are.

NOTE - I ran each test twice and took the highest result for each kernel. This wasn't about which kernel scored highest, it was to show consistency from test to test. Everyone's been reporting incredible Quadrant scores so I wanted to see if this was evident on other benchmarks.

First, the usual Quadrant Standard:

View attachment 2511

Clear difference between the two kernels (around 45%!).

Next, I tried Antutu:

View attachment 2512

Basically equal on this, big difference from Quadrant.

Then on to CF Bench:

View attachment 2513

Again, almost completely equal.

Not trying to burst everyone's Quadrant bubbles, but I think, at least based on this, the Quadrant tests for imoseyon's kernel is returning false scores (which I know has been said a million times, I'm hoping this is just evidence). The magnitude of the difference is too much in my opinion, it's only visible in Quadrant, and it's higher than any scores I could hit overclocked at 1.4Ghz on Froyo.

That said, thanks to all the dev's for their work. I think this is the best my phone has ever run (or looked). I can't wait to see what they'll do once the source is released.
 

Attachments

·
Average Android
Joined
·
192 Posts
Discussion Starter · #4 ·
Birthofahero said:
Interesting ....thoughts?
I think benchmarks are meaningless by themselves, their main use is comparing how different devices run a preset series of tests or comparing changes made to a particular phone (overclocking, ext4 conversion, etc). I'm more concerned with the latter. My main concern with the false numbers that Quadrant is giving is that it'll mask the results of any changes. Let's say imoseyon makes a change to the kernel that actually improves speed but the change results in Quadrant suddenly scoring correctly, the score would still drop, giving the impression of a decrease in performance.

People seemed to be getting hung up on the quadrant scores, and if the scores are all false, it's just misinformation and misleading.
 
1 - 3 of 3 Posts
Top